Conversational agents (CAs) like Amazon’s Alexa and Apple’s Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user’s experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
“I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector,” said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
“It’s extremely unlikely that it (automated empathy) won’t happen,” she said, “so it’s important that as it’s happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms.”
Cuadra will present “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” at CHI ’24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
“She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences,” Estrin said. “Along the way, she observed clear instances of the tension between compelling and disturbing ‘empathy.’”
Reference: “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI ’24.
DOI: 10.1145/3613904.3642336
News
NIH Scientists Discover Gene Responsible for Rare Eye Disease
Findings supported by the NIH pave the way for the development of genetic testing, clinical trials, and therapies. Researchers at the National Institutes of Health (NIH) and their collaborators have discovered a gene linked to certain [...]
Alzheimer’s Breakthrough: Synthetic THC Pill Proves Effective in Clinical Trial
Patients tolerated synthetic THC (dronabinol) well, without the adverse effects commonly associated with existing Alzheimer’s agitation medications. A study conducted by researchers from Johns Hopkins University School of Medicine and Tufts University School of Medicine found that a pill form [...]
The Future of Rare Disease Treatment with Precision Medicine
Understanding rare diseases Rare diseases affect less than 5 people out of 10,000. However, this still amounts to about 7% of the world’s population, with over 10,000 such conditions. Almost all are genetic in [...]
Doctors issue warning for upcoming ‘tripledemic
The term ‘tripledemic’ has hit headlines this week as the NHS begins its Covid and fluvaccine roll-out for vulnerable adults. As the cold weather sets in, many of us have experienced a decline in health, and this may [...]
The FDA approved a gel that can stop bleeding from wounds in seconds
Aug 15 (Reuters) - The U.S. Food and Drug Administration has cleared Cresilon's gel to quickly control bleeding, the privately held company said on Thursday, potentially giving emergency medical technicians and combat medics a [...]
High levels of microplastics found in prostate tumors, possibly linked to take-out food
The presence of microplastics in prostate tumors points to potential health risks, and researchers are calling for urgent studies to explore how take-out food may be driving this exposure. In a recent study published [...]
AI outperforms radiologists in brain tumor diagnosis
As artificial intelligence advances, its uses and capabilities in real-world applications continue to reach new heights that may even surpass human expertise. In the field of radiology, where a correct diagnosis is crucial to ensure [...]
Breakthrough Study Reveals Molecular Clues to Dementia Origins
Work could lead to the discovery of new therapeutic targets. For the first time, researchers have identified “molecular markers” linked to degeneration—detectable changes in cells and their gene-regulating networks—that are common across several types [...]
Better than blood tests? Nanoparticle potential found for assessing kidneys
In a study published July 29 in Advanced Materials, University of Texas at Dallas researchers found that X-rays of the kidneys using gold nanoparticles as a contrast agent might be more accurate in detecting kidney [...]
Greener nanomaterials could transform how our everyday stuff is made
Tiny nanoparticles are at the forefront of materials science—with special properties that make them great at absorbing light in solar panels, cleaning wastewater, and delivering drugs precisely. Some nanoparticles take the form of sheets or fibers. But nanomaterials all [...]
AI could predict breast cancer risk via ‘zombie cells’
Women worldwide could see better treatment with new AI technology, which enables better detection of damaged cells and more precisely predicts the risk of getting breast cancer, shows new research from the [...]
Through the eyes of a cat – biomimicry of feline eyes may revolutionize robotic vision
In a recent study published in the journal Science Advances, researchers leveraged crucial aspects of feline eyes, particularly their tapetum lucidum and vertically elongated pupils (VP), to develop a monocular artificial vision system capable of [...]
New Alzheimer’s Therapy Shows Remarkable Results in Animal Trials
A study from TUM demonstrates a promising therapeutic approach. Researchers at the Technical University of Munich (TUM) have made promising advances in preventing Alzheimer’s by developing a new therapeutic strategy. Their approach focuses on targeting the amyloid beta [...]
Rewriting Cancer’s Blueprint: New Study Challenges Old Theories
A new study argues for a revised clonal evolution model of cancer, incorporating genetic and non-genetic factors to improve understanding and treatment. Like all living organisms, cancer cells are driven by the fundamental need [...]
Microplastics Everywhere: Experts Demand Worldwide Treaty Before It’s Too Late
A new report calls for global action on plastic pollution, urging reductions in plastic production and microplastic emissions. Researchers stress the importance of addressing plastic pollution through both scientific and social science perspectives. A [...]
Blood tests could soon predict your risk of Alzheimer’s
Scientists are closing in on biomarkers that reflect the progression of Alzheimer’s disease and could improve treatments. Like many Alzheimer’s researchers, neurologist Randall Bateman is not prone to effusiveness, having endured disappointments in his [...]
