Conversational agents (CAs) like Amazon’s Alexa and Apple’s Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user’s experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
“I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector,” said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
“It’s extremely unlikely that it (automated empathy) won’t happen,” she said, “so it’s important that as it’s happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms.”
Cuadra will present “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” at CHI ’24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
“She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences,” Estrin said. “Along the way, she observed clear instances of the tension between compelling and disturbing ‘empathy.’”
Reference: “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI ’24.
DOI: 10.1145/3613904.3642336
![](https://www.nanoappsmedical.com/wp-content/uploads/2017/05/spacer.jpg)
News
Scientists May Have Discovered the First Sign of Autism: An Unusually Large Brain
Researchers at the University of California San Diego discovered that an unusually large brain could be the first sign of autism, potentially detectable in as early as the first trimester. Some children with autism [...]
$100M Funding – Phase 3 trial of first oral disease-modifying therapy for Alzheimer’s.
Clinical-stage biotech Alzheon has completed a $100 million Series E financing round to advance the development and commercialization of its oral drug for Alzheimer’s disease. The drug, called ALZ-801 (valiltramiprosate), is designed to inhibit the formation [...]
‘Dual mutant’ seasonal flu virus could make some treatments ineffective
Two human cases of "dual mutant" strains of H1N1 flu have been reported by U.S. health officials. Unfortunately, the genetic changes appear to render the leading flu antiviral, Tamiflu, less effective, researchers from the U.S. Centers [...]
Mutation Mystery Solved: Why Lung Cancer Treatments Often Fail in Non-Smokers
Researchers have discovered the reason why targeted treatment for non-small cell lung cancer fails to work for some patients, particularly those who have never smoked. The study shows that lung cancer cells with two [...]
Innovative blood test boosts accuracy of lung cancer screening
In a recent study published in Cancer Discovery, researchers developed and validated a blood-based, cell-free deoxyribonucleic acid (cfDNA) fragmentome assay for lung cancer detection, which, if the results were positive, would be followed by low-dose [...]
Researchers Uncover Significant Cognitive Differences Between Male and Female Brains
Research from Weill Cornell Medicine reveals that astrocyte receptors impact cognitive functions differently in males and females, suggesting a need for sex-specific approaches in developing treatments targeting these brain cells. Scientists at Weill Cornell Medicine have [...]
Columbia Scientists Unravel a 15-Year Mistake in Stem Cell Research
Columbia University’s research has uncovered a longstanding error in identifying gut stem cells, finding the true stem cells in a different site, which could revolutionize regenerative medicine by applying these findings to other organs. [...]
COVID-19 Triggers Lethal Lung Disease in Previously Unaffected Populations
Researchers discovered that during the COVID-19 pandemic, there was a surge in a rare autoimmune disease among previously unaffected demographic groups, leading to the deaths of nine patients. The disease is an autoimmune response where the [...]
Spicing Up Pathogen Defense: Scientists Transform Cinnamon Into Nanotech Antimicrobials
Cinnamon oil-based nanodevice effectively targets key pathogens, with potential uses in healthcare and food safety A team of researchers from the Universitat Politècnica de València (UPV) and the CIBER de Bioingeniería, Biomaterials y Nanomedicine [...]
High Excess Death Rates 3 Years Running Since COVID – “Serious Cause for Concern”
Despite containment and COVID-19 vaccines; “serious cause for concern,” say researchers. A study highlights persistent high excess death rates in the West from 2020 to 2022, despite COVID-19 measures and vaccines. Over three million excess deaths [...]
Super-resolution microscopy provides a nano-scale look
A new, nano-scale look at how the SARS-CoV-2 virus replicates in cells may offer greater precision in drug development, a Stanford University team reports in Nature Communications. Using advanced microscopy techniques, the researchers produced what might be [...]
Nature’s Painkiller: Natural Molecules Found in Cannabis Rival Morphine in Groundbreaking Study
Researchers at the Comprehensive Center for Pain & Addiction conducted tests on five terpenes, finding promising results in their pain-relief capabilities. A study from the University of Arizona Health Sciences, published in the journal PAIN, discovered [...]
The Ultimate Killer: Pollution Deadlier Than War, Terrorism, and Major Diseases
Manmade pollutants and climate change contribute to millions of deaths from cardiovascular disease each year, warn a coalition of leading scientists. A new series published today (June 3) in the Journal of the American College of Cardiology highlights [...]
Scientists Race To Perfect New Oral COVID-19 Treatment Poised To Replace Paxlovid
Rutgers researchers are advancing a potential new COVID-19 treatment, Jun12682, which is effective in animal studies and compatible with other medications, unlike the current leading treatment, Paxlovid. Researchers at Rutgers believe they are among the lead [...]
Penn Medicine Develops Potent mRNA Vaccine Against Lethal H5N1 Virus
Experimental mRNA vaccine proves highly effective against H5N1 in animal studies, offering hope for rapid pandemic response. Researchers from the Perelman School of Medicine at the University of Pennsylvania have developed an experimental mRNA [...]
The Future of Contraception: Non-Hormonal Birth Control Pill for Men
Baylor researchers developed a reversible, non-hormonal male contraceptive targeting the sperm-specific protein STK33, showing effectiveness and safety in mice studies. In the past six decades, the global population has surged more than 260%, and [...]