Conversational agents (CAs) like Amazon’s Alexa and Apple’s Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user’s experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
“I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector,” said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
“It’s extremely unlikely that it (automated empathy) won’t happen,” she said, “so it’s important that as it’s happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms.”
Cuadra will present “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” at CHI ’24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
“She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences,” Estrin said. “Along the way, she observed clear instances of the tension between compelling and disturbing ‘empathy.’”
Reference: “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI ’24.
DOI: 10.1145/3613904.3642336
News
Your Voice May Reveal How Fast Your Brain Is Aging
Researchers have developed a machine-learning “speech clock” that estimates age from subtle patterns in how people speak and what they say. It is possible that people’s voices may contain certain signals of aging that [...]
Optogenetics Pioneers Win 2026 Nobel Prize in Physiology or Medicine
The 2026 Nobel Prize in Physiology or Medicine has been awarded to Peter Hegemann, Georg Nagel, and Karl Deisseroth for the development of optogenetics, a revolutionary technology that uses light to control individual neurons [...]
Molecular Manufacturing: The Future of Nanomedicine – From NanoappsMedical Inc.
This book explores the revolutionary potential of atomically precise manufacturing technologies to transform global healthcare, as well as practically every other sector across society. This forward-thinking volume examines how envisaged Factory@Home systems might enable the cost-effective [...]
What is ‘American Covid’ and how bad is it?
Cases of so-called ‘American Covid’ are on the rise in the UK. The strain is officially called XFG, or ‘Stratus’, and is the most common variant in North America, hence the name. It might soon [...]
Researcher at Russian plague laboratory dies of ‘unknown’ infection
Russian authorities have imposed "anti-epidemic measures" after a laboratory worker at an institute in Siberia dedicated to studying the plague and other infectious diseases died of a mysterious illness. The researcher, a 28-year-old female [...]
GHCE Concept
From the preface of the book Global Health Care Equivalency in the Age of Nanotechnology, Nanomedicine and Artificial Intelligence, Edited by Frank Boehm: Since the publication of my first book (Nanomedical Device and Systems [...]
A Surprising Brain Signal Could Explain Why Eating Less Protein Extends Lifespan
A new paper suggests that limiting protein intake triggers a coordinated response throughout the body that may help explain how protein restriction affects healthy aging. Animals respond to protein restriction, or reduced protein in [...]
Back and shoulder surgery is often worse than useless
Millions of operations should be scrapped SIX IN TEN Britons have surgery at some point. So do lots of Americans, at a cost of perhaps $500bn a year, a third of all spending on [...]
Cocaine plays with mind, hijacks brain circuit, drives increasingly rigid and repetitive behavior
Researchers have found that cocaine hijacks brain circuit, which to drive increasingly rigid and repetitive behavior. The brain circuit that normally participates in selecting natural mouth and face movements, but which becomes disproportionately engaged [...]
Injectable nanodevices could provide effective treatment for drug-resistant glioblastoma
The brain cancer glioblastoma is one of the most aggressive and treatment-resistant cancers known to medicine, carrying a median survival of just 12-15 months, even with the best available care. Now, researchers at the [...]
New Treatment Cuts “Bad” Cholesterol in Half for a Full Year
A single infusion of an experimental CRISPR-Cas9 therapy was reported to safely lower LDL cholesterol by 52.5% and triglycerides by 47.8%, as measured 12 months after treatment. What if one infusion could keep cholesterol [...]
Too Much RNA Can Drain a Cell’s Energy, Scientists Discover
A virus may cripple a cell’s power supply simply by producing too much of a molecule the cell needs to survive. New research from the Texas A&M College of Veterinary Medicine and Biomedical Sciences [...]
West African scientists warn of weakening health systems and urge stronger outbreak detection
LOME, Togo (AP) — West African laboratory scientists called on governments Friday to strengthen capacity to detect disease outbreaks to avoid another major outbreak, saying health systems across the region are becoming weaker. Participants [...]
A Little-Known Protein Could Point to a New Way To Treat Alzheimer’s
A study suggests that increasing SORLA protein levels could help treat Alzheimer’s disease and other disorders involving tau protein. Increasing SORLA, a protein with a protective role in the brain, helped mice withstand damage [...]
Cancer’s Hidden Antioxidant Shield Helps It Escape the Immune System
Blocking an antioxidant protein that tumors use to suppress immune attacks improved cancer immunotherapy responses in mice. Cancer cells can release antioxidants that interfere with the immune cells trying to kill them. Researchers have [...]
Scientists Find a Berry Compound That Helps Muscle Cells Burn Fat
Pterostilbene, a compound found naturally in some foods, affects how skeletal muscles process fats by stabilizing a protein called PPARδ and increasing its signaling activity. Pterostilbene, a natural compound found in blueberries, grapes, and [...]
