Conversational agents (CAs) like Amazon’s Alexa and Apple’s Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user’s experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
“I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector,” said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
“It’s extremely unlikely that it (automated empathy) won’t happen,” she said, “so it’s important that as it’s happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms.”
Cuadra will present “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” at CHI ’24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
“She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences,” Estrin said. “Along the way, she observed clear instances of the tension between compelling and disturbing ‘empathy.’”
Reference: “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI ’24.
DOI: 10.1145/3613904.3642336
News
New book from NanoappsMedical Inc – Molecular Manufacturing: The Future of Nanomedicine
This book explores the revolutionary potential of atomically precise manufacturing technologies to transform global healthcare, as well as practically every other sector across society. This forward-thinking volume examines how envisaged Factory@Home systems might enable the cost-effective [...]
A Virus Designed in the Lab Could Help Defeat Antibiotic Resistance
Scientists can now design bacteria-killing viruses from DNA, opening a faster path to fighting superbugs. Bacteriophages have been used as treatments for bacterial infections for more than a century. Interest in these viruses is rising [...]
Sleep Deprivation Triggers a Strange Brain Cleanup
When you don’t sleep enough, your brain may clean itself at the exact moment you need it to think. Most people recognize the sensation. After a night of inadequate sleep, staying focused becomes harder [...]
Lab-grown corticospinal neurons offer new models for ALS and spinal injuries
Researchers have developed a way to grow a highly specialized subset of brain nerve cells that are involved in motor neuron disease and damaged in spinal injuries. Their study, published today in eLife as the final [...]
Urgent warning over deadly ‘brain swelling’ virus amid fears it could spread globally
Airports across Asia have been put on high alert after India confirmed two cases of the deadly Nipah virus in the state of West Bengal over the past month. Thailand, Nepal and Vietnam are among the [...]
This Vaccine Stops Bird Flu Before It Reaches the Lungs
A new nasal spray vaccine could stop bird flu at the door — blocking infection, reducing spread, and helping head off the next pandemic. Since first appearing in the United States in 2014, H5N1 [...]
These two viruses may become the next public health threats, scientists say
Two emerging pathogens with animal origins—influenza D virus and canine coronavirus—have so far been quietly flying under the radar, but researchers warn conditions are ripe for the viruses to spread more widely among humans. [...]
COVID-19 viral fragments shown to target and kill specific immune cells
COVID-19 viral fragments shown to target and kill specific immune cells in UCLA-led study Clues about extreme cases and omicron’s effects come from a cross-disciplinary international research team New research shows that after the [...]
Smaller Than a Grain of Salt: Engineers Create the World’s Tiniest Wireless Brain Implant
A salt-grain-sized neural implant can record and transmit brain activity wirelessly for extended periods. Researchers at Cornell University, working with collaborators, have created an extremely small neural implant that can sit on a grain of [...]
Scientists Develop a New Way To See Inside the Human Body Using 3D Color Imaging
A newly developed imaging method blends ultrasound and photoacoustics to capture both tissue structure and blood-vessel function in 3D. By blending two powerful imaging methods, researchers from Caltech and USC have developed a new way to [...]
Brain waves could help paralyzed patients move again
People with spinal cord injuries often lose the ability to move their arms or legs. In many cases, the nerves in the limbs remain healthy, and the brain continues to function normally. The loss of [...]
Scientists Discover a New “Cleanup Hub” Inside the Human Brain
A newly identified lymphatic drainage pathway along the middle meningeal artery reveals how the human brain clears waste. How does the brain clear away waste? This task is handled by the brain’s lymphatic drainage [...]
New Drug Slashes Dangerous Blood Fats by Nearly 40% in First Human Trial
Scientists have found a way to fine-tune a central fat-control pathway in the liver, reducing harmful blood triglycerides while preserving beneficial cholesterol functions. When we eat, the body turns surplus calories into molecules called [...]
A Simple Brain Scan May Help Restore Movement After Paralysis
A brain cap and smart algorithms may one day help paralyzed patients turn thought into movement—no surgery required. People with spinal cord injuries often experience partial or complete loss of movement in their arms [...]
Plant Discovery Could Transform How Medicines Are Made
Scientists have uncovered an unexpected way plants make powerful chemicals, revealing hidden biological connections that could transform how medicines are discovered and produced. Plants produce protective chemicals called alkaloids as part of their natural [...]
Scientists Develop IV Therapy That Repairs the Brain After Stroke
New nanomaterial passes the blood-brain barrier to reduce damaging inflammation after the most common form of stroke. When someone experiences a stroke, doctors must quickly restore blood flow to the brain to prevent death. [...]
