Conversational agents (CAs) like Amazon’s Alexa and Apple’s Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user’s experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
“I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector,” said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
“It’s extremely unlikely that it (automated empathy) won’t happen,” she said, “so it’s important that as it’s happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms.”
Cuadra will present “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” at CHI ’24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
“She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences,” Estrin said. “Along the way, she observed clear instances of the tension between compelling and disturbing ‘empathy.’”
Reference: “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI ’24.
DOI: 10.1145/3613904.3642336
News
Scientists discover natural ‘brake’ that could stop harmful inflammation
Researchers at University College London (UCL) have uncovered a key mechanism that helps the body switch off inflammation—a breakthrough that could lead to new treatments for chronic diseases affecting millions worldwide. Inflammation is the [...]
A Forgotten Molecule Could Revive Failing Antifungal Drugs and Save Millions of Lives
Scientists have uncovered a way to make existing antifungal drugs work again against deadly, drug-resistant fungi. Fungal infections claim millions of lives worldwide each year, and current medical treatments are failing to keep pace. [...]
Scientists Trap Thyme’s Healing Power in Tiny Capsules
A new micro-encapsulation breakthrough could turn thyme’s powerful health benefits into safer, smarter nanodoses. Thyme extract is often praised for its wide range of health benefits, giving it a reputation as a natural medicinal [...]
Scientists Develop Spray-On Powder That Instantly Seals Life-Threatening Wounds
KAIST scientists have created a fast-acting, stable powder hemostat that stops bleeding in one second and could significantly improve survival in combat and emergency medicine. Severe blood loss remains the primary cause of death from [...]
Oceans Are Struggling To Absorb Carbon As Microplastics Flood Their Waters
New research points to an unexpected way plastic pollution may be influencing Earth’s climate system. A recent study suggests that microscopic plastic pollution is reducing the ocean’s capacity to take in carbon dioxide, a [...]
Molecular Manufacturing: The Future of Nanomedicine – New book from Frank Boehm
This book explores the revolutionary potential of atomically precise manufacturing technologies to transform global healthcare, as well as practically every other sector across society. This forward-thinking volume examines how envisaged Factory@Home systems might enable the cost-effective [...]
New Book! NanoMedical Brain/Cloud Interface – Explorations and Implications
New book from Frank Boehm, NanoappsMedical Inc Founder: This book explores the future hypothetical possibility that the cerebral cortex of the human brain might be seamlessly, safely, and securely connected with the Cloud via [...]
Global Health Care Equivalency in the Age of Nanotechnology, Nanomedicine and Artificial Intelligence
A new book by Frank Boehm, NanoappsMedical Inc. Founder. This groundbreaking volume explores the vision of a Global Health Care Equivalency (GHCE) system powered by artificial intelligence and quantum computing technologies, operating on secure [...]
Miller School Researchers Pioneer Nanovanilloid-Based Brain Cooling for Traumatic Injury
A multidisciplinary team at the University of Miami Miller School of Medicine has developed a breakthrough nanodrug platform that may prove beneficial for rapid, targeted therapeutic hypothermia after traumatic brain injury (TBI). Their work, published in ACS [...]
COVID-19 still claims more than 100,000 US lives each year
Centers for Disease Control and Prevention researchers report national estimates of 43.6 million COVID-19-associated illnesses and 101,300 deaths in the US during October 2022 to September 2023, plus 33.0 million illnesses and 100,800 deaths [...]
Nanomedicine in 2026: Experts Predict the Year Ahead
Progress in nanomedicine is almost as fast as the science is small. Over the last year, we've seen an abundance of headlines covering medical R&D at the nanoscale: polymer-coated nanoparticles targeting ovarian cancer, Albumin recruiting nanoparticles for [...]
Lipid nanoparticles could unlock access for millions of autoimmune patients
Capstan Therapeutics scientists demonstrate that lipid nanoparticles can engineer CAR T cells within the body without laboratory cell manufacturing and ex vivo expansion. The method using targeted lipid nanoparticles (tLNPs) is designed to deliver [...]
The Brain’s Strange Way of Computing Could Explain Consciousness
Consciousness may emerge not from code, but from the way living brains physically compute. Discussions about consciousness often stall between two deeply rooted viewpoints. One is computational functionalism, which holds that cognition can be [...]
First breathing ‘lung-on-chip’ developed using genetically identical cells
Researchers at the Francis Crick Institute and AlveoliX have developed the first human lung-on-chip model using stem cells taken from only one person. These chips simulate breathing motions and lung disease in an individual, [...]
Cell Membranes May Act Like Tiny Power Generators
Living cells may generate electricity through the natural motion of their membranes. These fast electrical signals could play a role in how cells communicate and sense their surroundings. Scientists have proposed a new theoretical [...]
This Viral RNA Structure Could Lead to a Universal Antiviral Drug
Researchers identify a shared RNA-protein interaction that could lead to broad-spectrum antiviral treatments for enteroviruses. A new study from the University of Maryland, Baltimore County (UMBC), published in Nature Communications, explains how enteroviruses begin reproducing [...]
