Conversational agents (CAs) like Amazon's Alexa and Apple's Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user's experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
"I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector," said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
"It's extremely unlikely that it (automated empathy) won't happen," she said, "so it's important that as it's happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms."
Cuadra will present "The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction" at CHI '24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
"She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences," Estrin said. "Along the way, she observed clear instances of the tension between compelling and disturbing 'empathy.'"
Reference: "The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction" by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI '24.
DOI: 10.1145/3613904.3642336
News
Study Finds 95% of Tested Beers Contain Toxic “Forever Chemicals”
Researchers found PFAS in 95% of tested beers, with the highest levels linked to contaminated local water sources. Per- and polyfluoroalkyl substances (PFAS), better known as forever chemicals, are gaining notoriety for their ability [...]
Long COVID Symptoms Are Closer To A Stroke Or Parkinson’s Disease Than Fatigue
When most people get sick with COVID-19 today, they think of it as a brief illness, similar to a cold. However, for a large number of people, the illness doesn't end there. The World [...]
The world’s first AI Hospital, developed in China is transforming healthcare
Artificial Intelligence and its developments have had a revolutionary impact on society, and healthcare is not an exception. China has made massive strides in AI integrated healthcare, and continues to do so as AI [...]
Scientists Rewire Immune Cells To Supercharge Cancer-Fighting Power
Blocking a single protein boosts T cell metabolism and tumor-fighting strength. The discovery could lead to next-generation cancer immunotherapies. Scientists have identified a strategy to greatly enhance the cancer-fighting abilities of the immune system’s [...]
Scientists Discover 20 Percent of Human DNA Comes from a Mysterious Ancestor
Humans carry a complex genetic history that continues to reveal surprises. Scientists have found that 20% of our DNA may come from a mysterious ancestor, according to WP Tech. This discovery changes how we understand [...]
AI detects early prostate cancer missed by pathologists
Men assessed as healthy after a pathologist analyses their tissue sample may still have an early form of prostate cancer. Using AI, researchers at Uppsala University have been able to find subtle tissue changes [...]
The Rare Mutation That Makes People Immune to Viruses
Some people carry a rare mutation that makes them resistant to viruses. Now scientists have copied that effect with an experimental mRNA therapy that stopped both flu and COVID in animal trials — raising [...]
Nanopore technique for measuring DNA damage could improve cancer therapy and radiological emergency response
Scientists at the National Institute of Standards and Technology (NIST) have developed a new technology for measuring how radiation damages DNA molecules. This novel technique, which passes DNA through tiny openings called nanopores, detects [...]
AI Tool Shows Exactly When Genes Turn On and Off
Summary: Researchers have developed an AI-powered tool called chronODE that models how genes turn on and off during brain development. By combining mathematics, machine learning, and genomic data, the method identifies exact “switching points” that [...]
Your brain could get bigger – not smaller – as you age
recently asked myself if I’ll still have a healthy brain as I get older. I hold a professorship at a neurology department. Nevertheless, it is difficult for me to judge if a particular brain, [...]
Hidden Cost of Smart AI: 50× More CO₂ for a Single Question
Every time we ask an AI a question, it doesn’t just return an answer—it also burns energy and emits carbon dioxide. German researchers found that some “thinking” AI models, which generate long, step-by-step reasoning [...]
Genetically-engineered immune cells show promise for preventing organ rejection
A Medical University of South Carolina team reports in Frontiers in Immunology that it has engineered a new type of genetically modified immune cell that can precisely target and neutralize antibody-producing cells complicit in organ rejection. [...]
Building and breaking plastics with light: Chemists rethink plastic recycling
What if recycling plastics were as simple as flicking a switch? At TU/e, Assistant Professor Fabian Eisenreich is making that vision a reality by using LED light to both create and break down a [...]
Generative AI Designs Novel Antibiotics That Defeat Defiant Drug-Resistant Superbugs
Harnessing generative AI, MIT scientists have created groundbreaking antibiotics with unique membrane-targeting mechanisms, offering fresh hope against two of the world’s most formidable drug-resistant pathogens. With the help of artificial intelligence, MIT researchers have [...]
AI finds more breast tumors earlier than traditional double radiologist review
AI is detecting tumors more often and earlier in the Dutch breast cancer screening program. Those tumors can then be treated at an earlier stage. This has been demonstrated by researchers led by Radboud [...]
Lavender oil could speed recovery after brain surgery
A week of lavender-scented nights helped brain surgery patients sleep more deeply, shorten delirium, and feel calmer, pointing to a simple, natural aid for post-surgery care. A randomized controlled trial investigating the therapeutic impact [...]
