One question has relentlessly followed ChatGPT in its trajectory to superstar status in the field of artificial intelligence: Has it met the Turing test of generating output indistinguishable from human response?
ChatGPT may be smart, quick and impressive. It does a good job at exhibiting apparent intelligence. It sounds humanlike in conversations with people and can even display humor, emulate the phraseology of teenagers, and pass exams for law school.
But on occasion, it has been found to serve up totally false information. It hallucinates. It does not reflect on its own output.
Cameron Jones, who specializes in language, semantics and machine learning, and Benjamin Bergen, professor of cognitive science, drew upon the work of Alan Turing, who 70 years ago devised a process to determine whether a machine could reach a point of intelligence and conversational prowess at which it could fool someone into thinking it was human.
Their report titled “Does GPT-4 Pass the Turing Test?” is available on the arXiv preprint server.
They rounded up 650 participants and generated 1,400 “games” in which brief conversations were conducted between participants and either another human or a GPT model. Participants were asked to determine who they were conversing with.
The researchers found that GPT-4 models fooled participants 41% of the time, while GPT-3.5 fooled them only 5% to 14% of the time. Interestingly, humans succeeded in convincing participants they were not machines in only 63% of the trials.
The researchers concluded, “We do not find evidence that GPT-4 passes the Turing Test.”
They noted, however, that the Turing test still retains value as a measure of the effectiveness of machine dialogue.
“The test has ongoing relevance as a framework to measure fluent social interaction and deception, and for understanding human strategies to adapt to these devices,” they said.
They warned that in many instances, chatbots can still communicate convincingly enough to fool users in many instances.
“A success rate of 41% suggests that deception by AI models may already be likely, especially in contexts where human interlocutors are less alert to the possibility they are not speaking to a human,” they said. “AI models that can robustly impersonate people could have could have widespread social and economic consequences.”
The researchers observed that participants making correct identifications focused on several factors.
Models that were too formal or too informal raised red flags for participants. If they were too wordy or too brief, if their grammar or use of punctuation was exceptionally good or “unconvincingly” bad, their usage became key factors in determining whether participants were dealing with humans or machines.
Test takers also were sensitive to generic-sounding responses.
“LLMs learn to produce highly likely completions and are fine-tuned to avoid controversial opinions. These processes might encourage generic responses that are typical overall, but lack the idiosyncrasy typical of an individual: a sort of ecological fallacy,” the researchers said.
The researchers have suggested that it will be important to track AI models as they gain more fluidity and absorb more humanlike quirks in conversation.
“It will become increasingly important to identify factors that lead to deception and strategies to mitigate it,” they said.
News
Current Heart Health Guidelines Are Failing To Catch a Deadly Genetic Killer
New research reveals that standard screening misses most people with a common inherited cholesterol disorder. A Mayo Clinic study reports that current genetic screening guidelines overlook most people who have familial hypercholesterolemia, an inherited disorder that [...]
Scientists Identify the Evolutionary “Purpose” of Consciousness
Summary: Researchers at Ruhr University Bochum explore why consciousness evolved and why different species developed it in distinct ways. By comparing humans with birds, they show that complex awareness may arise through different neural architectures yet [...]
Novel mRNA therapy curbs antibiotic-resistant infections in preclinical lung models
Researchers at the Icahn School of Medicine at Mount Sinai and collaborators have reported early success with a novel mRNA-based therapy designed to combat antibiotic-resistant bacteria. The findings, published in Nature Biotechnology, show that in [...]
New skin-permeable polymer delivers insulin without needles
A breakthrough zwitterionic polymer slips through the skin’s toughest barriers, carrying insulin deep into tissue and normalizing blood sugar, offering patients a painless alternative to daily injections. A recent study published in the journal Nature examines [...]
Multifunctional Nanogels: A Breakthrough in Antibacterial Strategies
Antibiotic resistance is a growing concern - from human health to crop survival. A new study successfully uses nanogels to target and almost entirely inhibit the bacteria P. Aeruginosa. Recently published in Angewandte Chemie, the study [...]
Nanoflowers rejuvenate old and damaged human cells by replacing their mitochondria
Biomedical researchers at Texas A&M University may have discovered a way to stop or even reverse the decline of cellular energy production—a finding that could have revolutionary effects across medicine. Dr. Akhilesh K. Gaharwar [...]
The Stunning New Push to Protect the Invisible 99% of Life
Scientists worldwide have joined forces to build the first-ever roadmap for conserving Earth’s vast invisible majority—microbes. Their new IUCN Specialist Group reframes conservation by elevating microbial life to the same urgency as plants and [...]
Scientists Find a Way to Help the Brain Clear Alzheimer’s Plaques Naturally
Scientists have discovered that the brain may have a built-in way to fight Alzheimer’s. By activating a protein called Sox9, researchers were able to switch on star-shaped brain cells known as astrocytes and turn them into [...]
Vision can be rebooted in adults with amblyopia, study suggests
Temporarily anesthetizing the retina briefly reverts the activity of the visual system to that observed in early development and enables growth of responses to the amblyopic eye, new research shows. In the common vision [...]
Ultrasound-activated Nanoparticles Kill Liver Cancer and Activate Immune System
A new ultrasound-guided nanotherapy wipes out liver tumors while training the immune system to keep them from coming back. The study, published in Nano Today, introduces a biodegradable nanoparticle system that combines sonodynamic therapy and cell [...]
Magnetic nanoparticles that successfully navigate complex blood vessels may be ready for clinical trials
Every year, 12 million people worldwide suffer a stroke; many die or are permanently impaired. Currently, drugs are administered to dissolve the thrombus that blocks the blood vessel. These drugs spread throughout the entire [...]
Reviving Exhausted T Cells Sparks Powerful Cancer Tumor Elimination
Scientists have discovered how tumors secretly drain the energy from T cells—the immune system’s main cancer fighters—and how blocking that process can bring them back to life. The team found that cancer cells use [...]
Very low LDL-cholesterol correlates to fewer heart problems after stroke
Brigham and Women's Hospital's TIMI Study Group reports that in patients with prior ischemic stroke, very low achieved LDL-cholesterol correlated with fewer major adverse cardiovascular events and fewer recurrent strokes, without an apparent increase [...]
“Great Unified Microscope” Reveals Hidden Micro and Nano Worlds Inside Living Cells
University of Tokyo researchers have created a powerful new microscope that captures both forward- and back-scattered light at once, letting scientists see everything from large cell structures to tiny nanoscale particles in a single shot. Researchers [...]
Breakthrough Alzheimer’s Drug Has a Hidden Problem
Researchers in Japan found that although the Alzheimer’s drug lecanemab successfully removes amyloid plaques from the brain, it does not restore the brain’s waste-clearing system within the first few months of treatment. The study suggests that [...]
Concerning New Research Reveals Colon Cancer Is Skyrocketing in Adults Under 50
Colorectal cancer is striking younger adults at alarming rates, driven by lifestyle and genetic factors. Colorectal cancer (CRC) develops when abnormal cells grow uncontrollably in the colon or rectum, forming tumors that can eventually [...]
