One question has relentlessly followed ChatGPT in its trajectory to superstar status in the field of artificial intelligence: Has it met the Turing test of generating output indistinguishable from human response?
ChatGPT may be smart, quick and impressive. It does a good job at exhibiting apparent intelligence. It sounds humanlike in conversations with people and can even display humor, emulate the phraseology of teenagers, and pass exams for law school.
But on occasion, it has been found to serve up totally false information. It hallucinates. It does not reflect on its own output.
Cameron Jones, who specializes in language, semantics and machine learning, and Benjamin Bergen, professor of cognitive science, drew upon the work of Alan Turing, who 70 years ago devised a process to determine whether a machine could reach a point of intelligence and conversational prowess at which it could fool someone into thinking it was human.
Their report titled “Does GPT-4 Pass the Turing Test?” is available on the arXiv preprint server.
They rounded up 650 participants and generated 1,400 “games” in which brief conversations were conducted between participants and either another human or a GPT model. Participants were asked to determine who they were conversing with.
The researchers found that GPT-4 models fooled participants 41% of the time, while GPT-3.5 fooled them only 5% to 14% of the time. Interestingly, humans succeeded in convincing participants they were not machines in only 63% of the trials.
The researchers concluded, “We do not find evidence that GPT-4 passes the Turing Test.”
They noted, however, that the Turing test still retains value as a measure of the effectiveness of machine dialogue.
“The test has ongoing relevance as a framework to measure fluent social interaction and deception, and for understanding human strategies to adapt to these devices,” they said.
They warned that in many instances, chatbots can still communicate convincingly enough to fool users in many instances.
“A success rate of 41% suggests that deception by AI models may already be likely, especially in contexts where human interlocutors are less alert to the possibility they are not speaking to a human,” they said. “AI models that can robustly impersonate people could have could have widespread social and economic consequences.”
The researchers observed that participants making correct identifications focused on several factors.
Models that were too formal or too informal raised red flags for participants. If they were too wordy or too brief, if their grammar or use of punctuation was exceptionally good or “unconvincingly” bad, their usage became key factors in determining whether participants were dealing with humans or machines.
Test takers also were sensitive to generic-sounding responses.
“LLMs learn to produce highly likely completions and are fine-tuned to avoid controversial opinions. These processes might encourage generic responses that are typical overall, but lack the idiosyncrasy typical of an individual: a sort of ecological fallacy,” the researchers said.
The researchers have suggested that it will be important to track AI models as they gain more fluidity and absorb more humanlike quirks in conversation.
“It will become increasingly important to identify factors that lead to deception and strategies to mitigate it,” they said.
![](https://www.nanoappsmedical.com/wp-content/uploads/2017/05/spacer.jpg)
News
The Silent Battle Within: How Your Organs Choose Between Mom and Dad’s Genes
Research reveals that selective expression of maternal or paternal X chromosomes varies by organ, driven by cellular competition. A new study published today (July 26) in Nature Genetics by the Lymphoid Development Group at the MRC [...]
Study identifies genes increasing risk of severe COVID-19
Whether or not a person becomes seriously ill with COVID-19 depends, among other things, on genetic factors. With this in mind, researchers from the University Hospital Bonn (UKB) and the University of Bonn, in [...]
Small regions of the brain can take micro-naps while the rest of the brain is awake and vice versa
Sleep and wake: They're totally distinct states of being that define the boundaries of our daily lives. For years, scientists have measured the difference between these instinctual brain processes by observing brain waves, with [...]
Redefining Consciousness: Small Regions of the Brain Can Take Micro-Naps While the Rest of the Brain Is Awake
The study broadly reveals how fast brain waves, previously overlooked, establish fundamental patterns of sleep and wakefulness. Scientists have developed a new method to analyze sleep and wake states by detecting ultra-fast neuronal activity [...]
AI Reveals Health Secrets Through Facial Temperature Mapping
Researchers have found that different facial temperatures correlate with chronic illnesses like diabetes and high blood pressure, and these can be detected using AI with thermal cameras. They highlight the potential of this technology [...]
Breakthrough in aging research: Blocking IL-11 extends lifespan and improves health in mice
In a recent study published in the journal Nature, a team of researchers used murine models and various pharmacological and genetic approaches to examine whether pro-inflammatory signaling involving interleukin (IL)-11, which activates signaling molecules such [...]
Promise for a universal influenza vaccine: Scientists validate theory using 1918 flu virus
New research led by Oregon Health & Science University reveals a promising approach to developing a universal influenza vaccine—a so-called "one and done" vaccine that confers lifetime immunity against an evolving virus. The study, [...]
New Projects Aim To Pioneer the Future of Neuroscience
One study will investigate the alterations in brain activity at the cellular level caused by psilocybin, the psychoactive substance found in “magic mushrooms.” How do neurons respond to the effects of magic mushrooms? What [...]
Decoding the Decline: Scientific Insights Into Long COVID’s Retreat
Research indicates a significant reduction in long COVID risk, largely due to vaccination and the virus’s evolution. The study analyzes data from over 441,000 veterans, showing lower rates of long COVID among vaccinated individuals compared [...]
Silicon Transformed: A Breakthrough in Laser Nanofabrication
A new method enables precise nanofabrication inside silicon using spatial light modulation and laser pulses, creating advanced nanostructures for potential use in electronics and photonics. Silicon, the cornerstone of modern electronics, photovoltaics, and photonics, [...]
Caught in the actinium: New research could help design better cancer treatments
The element actinium was first discovered at the turn of the 20th century, but even now, nearly 125 years later, researchers still don't have a good grasp on the metal's chemistry. That's because actinium [...]
Innovative Light-Controlled Drugs Could Revolutionize Neuropathic Pain Treatment
A team of researchers from the Institute for Bioengineering of Catalonia (IBEC) has developed light-activated derivatives of the anti-epileptic drug carbamazepine to treat neuropathic pain. Light can be harnessed to target drugs to specific [...]
Green Gold: Turning E-Waste Into a Treasure Trove of Rare Earth Metals
Scientists are developing a process inspired by nature that efficiently recovers europium from old fluorescent lamps. The approach could lead to the long-awaited recycling of rare earth metals. A small molecule that naturally serves [...]
Cambridge Study: AI Chatbots Have an “Empathy Gap,” and It Could Be Dangerous
A new study suggests a framework for “Child Safe AI” in response to recent incidents showing that many children perceive chatbots as quasi-human and reliable. A study has indicated that AI chatbots often exhibit [...]
Nanoparticle-based delivery system could offer treatment for diabetics with rare insulin allergy
Up to 3% of people with diabetes have an allergic reaction to insulin. A team at Forschungszentrum Jülich has now studied a method that could be used to deliver the active substance into the [...]
Nanorobot kills cancer cells in mice with hidden weapon
Researchers at Karolinska Institutet in Sweden have developed nanorobots that kill cancer cells in mice. The robot's weapon is hidden in a nanostructure and is exposed only in the tumor microenvironment, sparing healthy cells. [...]