One question has relentlessly followed ChatGPT in its trajectory to superstar status in the field of artificial intelligence: Has it met the Turing test of generating output indistinguishable from human response?
ChatGPT may be smart, quick and impressive. It does a good job at exhibiting apparent intelligence. It sounds humanlike in conversations with people and can even display humor, emulate the phraseology of teenagers, and pass exams for law school.
But on occasion, it has been found to serve up totally false information. It hallucinates. It does not reflect on its own output.
Cameron Jones, who specializes in language, semantics and machine learning, and Benjamin Bergen, professor of cognitive science, drew upon the work of Alan Turing, who 70 years ago devised a process to determine whether a machine could reach a point of intelligence and conversational prowess at which it could fool someone into thinking it was human.
Their report titled “Does GPT-4 Pass the Turing Test?” is available on the arXiv preprint server.
They rounded up 650 participants and generated 1,400 “games” in which brief conversations were conducted between participants and either another human or a GPT model. Participants were asked to determine who they were conversing with.
The researchers found that GPT-4 models fooled participants 41% of the time, while GPT-3.5 fooled them only 5% to 14% of the time. Interestingly, humans succeeded in convincing participants they were not machines in only 63% of the trials.
The researchers concluded, “We do not find evidence that GPT-4 passes the Turing Test.”
They noted, however, that the Turing test still retains value as a measure of the effectiveness of machine dialogue.
“The test has ongoing relevance as a framework to measure fluent social interaction and deception, and for understanding human strategies to adapt to these devices,” they said.
They warned that in many instances, chatbots can still communicate convincingly enough to fool users in many instances.
“A success rate of 41% suggests that deception by AI models may already be likely, especially in contexts where human interlocutors are less alert to the possibility they are not speaking to a human,” they said. “AI models that can robustly impersonate people could have could have widespread social and economic consequences.”
The researchers observed that participants making correct identifications focused on several factors.
Models that were too formal or too informal raised red flags for participants. If they were too wordy or too brief, if their grammar or use of punctuation was exceptionally good or “unconvincingly” bad, their usage became key factors in determining whether participants were dealing with humans or machines.
Test takers also were sensitive to generic-sounding responses.
“LLMs learn to produce highly likely completions and are fine-tuned to avoid controversial opinions. These processes might encourage generic responses that are typical overall, but lack the idiosyncrasy typical of an individual: a sort of ecological fallacy,” the researchers said.
The researchers have suggested that it will be important to track AI models as they gain more fluidity and absorb more humanlike quirks in conversation.
“It will become increasingly important to identify factors that lead to deception and strategies to mitigate it,” they said.

News
Tiny robots made from human cells heal damaged tissue
The ‘anthrobots’ were able to repair a scratch in a layer of neurons in the lab. Scientists have developed tiny robots made of human cells that are able to repair damaged neural tissue1. The [...]
Antimicrobial Resistance – A Global Concern
Key facts Antimicrobial resistance (AMR) is one of the top global public health and development threats. It is estimated that bacterial AMR was directly responsible for 1.27 million global deaths in 2019 and contributed to [...]
Advancing Pancreatic Cancer Treatment with Nanoparticle-Based Chemotherapy
Pancreatic cancer, a particularly lethal form of cancer and the fourth leading cause of cancer-related deaths in the western world, often remains undiagnosed until its advanced stages due to a lack of early symptoms. [...]
The ‘jigglings and wigglings of atoms’ reveal key aspects of COVID-19 virulence evolution
Richard Feynman famously stated, "Everything that living things do can be understood in terms of the jigglings and wigglings of atoms." This week, Nature Nanotechnology features a study that sheds new light on the evolution of the coronavirus [...]
AI system self-organizes to develop features of brains of complex organisms
Cambridge scientists have shown that placing physical constraints on an artificially-intelligent system—in much the same way that the human brain has to develop and operate within physical and biological constraints—allows it to develop features [...]
How Blind People Recognize Faces via Sound
Summary: A new study reveals that people who are blind can recognize faces using auditory patterns processed by the fusiform face area, a brain region crucial for face processing in sighted individuals. The study employed [...]
Treating tumors with engineered dendritic cells
Cancer biologists at EPFL, UNIGE, and the German Cancer Research Center (Heidelberg) have developed a novel immunotherapy that does not require knowledge of a tumor's antigenic makeup. The new results may pave the way [...]
Networking nano-biosensors for wireless communication in the blood
Biological computing machines, such as micro and nano-implants that can collect important information inside the human body, are transforming medicine. Yet, networking them for communication has proven challenging. Now, a global team, including EPFL [...]
Popular Hospital Disinfectant Ineffective Against Common Superbug
Research conducted during World Antimicrobial Awareness Week examines the effects of employing suggested chlorine-based chemicals to combat Clostridioides difficile, the leading cause of antibiotic-related illness in healthcare environments worldwide. A recent study reveals that a [...]
Subjectivity and the Evolution of AI Philosophy
An Historical Overview of the Philosophy of Artificial Intelligence by Anton Vokrug Many famous people in the philosophy of technology have tried to comprehend the essence of technology and link it to society and human [...]
How Lockdowns Shaped the Virus: AI Uncovers COVID-19’s Evolutionary Secrets
A new research study shows that human behavior, like lockdowns, influences the evolution of COVID-19, leading to strains that are more transmissible earlier in their lifecycle. Using artificial intelligence technology and mathematical modeling, a research [...]
Groundbreaking therapy approved: chances of cure for 7000 diseases:
Hereditary diseases are usually not curable. Now, however, an epochal turning point is taking place in medicine: For the first time ever, a therapy with the CRISPR/Cas9 gene scissors has received approval. According to [...]
Uncovering the Genetic Mystery: Why Some Never Show COVID-19 Symptoms
New study shows that common genetic variation among people is responsible for mediating SARS-CoV-2 asymptomatic infection. Have you ever wondered why some people never became sick from COVID-19? A study published recently in the journal Nature shows that common [...]
AI maps tumor geography for tailored treatments
Researchers have integrated AI approaches from satellite mapping and community ecology to develop a tool to interpret data obtained from tumor tissue imaging, with the aim of implementing a more individualized approach to cancer care. [...]
Lung cancer cells’ ‘memories’ suggest new strategy for improving treatment
A new understanding of lung cancer cells' "memories" suggests a new strategy for improving treatment, Memorial Sloan Kettering Cancer Center (MSK) researchers have found. Research from the lab of cancer biologist Tuomas Tammela, MD, Ph.D. [...]
Artificial sensor similar to a human fingerprint can recognize fine fabric textures
An artificial sensory system that is able to recognize fine textures—such as twill, corduroy and wool—with a high resolution, similar to a human finger, is reported in a Nature Communications paper. The findings may help improve the subtle [...]