Artificial intelligence systems found to excel at imitation, but not innovation

Artificial intelligence (AI) systems are often depicted as sentient agents poised to overshadow the human mind. But AI lacks the crucial human ability of innovation, researchers at the University of California, Berkeley have found.

AI language models like ChatGPT are passively trained on data sets containing billions of words and images produced by humans. This allows AI systems to function as a “cultural technology” similar to writing that can summarize existing knowledge, Eunice Yiu, a co-author of the article, explained in an interview. But unlike humans, they struggle when it comes to innovating on these ideas, she said.

“Even young human children can produce intelligent responses to certain questions that [language learning models] cannot,” Yiu said. “Instead of viewing these AI systems as intelligent agents like ourselves, we can think of them as a new form of library or search engine. They effectively summarize and communicate the existing culture and knowledge base to us.”

Yiu and Eliza Kosoy, along with their doctoral advisor and senior author on the paper, developmental psychologist Alison Gopnik, tested how the AI systems’ ability to imitate and innovate differs from that of children and adults. They presented 42 children ages 3 to 7 and 30 adults with text descriptions of everyday objects.

In the first part of the experiment, 88% of children and 84% of adults were able to correctly identify which objects would “go best” with another. For example, they paired a compass with a ruler instead of a teapot.

In the next stage of the experiment, 85% of children and 95% of adults were also able to innovate on the expected use of everyday objects to solve problems. In one task, for example, participants were asked how they could draw a circle without using a typical tool such as a compass.

Given the choice between a similar tool like a ruler, a dissimilar tool such as a teapot with a round bottom, and an irrelevant tool such as a stove, the majority of participants chose the teapot, a conceptually dissimilar tool that could nonetheless fulfill the same function as the compass by allowing them to trace the shape of a circle.

When Yiu and colleagues provided the same text descriptions to five large language models, the models performed similarly to humans on the imitation task, with scores ranging from 59% for the worst-performing model to 83% for the best-performing model. The AIs’ answers to the innovation task were far less accurate, however. Effective tools were selected anywhere from 8% of the time by the worst-performing model to 75% by the best-performing model.

“Children can imagine completely novel uses for objects that they have not witnessed or heard of before, such as using the bottom of a teapot to draw a circle,” Yiu said. “Large models have a much harder time generating such responses.”

In a related experiment, the researchers noted, children were able to discover how a new machine worked just by experimenting and exploring. But when the researchers gave several large language models text descriptions of the evidence that the children produced, they struggled to make the same inferences, likely because the answers were not explicitly included in their training data, Yiu and colleagues wrote.

These experiments demonstrate that AI’s reliance on statistically predicting linguistic patterns is not enough to discover new information about the world, Yiu and colleagues wrote.

“AI can help transmit information that is already known, but it is not an innovator,” Yiu said. “These models can summarize conventional wisdom, but they cannot expand, create, change, abandon, evaluate, and improve on conventional wisdom in the way a young human can.”

The development of AI is still in its early days, though, and much remains to be learned about how to expand the learning capacity of AI, Yiu said. Taking inspiration from children’s curious, active, and intrinsically motivated approach to learning could help researchers design new AI systems that are better prepared to explore the real world, she said.