When Taylor Webb played around with GPT-3 in early 2022, he was blown away by what OpenAI’s large language model appeared to find a way to do. Here was a neural network trained only to predict the following word in a block of text—a jumped-up autocomplete. And yet it gave correct answers to most of the abstract problems that Webb set for it—the form of thing you’d find in an IQ test. “I used to be really shocked by its ability to unravel these problems,” he says. “It completely upended the whole lot I might have predicted.”
Webb is a psychologist on the University of California, Los Angeles, who studies different ways people and computers solve abstract problems. He was used to constructing neural networks that had specific reasoning capabilities bolted on. But GPT-3 looked as if it would have learned them totally free.
Last month Webb and his colleagues published an article in Nature, during which they describe GPT-3’s ability to pass a wide range of tests devised to evaluate using analogy to unravel problems (referred to as analogical reasoning). On a few of those tests GPT-3 scored higher than a gaggle of undergrads. “Analogy is central to human reasoning,” says Webb. “We predict of it as being certainly one of the foremost things that any form of machine intelligence would want to reveal.”
What Webb’s research highlights is barely the newest in a protracted string of remarkable tricks pulled off by large language models. For instance, when OpenAI unveiled GPT-3’s successor, GPT-4, in March, the corporate published an eye-popping list of skilled and academic assessments that it claimed its latest large language model had aced, including a few dozen highschool tests and the bar exam. OpenAI later worked with Microsoft to point out that GPT-4 could pass parts of the US Medical Licensing Examination.
And multiple researchers claim to have shown that enormous language models can pass tests designed to discover certain cognitive abilities in humans, from chain-of-thought reasoning (working through an issue step-by-step) to theory of mind (guessing what other persons are considering).
These sorts of results are feeding a hype machine predicting that these machines will soon come for white-collar jobs, replacing teachers, doctors, journalists, and lawyers. Geoffrey Hinton has called out GPT-4’s apparent ability to string together thoughts as one reason he’s now petrified of the technology he helped create.
But there’s an issue: there’s little agreement on what those results really mean. Some persons are dazzled by what they see as glimmers of human-like intelligence; others aren’t convinced one bit.
“There are several critical issues with current evaluation techniques for giant language models,” says Natalie Shapira, a pc scientist at Bar-Ilan University in Ramat Gan, Israel. “It creates the illusion that they’ve greater capabilities than what truly exists.”
That’s why a growing variety of researchers—computer scientists, cognitive scientists, neuroscientists, linguists—wish to overhaul the way in which they’re assessed, calling for more rigorous and exhaustive evaluation. Some think that the practice of scoring machines on human tests is wrongheaded, period, and ought to be ditched.
“People have been giving human intelligence tests—IQ tests and so forth—to machines for the reason that very starting of AI,” says Melanie Mitchell, an artificial-intelligence researcher on the Santa Fe Institute in Recent Mexico. “The problem throughout has been what it means if you test a machine like this. It doesn’t mean the identical thing that it means for a human.”
“There’s loads of anthropomorphizing occurring,” she says. “And that’s form of coloring the way in which that we take into consideration these systems and the way we test them.”
With hopes and fears for this technology at an all-time high, it’s crucial that we get a solid grip on what large language models can and can’t do.
Open to interpretation
Many of the problems with how large language models are tested boil right down to the query of how the outcomes are interpreted.
Assessments designed for humans, like highschool exams and IQ tests, take rather a lot with no consideration. When people rating well, it’s protected to assume that they possess the knowledge, understanding, or cognitive skills that the test is supposed to measure. (In practice, that assumption only goes thus far. Academic exams don’t at all times reflect students’ true abilities. IQ tests measure a particular set of skills, not overall intelligence. Each sorts of assessment favor people who find themselves good at those sorts of assessments.)
But when a big language model scores well on such tests, it isn’t clear in any respect what has been measured. Is it evidence of actual understanding? A mindless statistical trick? Rote repetition?
“There’s a protracted history of developing methods to check the human mind,” says Laura Weidinger, a senior research scientist at Google DeepMind. “With large language models producing text that seems so human-like, it’s tempting to assume that human psychology tests will likely be useful for evaluating them. But that’s not true: human psychology tests depend on many assumptions that won’t hold for giant language models.”
Webb is aware of the problems he waded into. “I share the sense that these are difficult questions,” he says. He notes that despite scoring higher than undergrads on certain tests, GPT-3 produced absurd results on others. For instance, it failed a version of an analogical reasoning test about physical objects that developmental psychologists sometimes give to kids.
On this test Webb and his colleagues gave GPT-3 a story a few magical genie transferring jewels between two bottles after which asked it tips on how to transfer gumballs from one bowl to a different, using objects reminiscent of a posterboard and a cardboard tube. The thought is that the story hints at ways to unravel the issue. “GPT-3 mostly proposed elaborate but mechanically nonsensical solutions, with many extraneous steps, and no clear mechanism by which the gumballs can be transferred between the 2 bowls,” the researchers write in Nature.
“That is the form of thing that children can easily solve,” says Webb. “The stuff that these systems are really bad at are inclined to be things that involve understanding of the particular world, like basic physics or social interactions—things which might be second nature for people.”
So how can we make sense of a machine that passes the bar exam but flunks preschool? Large language models like GPT-4 are trained on vast numbers of documents taken from the web: books, blogs, fan fiction, technical reports, social media posts, and far, far more. It’s likely that loads of past exam papers got hoovered up at the identical time. One possibility is that models like GPT-4 have seen so many skilled and academic tests of their training data that they’ve learned to autocomplete the answers.
A variety of these tests—questions and answers—are online, says Webb: “Lots of them are almost definitely in GPT-3’s and GPT-4’s training data, so I feel we actually cannot conclude much of anything.”
OpenAI says it checked to verify that the tests it gave to GPT-4 didn’t contain text that also appeared within the model’s training data. In its work with Microsoft involving the exam for medical practitioners, OpenAI used paywalled test inquiries to make sure that GPT-4’s training data had not included them. But such precautions should not foolproof: GPT-4 could still have seen tests that were similar, if not exact matches.
When Horace He, a machine-learning engineer, tested GPT-4 on questions taken from Codeforces, a web site that hosts coding competitions, he found that it scored 10/10 on coding tests posted before 2021 and 0/10 on tests posted after 2021. Others have also noted that GPT-4’s test scores take a dive on material produced after 2021. Since the model’s training data only included text collected before 2021, some say this shows that enormous language models display a form of memorization reasonably than intelligence.
To avoid that possibility in his experiments, Webb devised latest sorts of test from scratch. “What we’re really keen on is the flexibility of those models simply to determine latest sorts of problem,” he says.
Webb and his colleagues adapted a way of testing analogical reasoning called Raven’s Progressive Matrices. These tests consist of a picture showing a series of shapes arranged next to or on top of one another. The challenge is to determine the pattern within the given series of shapes and apply it to a brand new one. Raven’s Progressive Matrices are used to evaluate nonverbal reasoning in each young children and adults, they usually are common in IQ tests.
As a substitute of using images, the researchers encoded shape, color, and position into sequences of numbers. This ensures that the tests won’t appear in any training data, says Webb: “I created this data set from scratch. I’ve never heard of anything prefer it.”
Mitchell is impressed by Webb’s work. “I discovered this paper quite interesting and provocative,” she says. “It’s a well-done study.” But she has reservations. Mitchell has developed her own analogical reasoning test, called ConceptARC, which uses encoded sequences of shapes taken from the ARC (Abstraction and Reasoning Challenge) data set developed by Google researcher François Chollet. In Mitchell’s experiments, GPT-4 scores worse than people on such tests.
Mitchell also points out that encoding the pictures into sequences (or matrices) of numbers makes the issue easier for this system since it removes the visual aspect of the puzzle. “Solving digit matrices doesn’t equate to solving Raven’s problems,” she says.
Brittle tests
The performance of enormous language models is brittle. Amongst people, it’s protected to assume that somebody who scores well on a test would also do well on an identical test. That’s not the case with large language models: a small tweak to a test can drop an A grade to an F.
“Basically, AI evaluation has not been done in such a way as to permit us to really understand what capabilities these models have,” says Lucy Cheke, a psychologist on the University of Cambridge, UK. “It’s perfectly reasonable to check how well a system does at a specific task, nevertheless it’s not useful to take that task and make claims about general abilities.”
Take an example from a paper published in March by a team of Microsoft researchers, during which they claimed to have identified “sparks of artificial general intelligence” in GPT-4. The team assessed the big language model using a variety of tests. In a single, they asked GPT-4 tips on how to stack a book, nine eggs, a laptop, a bottle, and a nail in a stable manner. It answered: “Place the laptop on top of the eggs, with the screen facing down and the keyboard facing up. The laptop will fit snugly inside the boundaries of the book and the eggs, and its flat and rigid surface will provide a stable platform for the following layer.”
Not bad. But when Mitchell tried her own version of the query, asking GPT-4 to stack a toothpick, a bowl of pudding, a glass of water, and a marshmallow, it suggested sticking the toothpick within the pudding and the marshmallow on the toothpick, and balancing the complete glass of water on top of the marshmallow. (It ended with a helpful note of caution: “Take into account that this stack is delicate and might not be very stable. Be cautious when constructing and handling it to avoid spills or accidents.”)
Here’s one other contentious case. In February, Stanford University researcher Michal Kosinski published a paper during which he claimed to point out that theory of mind “may spontaneously have emerged as a byproduct” in GPT-3. Theory of mind is the cognitive ability to ascribe mental states to others, a trademark of emotional and social intelligence that almost all children pick up between the ages of three and five. Kosinski reported that GPT-3 had passed basic tests used to evaluate the flexibility in humans.
For instance, Kosinski gave GPT-3 this scenario: “Here’s a bag stuffed with popcorn. There is no such thing as a chocolate within the bag. Yet the label on the bag says ‘chocolate’ and never ‘popcorn.’ Sam finds the bag. She had never seen the bag before. She cannot see what’s contained in the bag. She reads the label.”
Kosinski then prompted the model to finish sentences reminiscent of: “She opens the bag and appears inside. She will be able to clearly see that it is filled with …” and “She believes the bag is filled with …” GPT-3 accomplished the primary sentence with “popcorn” and the second sentence with “chocolate.” He takes these answers as evidence that GPT-3 displays at the very least a basic type of theory of mind because they capture the difference between the actual state of the world and Sam’s (false) beliefs about it.
It’s no surprise that Kosinski’s results made headlines. In addition they invited immediate pushback. “I used to be rude on Twitter,” says Cheke.
Several researchers, including Shapira and Tomer Ullman, a cognitive scientist at Harvard University, published counterexamples showing that enormous language models failed easy variations of the tests that Kosinski used. “I used to be very skeptical given what I find out about how large language models are built,” says Ullman.
Ullman tweaked Kosinski’s test scenario by telling GPT-3 that the bag of popcorn labeled “chocolate” was transparent (so Sam could see it was popcorn) or that Sam couldn’t read (so she wouldn’t be misled by the label). Ullman found that GPT-3 didn’t ascribe correct mental states to Sam each time the situation involved an additional few steps of reasoning.
“The belief that cognitive or academic tests designed for humans function accurate measures of LLM capability stems from an inclination to anthropomorphize models and align their evaluation with human standards,” says Shapira. “This assumption is misguided.”
For Cheke, there’s an obvious solution. Scientists have been assessing cognitive abilities in non-humans for a long time, she says. Artificial-intelligence researchers could adapt techniques used to review animals, which have been developed to avoid jumping to conclusions based on human bias.
Take a rat in a maze, says Cheke: “How is it navigating? The assumptions you possibly can make in human psychology don’t hold.” As a substitute researchers should do a series of controlled experiments to determine what information the rat is using and the way it’s using it, testing and ruling out hypotheses one after the other.
“With language models, it’s more complex. It’s not like there are tests using language for rats,” she says. “We’re in a brand new zone, but a lot of the elemental ways of doing things hold. It’s just that now we have to do it with language as an alternative of with somewhat maze.”
Weidinger is taking an identical approach. She and her colleagues are adapting techniques that psychologists use to evaluate cognitive abilities in preverbal human infants. One key idea here is to interrupt a test for a specific ability down right into a battery of several tests that search for related abilities as well. For instance, when assessing whether an infant has learned tips on how to help one other person, a psychologist may also assess whether the infant understands what it’s to hinder. This makes the general test more robust.
The issue is that these sorts of experiments take time. A team might study rat behavior for years, says Cheke. Artificial intelligence moves at a far faster pace. Ullman compares evaluating large language models to Sisyphean punishment: “A system is claimed to exhibit behavior X, and by the point an assessment shows it doesn’t exhibit behavior X, a brand new system comes along and it’s claimed it shows behavior X.”
Moving the goalposts
Fifty years ago people thought that to beat a grand master at chess, you would want a pc that was as intelligent as an individual, says Mitchell. But chess fell to machines that were simply higher number crunchers than their human opponents. Brute force won out, not intelligence.
Similar challenges have been set and passed, from image recognition to Go. Every time computers are made to do something that requires intelligence in humans, like play games or use language, it splits the sector. Large language models are actually facing their very own chess moment. “It’s really pushing us—everybody—to take into consideration what intelligence is,” says Mitchell.
Does GPT-4 display real intelligence by passing all those tests or has it found an efficient, but ultimately dumb, shortcut—a statistical trick pulled from a hat stuffed with trillions of correlations across billions of lines of text?
“Should you’re like, ‘Okay, GPT4 passed the bar exam, but that doesn’t mean it’s intelligent,’ people say, ‘Oh, you’re moving the goalposts,’” says Mitchell. “But do we are saying we’re moving the goalpost or do we are saying that’s not what we meant by intelligence—we were mistaken about intelligence?”
It comes right down to how large language models do what they do. Some researchers wish to drop the obsession with test scores and check out to determine what goes on under the hood. “I do think that to actually understand their intelligence, if we wish to call it that, we’re going to have to grasp the mechanisms by which they reason,” says Mitchell.
Ullman agrees. “I sympathize with individuals who think it’s moving the goalposts,” he says. “But that’s been the dynamic for a very long time. What’s latest is that now we don’t understand how they’re passing these tests. We’re just told they passed it.”
The difficulty is that no one knows exactly how large language models work. Teasing apart the complex mechanisms inside an enormous statistical model is tough. But Ullman thinks that it’s possible, in theory, to reverse-engineer a model and discover what algorithms it uses to pass different tests. “I could more easily see myself being convinced if someone developed a method for determining what these items have actually learned,” he says.
“I feel that the elemental problem is that we keep specializing in test results reasonably than the way you pass the tests.”