AI language models are usually not humans, and yet we evaluate them as in the event that they were, using tests just like the bar exam or the US Medical Licensing Examination.
The models are inclined to do very well in these exams, probably because examples of such exams are abundant within the models’ training data. As my colleague Will Douglas Heaven writes in his most up-to-date article, “some individuals are dazzled by what they see as glimmers of human-like intelligence; others aren’t convinced one bit.”
A growing variety of experts have called for these tests to be ditched, saying they boost AI hype and create “the illusion that [AI language models] have greater capabilities than what truly exists.” Read the total story here.
What stood out to me in Will’s story is that we all know remarkably little about how AI language models work and why they generate the things they do. With these tests, we’re attempting to measure and glorify their “intelligence” based on their outputs, without fully understanding how they function under the hood.
Other highlights:
Our tendency to anthropomorphize makes this messy: “People have been giving human intelligence tests—IQ tests and so forth—to machines for the reason that very starting of AI,” says Melanie Mitchell, an artificial-intelligence researcher on the Santa Fe Institute in Recent Mexico. “The problem throughout has been what it means while you test a machine like this. It doesn’t mean the identical thing that it means for a human.”
Kids vs. GPT-3: Researchers on the University of California, Los Angeles, gave GPT-3 a story a few magical genie transferring jewels between two bottles after which asked it the right way to transfer gumballs from one bowl to a different, using objects resembling a posterboard and a cardboard tube. The thought is that the story hints at ways to resolve the issue. GPT-3 proposed elaborate but mechanically nonsensical solutions. “That is the kind of thing that children can easily solve,” says Taylor Webb, one in every of the researchers.
AI language models are usually not humans: “With large language models producing text that seems so human-like, it’s tempting to assume that human psychology tests shall be useful for evaluating them. But that’s not true: human psychology tests depend on many assumptions that won’t hold for giant language models,” says Laura Weidinger, a senior research scientist at Google DeepMind.
Lessons from the animal kingdom: Lucy Cheke, a psychologist on the University of Cambridge, UK, suggests AI researchers could adapt techniques used to review animals, which have been developed to avoid jumping to conclusions based on human bias.
No person knows how language models work: “I believe that the elemental problem is that we keep specializing in test results slightly than the way you pass the tests,” says Tomer Ullman, a cognitive scientist at Harvard University.
Read the total story here.
Deeper Learning
Google DeepMind has launched a watermarking tool for AI-generated images
Google DeepMind has launched a brand new watermarking tool that labels whether images have been generated with AI. The tool, called SynthID, will initially be available only to users of Google’s AI image generator Imagen. Users will give you the option to generate images after which select whether so as to add a watermark or not. The hope is that it could help people tell when AI-generated content is being passed off as real, or protect copyright.
Baby steps: Google DeepMind is now the primary Big Tech company to publicly launch such a tool, following a voluntary pledge with the White House to develop responsible AI. Watermarking—a method where you hide a signal in a bit of text or a picture to discover it as AI-generated—has turn out to be probably the most popular ideas proposed to curb such harms. It’s a great start, but watermarks alone won’t create more trust online. Read more from me here.
Bits and Bytes
Chinese ChatGPT alternatives just got approved for most people
Baidu, one in every of China’s leading artificial-intelligence corporations, has announced it’ll open up access to its ChatGPT-like large language model, Ernie Bot, to most people. Our reporter Zeyi Yang looks at what this implies for Chinese web users. (MIT Technology Review)
Brain implants helped create a digital avatar of a stroke survivor’s face
Incredible news. Two papers in Nature show major advancements in the trouble to translate brain activity into speech. Researchers managed to assist women who had lost their ability to talk communicate again with the assistance of a brain implant, AI algorithms and digital avatars. (MIT Technology Review)
Contained in the AI porn marketplace where every part and everyone seems to be on the market
This was a wonderful investigation how the generative AI boom has created a seedy marketplace for deepfake porn. Completely predictable and frustrating how little we’ve got done to forestall real-life harms like nonconsensual deepfake pornogrpahy. (404 Media)
A military of overseas staff in “digital sweatshops” power the AI boom
Hundreds of thousands of individuals working within the Philippines work as data annotators for data company Scale AI. But as this investigation into the questionable labor conditions shows, many staff are earning below the minimum wage and have had payments delayed, reduced, or canceled.
(The Washington Post)
The tropical Island with the recent domain name
Lol. The AI boom has meant Anguilla has hit the jackpot with its .ai domain name. The country is predicted to make thousands and thousands this 12 months from corporations wanting the buzzy designation. (Bloomberg)
P.S. We’re hiring!
MIT Technology Review is on the lookout for an ambitious AI reporter to hitch our team with an emphasis on the intersection of hardware and AI. This position is predicated in Cambridge, Massachusetts. Feels like you, or someone you realize? Read more here.