Fly, goat, fly! A brand new AI agent from Google DeepMind can play different games, including ones it has never seen before equivalent to Goat Simulator 3, a fun motion game with exaggerated physics. Researchers were capable of get it to follow text commands to play seven different games and move around in three different 3D research environments. It’s a step toward more generalized AI that may transfer skills across multiple environments.  

Google DeepMind has had huge success developing game-playing AI systems. Its system AlphaGo, which beat top skilled player Lee Sedol at the sport Go in 2016, was a significant milestone that showed the facility of deep learning. But unlike earlier game-playing AI systems, which mastered just one game or could only follow single goals or commands, this latest agent is capable of play a wide range of different games, including Valheim and No Man’s Sky. It’s called SIMA, an acronym for “scalable, instructable, multiworld agent.”

In training AI systems, games are a great proxy for real-world tasks. “A general game-playing agent could, in principle, learn rather a lot more about find out how to navigate our world than anything in a single environment ever could,” says Michael Bernstein, an associate professor of computer science at Stanford University, who was not a part of the research. 

“One could imagine sooner or later somewhat than having superhuman agents which you play against, we could have agents like SIMA playing alongside you in games with you and with your mates,” says Tim Harley, a research engineer at Google DeepMind who was a part of the team that developed the agent. 

The team trained SIMA on a lot of examples of humans playing video games, each individually and collaboratively, alongside keyboard and mouse input and annotations of what the players did in the sport, says Frederic Besse, a research engineer at Google DeepMind.  

Then they used an AI technique called imitation learning to show the agent to play games as humans would. SIMA can follow 600 basic instructions, equivalent to “Turn left,” “Climb the ladder,” and “Open the map,” each of which could be accomplished in lower than roughly 10 seconds.

The team found that a SIMA agent that was trained on many games was higher than an agent that learned find out how to play only one. It is because it was capable of benefit from concepts shared between games to learn higher skills and improve at carrying out instructions, says Besse. 

“That is again a extremely exciting key property, as we’ve an agent that may play games it has never seen before, essentially,” he says. 

Seeing this kind of data transfer between games is a major milestone for AI research, says Paulo Rauber, a lecturer in artificial Intelligence at Queen Mary University of London. 

The essential idea of learning to execute instructions on the premise of examples provided by humans may lead to more powerful systems in the long run, especially with larger data sets, Rauber says. SIMA’s relatively limited data set is what’s holding back its performance, he says. 

Although the variety of game environments it’s been trained on remains to be small, SIMA is on the best track for scaling up, says Jim Fan, a senior research scientist at Nvidia who runs its  AI Agents Initiative. 

However the AI system remains to be not near human level, says Harley. For instance, in the sport No Man’s Sky, the AI agent could do exactly 60% of the tasks humans could do. And when the researchers removed the flexibility for humans to present SIMA instructions, they found the agent performed much worse than before. 

Next, Besse says, the team is working on improving the agent’s performance. The researchers need to get it to work in as many environments as possible and learn latest skills, and so they want people to have the ability to speak with the agent and get a response. The team also wants SIMA to have more generalized skills, allowing it to quickly pick up games it has never seen before, very like a human. 

Humans “can generalize thoroughly to unseen environments and unseen situations,” says Besse. “And we would like our agents to be just the identical.”  

SIMA inches us closer to a “ChatGPT moment” for autonomous agents, says Roy Fox, an assistant professor on the University of California, Irvine.  

Nevertheless it is a great distance away from actual autonomous AI. That will be “a complete different ball game,” he says.