We’ve all seen videos over the past few years demonstrating how agile humanoid robots have grow to be, running and jumping with ease. We’re now not surprised by this type of agility—in reality, we’ve grown to expect it.

The issue is, these shiny demos lack real-world applications. In terms of creating robots which can be useful and secure around humans, the basics of movement are more necessary. Because of this, researchers are using the identical techniques to coach humanoid robots to attain far more modest goals. 

Alan Fern, a professor of computer science at Oregon State University, and a team of researchers have successfully trained a humanoid robot called Digit V3 to face, walk, pick up a box, and move it from one location to a different. Meanwhile, a separate group of researchers from the University of California, Berkeley, have focused on teaching Digit to walk in unfamiliar environments while carrying different loads, without toppling over. Their research is published in a paper in today. 

Each groups are using an AI technique called sim-to-real reinforcement learning, a burgeoning method of coaching two-legged robots like Digit. Researchers imagine it’ll result in more robust, reliable two-legged machines able to interacting with their surroundings more safely—in addition to learning far more quickly.

Sim-to-real reinforcement learning involves training AI models to finish certain tasks in simulated environments billions of times before a robot powered by the model attempts to finish them in the actual world. What would take years for a robot to learn in real life can take just days because of repeated trial-and-error testing in simulations.

A neural network guides the robot using a mathematical reward function, a method that rewards the robot with a big number each time it moves closer to its goal location or completes its goal behavior. If it does something it’s not purported to do, like falling down, it’s “punished” with a negative number, so it learns to avoid these motions over time.

In previous projects, researchers from the University of Oregon had used the identical reinforcement learning technique to show a two-legged robot named Cassie to run. The approach paid off—Cassie became the primary robot to run an out of doors 5K before setting a Guinness World Record for the fastest bipedal robot to run 100 meters and mastering the flexibility to leap from one location to a different with ease.

Training robots to behave in athletic ways requires them to develop really complex skills in very narrow environments, says Ilija Radosavovic, a PhD student at Berkleley who trained Digit to hold a big selection of loads and stabilize itself when poked with a stick. “We’re form of the other—specializing in fairly easy skills in broad environments.”

This latest wave of research in humanoid robotics is less concerned with speed and skill, and more focused on making machines robust and in a position to adapt—which is ultimately what’s needed to make them useful in the actual world. Humanoid robots remain a relative rarity in work environments, as they often struggle to balance while carrying heavy objects. For this reason most robots designed to lift objects of various weights in factories and warehouses are inclined to have 4 legs or larger, more stable bases. But researchers hope to vary that by making humanoid robots more reliable using AI techniques. 

Reinforcement learning will usher in a “latest, far more flexible and faster way for training these kinds of manipulation skills,” Fern says. He and his team are on account of present their findings at ICRA, the International Conference on Robotics and Automation, in Japan next month.

The final word goal is for a human to give you the chance to point out the robot a video of the specified task, like picking up a box from one shelf and pushing it onto one other higher shelf, after which have the robot do it without requiring any further instruction, says Fern.

Getting robots to watch, copy, and quickly learn these sorts of behaviors can be really useful, but it surely still stays a challenge, says Lerrel Pinto, an assistant professor of computer science at Latest York University, who was not involved within the research. “If that may very well be done, I can be very impressed by that,” he says. “These are hard problems.”