A team of researchers at the University of California, Berkeley, has built a two-legged robot with the ability to teach itself to walk using reinforcement learning.
The work involved building a robot consisting of a pair of legs attached together and connected to a smallholding frame. Currently, the robot carries out its tasks tethered to the frame, which is guided by one of the researchers.
Compared to other robots, such as several made by Boston Dynamics, the robot seems primitive. But the robot, which the team has named Cassie, represents the leading edge of a new kind of technology—in which a robot teaches itself to walk, rather than having it learn it through direct programming or mimicry.
Reinforcement learning is the kind exhibited by humans—babies learn to walk by trying new movements. They do not get the whole process at once but remember the steps that lead to walking. And then they build on what they have learned, adding more information that eventually leads to walking upright. But they do not stop there; they keep learning, and in so doing, grow better at it. In time, they learn new tricks, such as running, jumping, or even skipping down a sidewalk.
To help a robot learn to walk in the same way, the researchers began with a simulation of a robot in a virtual world. In the simulation, the robot was trained with information that described goals such as walking upright; an AI engine could remember and use what was learned. The simulation allowed the robot to learn to walk without damaging the actual hardware and sped the process.
Once the robot in the simulation learned to walk, the researchers ported its knowledge to Cassie, who used it to walk in ways similar to a toddler. And just like a toddler, Cassie continued to learn. She learned how to keep from falling when slipping slightly or to recover when shoved from the side. She also learned to compensate when two of her motors were damaged. The researchers plan to continue their work with reinforcement learning in robots to see how far they can go with it.
