How Robots Learn to Grasp and Walk

For decades, robots were fast and precise but blind to change. A welding robot on an assembly line repeats the same path millions of times; move the car body by two centimeters and it welds into the air. Robots that work among people, in warehouses, kitchens or hospitals, need something different: they have to cope with objects they have never seen, in places that change every day. That is where AI comes in.

This is part 3 of Level 0 in our Physical AI learning path. You will learn the four ways robots get their skills today, why practice in simulation is so important, what the new foundation models for robots do and how to read the data on the notable robot models of recent years.

Four ways to teach a robot

Programming is the classic way. An engineer defines every point of the movement, often with a hand-held control panel. The result is exact and repeatable, and it fails as soon as something is not where it should be.

Learning from demonstrations, also called imitation learning, shows the robot what to do. A person moves the robot arm by hand or steers it remotely, the robot records the camera images and joint positions, and a model learns to copy the movement from the images. A few dozen to a few hundred demonstrations can be enough for one task, such as putting dishes into a rack.

Practice in simulation uses reinforcement learning: the robot tries actions in a virtual world and gets a reward for success, for example for every step it walks without falling. Over millions of attempts, a model learns a strategy that no engineer wrote down. Walking robots with legs learn their gait this way.

Foundation models for robots are the newest way. They are large models that link images, language and actions. Because parts of them were trained on images and text from the internet, they bring general knowledge with them: a robot can follow the instruction “pick up the toy that is an extinct animal” and grab the toy dinosaur, even though no one demonstrated that exact task.

Why simulation matters so much

A real robot can only practice as fast as the real world allows, and every fall can break hardware. In a simulation, thousands of virtual robots practice in parallel, faster than real time. The catch is the sim-to-real gap: friction, light, weight and sensor noise in a simulation never match reality exactly. A common trick is to vary these properties at random during practice, a little more slippery here, a little darker there, so that the model learns a strategy that works across many conditions, and the real world becomes just one more variation.

Foundation models for robots

The most important change of recent years is the vision-language-action model (VLA). It takes a camera image and an instruction in words, and it outputs actions, such as the next positions of the arm joints, several times per second.

Epoch AI, a research institute that records notable AI models, lists 31 models for robots and vehicles since the first maze-solving robot mouse in 1950. 19 of them were published in 2022 or later, most of them for robotic manipulation, which means grasping and moving objects. The newest ones in the table are foundation models of this kind.

Why so many new robots look like humans

Many of the robots presented in recent years are humanoids, with two legs, two arms and hands. The reason is practical: our world is built for human bodies, with stairs, door handles, shelves at arm’s height and tools made for hands. A robot shaped like a person can, in principle, work anywhere a person can, and it can learn from recordings of people doing the same tasks. But humanoids are also the hardest robots to build: walking on two legs costs energy, and a human hand has more degrees of freedom than most grippers. Most robots in real use today are therefore not humanoids, but arms on fixed bases and wheeled machines that carry goods through warehouses.

What robots still find hard

Robot foundation models are where language models were a few years ago: impressive in demonstrations, unreliable in daily use. Tasks that a child does without thinking remain difficult: folding a crumpled shirt, opening an unfamiliar door, handling soft or transparent objects. The main bottleneck is data. A language model learns from text of a large part of the internet; robot data has to be recorded with real robots or produced in simulations. Teams around the world therefore share their recordings in open datasets, and companies collect recordings from robots at work to train the next model.

When you watch a video of a robot doing a household task, look for three things: whether it is sped up, whether a person controls the robot remotely, and how often the task succeeds when repeated. The next part of Level 0 looks at a very different kind of physical AI: interfaces that connect the brain directly to a computer.