A humanoid robot learned to walk toward an object and then grasp it after training in more than 150,000 virtual scenes.
Making a robot walk and grasp an object are already two difficult tasks separately. The machine must spot its target, approach it without falling, position its body correctly, and then coordinate its arm and hand.
The Unitree G1 robot
The researchers named their method FetchMan. Training begins entirely in a simulation. Synthetic demonstrations give the robot examples of behaviors to reproduce. This first step teaches it the basics, but the researchers found that continually adding more demonstrations eventually brings little progress.
They therefore added a second method: reinforcement learning. The robot attempts actions in the virtual world and receives a reward when it achieves its goal. It can thus improve the strategy it initially learned, without a human having to show it every movement to perform.
This combination was applied to more than 150,000 simulated scenes. Their variety forces the machine to cope with different starting positions, targets, and configurations. The goal is to prevent it from merely memorizing a few precise situations and failing as soon as its environment changes.
The transition to the real world is the most interesting test. The researchers directly transferred the learned behavior to a Unitree G1, a commercial humanoid robot. No additional training in the physical world was necessary for this step, an approach known as direct sim-to-real transfer.
Faced with scenes it had not encountered, the robot had to reach a single object and then grasp it. It succeeded in 73.3% of trials. This rate still leaves significant room for improvement before reliable use in ordinary settings, where objects, obstacles, and people can change constantly.
The result mainly demonstrates the value of combining imitation and repeated trials. Imitation quickly provides usable starting behavior. Reinforcement learning can then overcome the limits of this copy by discovering more effective actions over the course of virtual attempts.
The researchers have also begun training FetchMan with multiple objects. This extension should prepare robots capable of choosing and manipulating different targets rather than a single object specified in advance.