A robot can learn more effectively when it decides for itself which movements deserve its training time.
A robot capable of manipulating objects often has several skills: grasping, moving, opening, or using a device. But it cannot necessarily repeat every movement hundreds of times. Researchers therefore studied another question: how can it choose the exercises that will most improve its future tasks?
Fictitious illustrative image
Their method, called Deliberate Practice, assigns each skill a share of the available time.
A difficult movement may seem uninteresting when examined on its own. Yet mastering it can provide access to a better way of completing an entire task. Conversely, further perfecting a movement that is already reliable may contribute very little to the final result.
In practice, the algorithm therefore compares the cost of learning with the expected benefit. With little time, the robot may favor a simple strategy that it can quickly master. With more training, it can work on more difficult skills that then enable a more advantageous sequence of actions.
The researchers tested this approach in simulations and on a real robot tasked with multistep manipulations. One demonstration involves preparing breakfast. Depending on its training budget, the machine can learn a simple sequence involving toast or invest more time to master a different preparation in a microwave oven.
The method therefore does not involve letting the robot experiment at random. A program calculates in advance how to allocate the budget to maximize the expected result according to the model being used. The authors formulated this calculation so that it can be handled with existing optimization software, even when many possible sequences of movements exist.
There is, however, one important limitation. The system must correctly estimate the rate at which each skill improves with practice. If this prediction is wrong, the program may devote too much time to a movement that improves more slowly than expected. The best theoretical program therefore depends directly on the quality of these estimates.
Future experiments may test this method on more robots and in less controlled environments. One challenge will be to update the predictions during learning, as the machine discovers its actual difficulties. The robot could then reorganize its training when a movement improves faster or more slowly than expected.