A robot can now prepare its next movement by representing what might happen, without generating a complete video of the future scene.
To act correctly, a robot often needs to anticipate the consequences of its movements. Moving a cup, grasping an object or opening a drawer changes its environment. Some artificial intelligences therefore produce several images of the presumed future before choosing an action. This method provides useful information, but requires a great deal of computation for each decision.
Some everyday tasks require great dexterity and anticipation of one's movements.
AI-generated image
A team associated in particular with Shanghai Jiao Tong University and ACE Robotics has proposed another approach, called ForeWAM. Instead of constructing visible future images, the system maintains an internal representation of possible changes. This contains clues about the movement of objects, contacts with the robot and the progress of a task.
In other words, the AI does not need to “see” a video it has just created itself. It prepares internal states corresponding to possible futures, then reuses this information when calculating the next movement. This avoids several successive stages of image generation.
Training still requires real observations before and after an action. A model used only during this phase helps ForeWAM recognize relevant changes. Once trained, the system operates without these future observations and without this additional model. The robot then receives the current image and an instruction, and directly calculates its action.
The researchers evaluated their method on LIBERO, a set of simulated robotic manipulation tasks. ForeWAM succeeds in an average of 96.7% of trials. Another version reaches 96.9%. Each task in the standard test was repeated 50 times, making it possible to measure the actual frequency of successful attempts in this digital environment.
A second test, LIBERO-Plus, changes the camera, lighting, robot position, instructions and even the arrangement of objects. ForeWAM achieves a 61.6% success rate on the evaluated sample. This result therefore measures its ability to cope with situations different from those encountered during training.
It remains to be determined whether this method retains its advantages outside simulation. The authors specify that their tests cover only LIBERO and LIBERO-Plus. Other types of robots, longer tasks and real physical environments could produce different results. The work is currently available as a preprint on arXiv.