This robot develops "self-awareness" by observing itself in a mirror 🤖

Robots are developing an astonishing ability: understanding their own bodies by observing their movements. A breakthrough that could make them more autonomous and resilient in complex environments.


This innovation, stemming from the work of researchers at Columbia University, relies on a simple yet powerful method. By analyzing videos of their own movements, robots create an internal representation of their structure and capabilities. This "self-awareness" allows them to adapt to damage or unexpected changes without human intervention.

Learning by observing

Traditionally, robots are trained in virtual simulations designed by engineers. These simulations, while precise, require considerable time and resources. The new approach bypasses this step by allowing robots to generate their own models. This method, inspired by how humans learn by observing their reflection in a mirror, paves the way for increased autonomy.

Using deep neural networks, robots interpret 2D images to reconstruct a 3D understanding of their bodies. These networks, composed of three distinct modules, analyze the robot's movements and deduce an internal representation of its structure. This ability to "see" themselves move allows the robot to plan its actions and adapt to unexpected situations.

One of the major advantages of this technique is its simplicity. Unlike traditional simulations, which require advanced technical expertise, this method relies on accessible visual data. A robot can thus evolve and refine its model over time, without human intervention. This flexibility is essential for applications in dynamic and unpredictable environments.

Unprecedented adaptation

One of the most promising aspects of this technology is its ability to detect and compensate for damage. For example, a robot with a damaged arm can adjust its movements to continue functioning. This resilience is promising for domestic, industrial, or even disaster zone applications.

Researchers tested this capability by simulating damage to a robotic arm. After observing its new state, the robot successfully recalibrated its actions to complete its tasks. Such adaptability could reduce downtime in factories and improve the reliability of domestic robots. This autonomy represents a giant leap toward truly independent machines.

In complex environments, such as disaster zones, this technology could save lives. A robot capable of adapting to damage or unexpected obstacles could intervene in situations too dangerous for humans. Similarly, in homes, robot vacuum cleaners or personal assistants could operate more reliably, without requiring constant assistance. This advancement marks a key step toward the harmonious integration of robots into our daily lives.

To go further: What is "self-awareness" in a robot?

"Self-awareness" in a robot refers to its ability to understand its own structure and movements. Thanks to deep learning algorithms, the robot analyzes videos of itself to create an internal model. This understanding allows it to adapt to physical changes, such as damage or modifications to its environment.

Unlike human self-awareness, which involves a complex emotional and cognitive dimension, that of robots is limited to a mechanical and functional representation. The robot does not "feel" anything, but it can predict the consequences of its actions and adjust its behavior accordingly. This capability is essential for tasks requiring high precision and rapid adaptation.

How do deep neural networks work?

Deep neural networks are artificial intelligence systems that mimic the functioning of the human brain. They analyze data, such as images or videos, to extract complex information. In this study, three neural networks work together to transform 2D images into a 3D understanding of the robot's movements.

The first network, called the coordinate encoder, processes raw images captured by the camera. It identifies key points of the robot, such as its joints, and translates them into usable data. The second network, the kinematic encoder, converts this data into a simplified binary image, representing the robot's structure. Finally, the predictive model uses this information to plan the robot's movements in the real world.

These neural networks are trained on large amounts of data, allowing them to learn to recognize patterns and make accurate predictions. The more they are exposed to varied situations, the better their ability to interpret and react improves. This approach allows robots to adapt to changing environments without requiring manual reprogramming.

In the future, these networks could be further enhanced to include additional data, such as tactile or auditory information. This would allow robots to better understand their environment and interact more naturally with humans and objects around them. Another step toward truly intelligent and autonomous machines.