🧠 Continuous learning of AIs: towards an approach inspired by biological synapses

The continuous assimilation of knowledge by artificial intelligence systems relies on a delicate compromise between their tendency to forget old knowledge and their rigidity when incorporating new data. In a study published in Nature Communications, scientists used Bayesian approaches inspired by biological synapses, to introduce uncertainty and better balance memory and adaptation.

The human brain continuously learns while preserving acquired knowledge, a balance that artificial intelligence (AI) systems still struggle to reproduce. When an AI model assimilates new information, it often tends to erase previously acquired knowledge (catastrophic forgetting) or, conversely, become too rigid to integrate new data (catastrophic recall).


To overcome this challenge, scientists from the Center for Nanosciences and Nanotechnologies (C2N, CNRS/Université Paris-Saclay), the CEA-Leti and the CEA-List drew inspiration from neuroscience, where recent work suggests that biological synapses follow Bayesian principles: they adjust their representations of the world by weighting new observations against prior knowledge, while taking into account their degree of uncertainty.

On this basis, the team proposed a new continuous learning framework, called Metaplasticity from Synaptic Uncertainty (MESU).

In MESU, each connection in the network acts as a Bayesian synapse, maintaining its own uncertainty estimate. It thus adapts its learning speed according to the confidence placed in new information, while incorporating a progressive forgetting mechanism for data deemed less relevant. MESU therefore translates certain neuroscientific hypotheses about how the brain reconciles memory stability and cognitive flexibility.

The experiments conducted showed that MESU achieves a solid balance between memorization and adaptation. On several datasets, including animal image classification, permuted digit recognition, and incremental object learning, MESU significantly reduces both forgetting and learning rigidity, while providing reliable uncertainty estimates. It outperforms continuous learning methods based on consolidation or explicit task separation.

Beyond these results, MESU establishes a strong theoretical link between neuroscience and machine learning, by formalizing an approach inspired by brain functioning to manage continuous learning. Our next step will be to extend MESU towards probabilistic models compatible with embedded hardware, in order to make this bio-inspired continuous learning applicable to real, low-power AI devices.