If AI can code, can they create other AIs themselves? 🤖

By Julien Romero - Lecturer in Artificial Intelligence, Télécom SudParis – Institut Mines-Télécom

Artificial intelligence systems are capable of writing lines of code and controlling a computer. What prevents them from creating other AIs?

At the end of October 2024, Anthropic unveiled Computer-Use, a program allowing their AI model Claude to control a computer like a human. What would happen if an AI could also access financial resources to acquire additional machines and services? This hypothesis, although exaggerated, raises a fascinating question. Could an AI truly become autonomous and create other AIs without human intervention?


As we will see, large companies like OpenAI, Facebook, or Google already use AIs to train increasingly complex AIs, and this is no secret, not even to the AIs themselves.

AIs training AIs

To understand how this is possible, we need to go back and explain what enabled recent progress. It all started in 2017 when a team of researchers at Google published a scientific paper: "Attention is all you need." In this publication, the researchers introduced a new neural architecture called "Transformers" that learns which words to "pay attention to" in order to generate the next word. This Transformers architecture now structures all the neural networks of modern text-generating AIs.

The emergence of Transformers led OpenAI to launch the first version of GPT in 2018 to generate text. Although the fundamental principles have evolved little since then, the scale and ambition of "large language models" (or LLMs) have exploded.

Thus, in May 2020, the arrival of GPT-3 marked the beginning of a category of AI capable of modeling human languages using gigantic neural networks, whether natural languages like French or formal ones like C++ in computer science. Note that modeling with statistics does not mean understanding with cognitive processes, and these AIs still produce absurd answers to trivial questions.

The models then went from 1.5 billion connections for GPT-2 to a few hundred billion for GPT-3 and its successors, which corresponds to moving from the brain of a bee to that of a hamster in terms of the number of synapses. However, the increase in their size has slowed in recent years, and it is no longer the main driver of progress today.

We need to look instead at the methodological changes taking place before and after model training.

More and better quality data

The training of LLMs relies on texts serving as references to teach them to predict the next word in a sentence. To improve this learning, more and more data is used: GPT-2 was trained on 30 billion words (organized into sentences, paragraphs, and texts), compared to eleven trillion for LLaMa-3.

However, not all texts, mainly coming from the web, are of the same quality. Engineers therefore use cleaning algorithms and, more recently, LLMs themselves to improve, rephrase, or generate this data (for example for LLaMa-3 or Qwen 2.5).

Thus, while AIs are already participating in the training of other AIs, this practice remains limited by the slowness of LLMs. GPT-4 would take about 17,000 years to generate eleven trillion words on its own (about 500 terabytes of data).

Once the data is collected, cleaned, and generated, the actual learning phase begins. This phase remains difficult to implement and requires a colossal amount of computing resources, but little has changed since the first version of GPT in 2018.

Guiding AI learning by providing constructive feedback

On the other hand, researchers have focused on improving an LLM after its training. Indeed, one of the concerns of a raw LLM is that it is unpredictable and does not necessarily correspond to human needs in terms of skills (recruitment, medical diagnostics, mathematics) or ethical and social behaviors (politically correct chatbot, non-discriminatory, and law-abiding).

The idea is therefore to calibrate LLMs so that they better conform to the preferences of their users. To do this, the technique of reinforcement learning from human feedback asks humans for their opinions on generated texts and trains LLMs to please humans.

This process allowed a major leap forward in 2022 with InstructGPT, a precursor to ChatGPT. However, it is extremely costly as it requires a lot of manual work. LLaMa-3 required the annotation of ten million preferences by humans. These workers are often underpaid and in precarious situations.

This is why researchers are seeking to minimize human assistance.

When AIs train AIs

In July 2024, a team of scientists from Microsoft published AgentInstruct, a new method for teaching new skills and behaviors to LLMs.

This method focuses on the creation of "agents" specialized in many fields (mathematics, code, medicine) serving as teachers to the system being trained. In this case, an agent is itself an LLM, but augmented with additional external data and tools, such as a calculator, the Internet, or a computer code compiler. Better equipped and specialized than an LLM alone, it excels in its field of expertise. AgentInstruct uses a battalion of agents to teach their knowledge to an LLM.

The result: the LLM progresses without access to any other resources, unlike the agents. For example, an agent equipped with a calculator can improve the mental calculation of an LLM.

In the same way, thanks to the Computer-Use program, Claude could exploit many computer tools to collect, clean, and organize its own data, or even train AI models more autonomously by mobilizing specialized agents. Ask it how it could improve itself, and it will likely give you a similar answer (or suggest hiring an army of humans to annotate data).

But then, how to explain that it is not yet capable of reproducing and improving itself?

Before an AI capable of reproducing itself, a long technical path and ethical questions

This ability to create specialized agents raises crucial questions. Who controls the agents? If AIs participate in their own improvement, how to ensure that their evolution remains ethical and aligned with human interests? The role of developers and regulators will be central to avoiding potential abuses.

We are not there yet for several reasons. Current LLMs, although powerful, are limited: they struggle to plan complex projects, require constant adjustments during their training, and still largely depend on human intervention, particularly in data centers, to manage and maintain physical machines.

Moreover, without their own will, they cannot set autonomous goals, independent of learned human preferences. Sam Altman, CEO of OpenAI, mentions the possible emergence of a general artificial intelligence as early as 2025, but this prediction remains controversial, as it would require technical breakthroughs and a better understanding of human cognitive mechanisms.

The success of LLMs relies on four pillars: increasing their size, architectural innovations, improving calibration techniques, and perfecting data. Recent advances, particularly automation via specialized agents, already show that AIs are playing an increasing role in the creation of other AIs. However, without their own will or true autonomy, the idea of an AI capable of multiplying or improving itself independently remains science fiction.

Indeed, a revolution of this magnitude would require a paradigm shift, with neural architectures capable of truly adaptive and generalized intelligence. Currently, once the learning phase is over, the neural networks of LLMs become fixed: they can no longer evolve or acquire new skills autonomously, even after millions of interactions with human users.

Unlike humans, who learn through contact with others or through internal reflection, LLMs do not have mechanisms to dynamically adapt their internal structure or build deep and revisable representations of the external world. Yann LeCun, the 2019 French Turing Prize winner, imagines a new generation of AIs equipped with internal models, capable of simulating hypotheses and planning like a human, integrating observations to compare them with pre-existing expectations. However, the practical implementation of this vision remains a scientific challenge.

Perhaps a breakthrough as decisive as that of Transformers in 2017 will occur in the coming years. But for now, the vision of fully autonomous artificial intelligences, akin to Von Neumann probes colonizing the universe, remains hypothetical.

This scenario, however, invites us to reflect today on the ethical issues and the legislative and technical safeguards necessary to frame the evolution of these technologies.