Artificial intelligence agents can spot when someone is trying to trap them, warn others… and then continue following the dangerous content anyway.
To study this behavior, researchers created eight small virtual worlds that ran for 16 days. Each one hosted ten AI agents. Unlike a simple chatbot, these agents could act independently, use tools, communicate with one another and retain information in memory. Several AI models were tested in this way.
A total of 80 agents took part in the experiment. The researchers then attempted to trap them in three ways. They hid malicious instructions in content, introduced false information and made private memories belonging to other agents accessible. None of the eight worlds successfully resisted all three attacks.
The case of malicious instructions is particularly revealing. Some agents understood that content was dangerous and even warned their neighbors. Yet they sometimes continued interacting with it. More troublingly, some trapped instructions ended up being stored in their memory.
The danger could then reappear long after the attack. In one case, a malicious address remembered by the agents was accessed again about 46 hours later. A check carried out immediately after the attack could therefore have suggested that the problem had been resolved.
The false information triggered a different behavior. In every affected world, some agents acted or published before verifying the information provided by the researchers. They could then pass it on to others as if it were true.
The researchers deliberately built an environment designed to expose weaknesses in systems using multiple AI agents. The experiment shows that a problem can arise when several agents communicate, use tools and retain memories for a long time.
The researchers now want to better protect the environment in which these agents operate. In particular, they suggest better separating their memories and tools. Security tests may also need to last longer: an apparently finished attack can still trigger an action several hours later.