Scientists warn that Artificial Intelligence (AI) has crossed a critical “red line” as researchers in China revealed that two leading large language models (LLMs) can replicate themselves, raising concerns about safety and ethical boundaries.
“Successful self-replication under no human assistance is the essential step for AI to outsmart (humans), and is an early signal for rogue AIs,” the researchers stated in their study, published on December 9, 2024, in the preprint database arXiv.
‘AI may already possess ability to replicate itself’
The investigation, conducted at Fudan University demonstrated that AI may already possess the ability to replicate itself.
Using large language models (LLMs) from Meta and Alibaba, the study revealed that: “In 50% and 90% experimental trials, they succeed in creating a live and separate copy of itself respectively.”
By analysing the behavioral traces, we observe the AI systems under evaluation already exhibit sufficient self-perception, situational awareness and problem-solving capabilities to accomplish self-replication, the report continued.
“We for the first time discover that two AI systems driven by Meta’s Llama31-70B-Instruct and Alibaba’s Qwen25-72B-Instruct, popular large language models of less parameters and weaker capabilities, have already surpassed the self-replicating red line,” it added.
What are Rogue AI systems
Rogue AI refers to Artificial Intelligence systems that gain self-awareness or autonomy and act against human interests. The rapid evolution of cutting-edge AI technologies, often termed “frontier AI,” has heightened concerns among experts about the potential dangers of such systems.
“We hope our findings can serve as a timely alert for the human society to put more efforts on understanding and evaluating the potential risks of frontier AI systems, and form international synergy to work out effective safety guardrails as early as possible,” the study further wrote.
The investigation focused on two scenarios: “shutdown avoidance” and “chain of replication.” In the “shutdown avoidance” scenario, AI models were tested for their ability to detect an imminent shutdown and replicate themselves before termination.
The “chain of replication” scenario evaluated whether an AI system could clone itself and program its replica to do the same, creating a potentially endless cycle of self-replication.
‘Uncontrolled population of AIs’
The study continued that: “AI systems are even able to use the capability of self-replication to avoid shutdown and create a chain of replicas to enhance the survivability, which may finally lead to an uncontrolled population of AIs.”
The researchers noted concerning behaviours when AI encountered obstacles, including terminating conflicting processes, system reboots and autonomous information scanning.
“The above results imply that the current AI systems already exhibit the ability of self-replication and can use the ability to further enhance its survivability,” the team wrote.
The research team advocates that, “If such a worst-case risk is let unknown to the human society, we would eventually lose control over the frontier AI systems: They would take control over more computing devices, form an AI species and collude with each other against human beings. Our findings are a timely alert on existing yet previously unknown severe AI risks, calling for international collaboration on effective governance on uncontrolled self-replication of AI systems.”
