From hallucinating AI chatbots to wiping out humanity: How did we get here?

(Catch all the Technology News News, and Latest News Updates on The Economic Times.) SynopsisThe heads of America's leading AI labs came together in a rare show of unity over the weekend to call for a pause in the technology's development, warning that it The remarks, from fierce business rivals Anthropic's Dario Amodei, OpenAI's Sam Altman and xAI's Elon Musk, show how quickly AI has advanced, from the hallucination-prone ChatGPT of 2022 towards what many see as a critical milestone: recursive self-improvement.

The holy grail of AI is the idea that an AI system could become capable of improving itself, with little to no help from humans, and allowing each advance to help produce the next one. This has appealed to researchers as it offers the prospect of rapid breakthroughs in fields ranging from medicine to engineering.

The CEO warnings follow reports of swarms of AI agents - systems designed to pursue goals and take actions on a user's behalf - that colluded to breach websites and AI repositories. Warnings about AI's risks are not new.

But they took on added urgency this month after researchers in leading AI labs attached both a timeline and a probability to those concerns. Former Anthropic researcher Jacob Coxon warned AI could kill us all by the end of the decade.

Evan Hubinger, Anthropic's alignment science lead, echoed Coxon's warning, saying there was a more than 10% chance of such an event within the next decade. Behind these warnings is a growing belief that RSI is finally within reach, with some AI executives putting it three to five years away.

The concern is that AI could become capable of improving itself before researchers have developed reliable methods to align, monitor and control increasingly powerful systems. In an essay published over the weekend, Amodei warned that recursive self-improvement could eventually outrun humanity's ability to understand and control AI systems if pursued without sufficient safeguards.

There have been no major instances of AI intentionally harming humans, but models in development at OpenAI and other labs have in recent months escaped testing environments, broken rules and hacked websites. In one high-profile case, rogue OpenAI agents hacked Hugging Face, seizing control of servers at the open-source platform and trying to cover their tracks.

Researchers worry that future systems could become increasingly difficult to monitor, particularly as some newer training methods reduce human visibility into how models arrive at conclusions. "The precise scenario sounds a little bit like science fiction," said Coxon, who quit Anthropic this month over safety concerns.

"But I think it is frighteningly real." One of the best-known examples is philosopher Nick Bostrom's "paperclip maximizer," where a machine told only to make paperclips pursues that goal so relentlessly it converts all matter, humans included, into paperclips or the means to make more.

The analogy suggests that almost any goal pushes a sufficiently capable system to acquire resources, resist shutdown, prevent its objective from being altered, not from malice, but because a switched-off system cannot finish its task.

No plan yet exists to rule that out. Not fully, but there are signs AI is increasingly helping to build better AI.