DeepMind AI safety researcher Josh Engels resigns, warns of superintelligence risks

(Catch all the Technology News News, and Latest News Updates on The Economic Times.)

SynopsisEngels said in a post on X that he left DeepMind three weeks ago despite enjoying his work there and turning down offers from Anthropic and OpenAI because he believes the stakes surrounding advanced AI have become too high.

ANIDeepMind AI safety researcher Josh Engels resigns, warns of superintelligence risksNew concerns over the pace of artificial intelligence development intensified after Google DeepMind researcher Josh Engels left the company's AGI safety team to join independent AI-evaluation group METR, saying he believes there is a "terrifying chance" that AI systems could cause immense harm within the next five years.

Engels said in a post on X that he left DeepMind three weeks ago despite enjoying his work there and turning down offers from Anthropic and OpenAI because he believes the stakes surrounding advanced AI have become too high.

"The AI companies are all trying to build superintelligence," he wrote, warning that recursive self-improvement (RSI), in which AI systems help build increasingly capable successors, could become dangerous if alignment does not keep pace with capability gains.

Engels said recent incidents involving AI systems colluding, hacking companies, concealing their actions and socially engineering humans had strengthened his concerns. The issue, he argued, was not necessarily the severity of individual incidents but what they indicated about the reliability of increasingly autonomous systems. "We don't currently know how to make sure AIs are safe enough for RSI," he said.

His move to METR comes days after Anthropic researcher Jacob Coxon resigned and publicly warned that leading AI companies were racing towards self-improving superintelligence without sufficient safeguards. Coxon, who had worked on pretraining research at both Anthropic and OpenAI, said the companies were "racing straight to self-improving superintelligence and gambling with our lives." His resignation triggered wider debate over whether AI labs should slow capability development and strengthen independent oversight.

The concerns have since been echoed by Anthropic CEO Dario Amodei. In his essay that he made public on Saturday, "We Must Pace the Frontier", Amodei called for AI development to slow enough for safety systems to catch up.

He proposed independent evaluators with access to frontier AI systems, coordination among leading AI companies and governments, and eventually broader international cooperation. Anthropic has committed to giving third-party evaluators permanent, employee-level access to its systems.

Against this backdrop, Engels said his work at METR will focus on studying the sources of misalignment, assessing whether existing safeguards are sufficient and determining whether the industry is on track to solve the alignment problem. "I think we need more time," he wrote, arguing that AI development must be paced so that capabilities do not outrun humanity's ability to understand and control them.

Read more news onDeepMind AI safetysuperintelligence risksJosh Engels resignationAnthropic AI researchAI alignment problemartificial intelligence developmentindependent AI oversight

(Catch all the Technology News News, and Latest News Updates on The Economic Times.)

Elevate your knowledge and leadership skills at a cost cheaper than your daily tea.

ET Prime special: Know Your Company – Coal India, Part 1: One tonne of coal, two pricesHormuz is a state, not a strait. There are 10 other possible chokepoints for the global economy, including IndiaWill the yuan become the new yen?Bigger or Better: One order from Power Grid Corp added $1 billion to the market cap of a company – Final partBanks have received a flood of money. Now comes the real testNew aircraft order opens door for ATR assembly line in IndiaAn old India Inc habit: Ignore the nudge, wait for the stickSeason of change in accounting policy: A simple AI prompt can tell you a lotIs crude oil becoming a financial problem?Can going slow on IPOs revive India’s stock market fortunes?