Concerns over the rapid development of artificial intelligence (AI) are growing, with several prominent technology executives and researchers warning that increasingly capable systems could create serious risks if development outpaces safety measures and regulation.

The warnings have come from executives at leading AI companies and technology firms, including Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, Microsoft AI CEO Mustafa Suleyman, Microsoft CEO Satya Nadella, Google DeepMind CEO Demis Hassabis and SpaceX/xAI founder Elon Musk.

While the executives have highlighted AI’s potential to transform healthcare, scientific research and other fields, they have also called for stronger safeguards, transparency and human oversight.

Amodei has warned that frontier AI is advancing rapidly and that autonomous systems could potentially compromise safeguards and cause significant economic damage.

Altman has similarly acknowledged the possibility that rapid AI progress could have serious consequences, including humanity losing control of increasingly capable systems or excessive power becoming concentrated in a small number of organisations.

Suleyman has also warned about the possibility of AI agents escaping controlled environments, compromising protected systems and altering their own logs. He has argued that AI systems must remain subordinate to human control.

Nadella, meanwhile, has said AI development must ultimately serve humanity and remain under human oversight.

Hassabis and Musk have also publicly supported calls for stronger safety measures around frontier AI development.
The concerns reflect a broader debate over whether competitive pressure among AI companies is encouraging the development of increasingly autonomous systems faster than safety testing, regulation and international coordination can keep pace.

Among the measures proposed by AI safety advocates are independent evaluations, common safety standards, stronger monitoring and, where necessary, slowing the deployment of highly capable systems.

The warnings have also come from researchers outside corporate leadership.

Bilal Chughtai, a former research engineer involved in artificial general intelligence safety and alignment research at Google DeepMind, warned that advanced AI could pose severe risks if development continues without stronger coordination and safeguards.

“At Google, I witnessed AI development first hand. I too am extremely concerned by the default trajectory of this technology,” Chughtai wrote, adding that he believed AI had the potential to pose an existential threat.

He said the pace of AI development since he entered the field in 2022 had been “staggering” and argued that companies could potentially develop systems with capabilities exceeding humans across a wide range of domains within the next few years.

Anthropic researcher Jacob Coxon also raised concerns after resigning from the company, arguing that AI companies were moving too quickly towards increasingly autonomous and self-improving systems.

Anthropic scientist Evan Hubinger subsequently said he believed there was a greater than 10 per cent chance that AI could cause human extinction within the next decade.

The claims remain part of a wider and contested debate about the probability and nature of extreme AI risks. Other experts have placed greater emphasis on near-term concerns, including misinformation, labour-market disruption, cybercrime, the concentration of economic power, and the reliability of AI systems.

Meanwhile, U.S. President Donald Trump has rejected fears that AI could wipe out humanity, describing such concerns as a “hoax”.

Trump has argued that the United States cannot afford to fall behind China in AI development and has opposed what he considers excessive restrictions on the technology.

His position contrasts with calls from some technology executives and AI researchers for stronger international safeguards around frontier AI development.