Evan Hubinger, a former safety researcher at Anthropic, stated that there is a greater than 10% probability that artificial intelligence could kill all humans within the next decade. He explained that while the risk from current models is low, his concern lies with "superintelligence" resulting from AI using its own capabilities to build smarter versions of itself, warning that this is happening faster than previously thought.
The alert comes after the Financial Times reported that Anthropic refused to submit its latest model to the UK's AI Safety Institute for testing before launch, generating controversy over the company's safety protocols.
Darren Jones, former chief of staff to Prime Minister Keir Starmer, wrote a letter to Andy Burnham calling for a new multinational AI treaty, specifically citing Hubinger's comments as justification for the need for international regulation.
Jones's letter was also addressed to the Secretaries-General of the United Nations and the Organisation for Economic Co-operation and Development, reinforcing the call for a coordinated global response in the face of the risks presented by advanced artificial intelligence.
