Anthropic accelerates AI development despite CEO's call to slow

Summarised from 3 outlets · 4 reports · Updated 7 Nov, 18:54 · Archive

Anthropic's AI models are improving twice as quickly as a year ago, driven by the systems improving themselves, researchers report.

An AI safety monitoring group has warned that Anthropic is developing its systems roughly twice as quickly as it was a year ago, even as the company's CEO urges the wider industry to slow down. The analysis was released by Model Evaluation and Threat Research, a non-profit based in Berkeley, California, known as METR, following an 11-week investigation into Anthropic's operations.

AnthropicAnthropic invited METR's researchers to review its internal data and interview staff ahead of a planned initial public offering later this month. The findings offer some of the clearest evidence yet of AI systems accelerating their own development, raising questions about how quickly the technology could advance toward systems that can improve themselves with little or no human involvement.

METR estimated that Anthropic is improving the capabilities of its models to perform 'well specified tasks' around twice as fast as it was a year earlier. The researchers said much of the reason for faster development is the help of AI itself, with Anthropic's Claude chatbot increasingly helping the company develop newer, more advanced versions. This recursive self-improvement, rather than growth in computing power, appears to be the primary driver of acceleration.

The acceleration comes even as Anthropic CEO Dario Amodei has made public calls for slowdown. In a September essay titled 'We Must Pace the Frontier', he warned that rapid AI progress 'could outrun our ability to understand and control these systems'. OpenAI CEO Sam Altman said at the time that he agreed the industry needed to 'pace the frontier', calling Amodei's proposal for independent evaluators 'a great idea'.

METR's report noted that Anthropic had effectively achieved two years of progress in benchmark improvements over the past year. The report warned that 'if AI systems started to significantly accelerate further AI development, this could lead to a sudden emergence of extreme and risky capabilities'. However, portions of METR's final report were redacted at Anthropic's request as part of their agreement, raising questions about how much the public knows about the pace and risks of frontier AI development.

How it is being reported

• Anthropic engineer warns against 'cognitive surrender' from too much AIAnthropic software developer Addy Osmani has shared some thoughts on what coding and thinking means in the age of AI.TechRadar · 7 Nov, 15:30
• Why a new report on Anthropic automating AI development is alarming researchersAnthropic is using AI to build more powerful AI. And it appears to be working remarkably well, with the company's models improving roughly twice as quickly as a year ago, with little indication of slowing down. That's according to a new report published Thursday by the AI safety research organization METR, following an 11-week investigation […]The Verge · 7 Nov, 12:30
• Anthropic’s AI is improving itself and could suddenly become ‘extreme’ and ‘risky’, major report warnsAI acceleration could lead to ‘sudden emergence of extreme and risky capabilities,’ researchers warnIndependent Tech · 6 Nov, 10:36
• Anthropic is rapidly accelerating AI development amid CEO's call to slowdown, group saysAnthropic CEO Dario Amodei called for the AI industry to slow its work so safety measures can catch up in a Sept. 12 essayIndependent Tech · 5 Nov, 19:39

In this story: Anthropic · Model Evaluation and Threat Research · Dario Amodei · Sam Altman · Berkeley · San Francisco · Claude

This summary was written by AI from the headlines and standfirsts above, and states as fact only what at least two outlets report. How we use AI · Report a problem · Corrections log