THE MACHINES ARE ON THE ASSEMBLY LINE! A new report from the AI safety group METR, published Thursday after an 11-week investigation inside Anthropic, says the company is using AI to build more powerful AI, and it is working, The Verge reports.
The headline finding: Anthropic’s models are improving roughly twice as quickly as a year ago, with benchmark scores suggesting “that Anthropic has made roughly two years of historical progress over the last year.” METR said it is “confident” that isn’t just more computing power. As of October, the majority of the acceleration “is due to AI.”
Staff using Anthropic’s most capable internal model self-reported being around 12 times as productive as they would be without AI, about double what they reported a few months earlier. METR also found three unreleased models (codenamed Model 2, Model 3 and Model 4) that finished training between July and October, and said it saw no clear signs of diminishing returns.
Now the catch. Anthropic was allowed to redact the public version, and it did. “We submitted a ~100-page full report on our assessment to Anthropic. For the public version, Anthropic redacted a majority of the evidence we found most informative,” METR wrote on X. Anthropic cofounder Sam McCandlish said sharing what drives the speedup is commercially sensitive and “risks handing out a roadmap for others to go faster.” He said Anthropic will publish its own report in the coming weeks.
Critics aren’t buying it. Former OpenAI researcher Daniel Kokotajlo accused the company of “recklessly gambling with our lives.” New York Assemblyman Alex Bores called on Congress to “ban RSI,” meaning AI systems that improve themselves, “this year if possible.” Apollo Research CEO Marius Hobbhahn had the bluntest forecast: “Brace for craziness. It is going to go very fast.”
Featured image: AI-generated editorial illustration by Bugle Blast.
Source: The Verge
Leave a Reply