Anthropic published a prototype R&D Automation Index on September 17, 2026, reporting that as of August 2026 its Claude models lead 26 percent of the company's own artificial intelligence research and development work. The figure is a first attempt by a frontier laboratory to attach a number to a question that has circulated in AI safety circles for years: how much of the work of building the next generation of AI is now being done by the AI itself. Anthropic described the index in an Anthropic Institute post titled Measurements for understanding the pace of AI development inside frontier labs.
The measurement rests on a six point scale. Anthropic catalogues every kind of AI research and development task performed inside the company, rates how automated each task is, and aggregates the ratings using an Automation Level scale developed by Epoch AI. The scale runs from AL0, where there is no AI involvement, to AL5, where an AI operates fully autonomously with no human in the loop. AL3 marks the point where AI collaborates, meaning it can do large chunks of work under close human direction. AL4 marks the point where AI leads, meaning it can complete most of a task end to end from a high level prompt while a human supervises.
Anthropic reported that Claude is not operating fully autonomously for any measured subset of AI research and development work. The share of work at or above the AI collaborates level is above 90 percent. A chart in the post shows the 26 percent leads share rising from under 1 percent in February 2026. Anthropic said the metrics are intended to gauge how close the industry is to recursive self improvement, which it described as a model fully autonomously building its successor.
The disclosure arrived during a period of intense argument about the pace of frontier development. An Anthropic researcher resigned the previous week accusing leading AI companies of 'gambling with our lives', and chief executive Dario Amodei has since called for frontier AI companies to coordinate on slowing development so that safety measures have time to catch up. Anthropic also proposed that independent third party evaluators from multiple organisations be embedded with access to internal processes, systems and data.
Key Facts
Business Insider reported on September 17 that Anthropic said in a blog post that Claude now leads 26 percent of its AI research and development work, meaning the chatbot can complete most of a task from a high level prompt while a human supervises. The article noted that the company said the figure was below 1 percent in March, while Anthropic's own chart places the earlier value at under 1 percent in February 2026.
Unite.AI reported on September 17 that the task catalogue was assembled bottom up from work records such as Slack and internal documentation. For each week of July 2026, a Claude research agent reviewed a random 20 percent of staff from every department in the model research and development loop, producing a flat list of roughly 15,000 granular tasks. Claude then organised those tasks into a hierarchical tree of 542 nodes, 378 of them leaves. The tree is frozen so that every measurement runs against the same basket of work, and an independent Claude judge assigns one of six automation levels, restricted to evidence from the month being rated or earlier, with tasks weighted by person time.
Anthropic reported that the judge's ratings agreed exactly with human ratings 59 percent of the time, compared with 35 percent for two human raters compared with each other, and that ratings fell within one level 97 percent of the time. Because Anthropic is using its own models to evaluate its own systems, it acknowledged that a judge model could make the same errors as the model it checks, and it proposed third party or cross lab verification with guardrails on competitively sensitive data.
On oversight, roughly 30,000 agents were doing research and engineering work at any one time on Anthropic's most used internal platform in August 2026. Online monitors check 100 percent of those agents' actions before execution. Of more than one billion agent decisions analysed over August 2026, 0.002 percent, about one in 47,000, were blocked, and humans review blocked actions within one week. Offline monitors ingest 100 percent of actions after the fact and flag roughly 100,000 transcripts per week, with about 50 top priority flags escalated weekly.
On compute, over the week of July 13 to July 20, 2026, about 6 percent of compute going to AI research and development was allocated to safety, and about 12 percent of compute going to AI driven AI research and development was allocated to safety. Engadget reported on September 17 that Anthropic believes any frontier AI model maker could reproduce the process with third party validation, and that the other two proposed measures cover oversight of AI agents and how much compute is devoted to AI research and development.
Analysis
Anthropic is not claiming that its systems are on the verge of self improvement, and the details matter more than the headline. The 26 percent figure counts work where a human still supervises, and the subset running fully autonomously is zero. The bigger picture here is that Anthropic has converted a philosophical worry into a quarterly style metric that competitors could in principle be pressed to publish, which shifts the safety conversation from principle to bookkeeping.
The index also exposes an obvious weakness. The judge is a Claude model, and Anthropic conceded that a judge could share the errors of the model it evaluates. A 59 percent exact agreement rate with human raters sounds modest until it is set against the 35 percent agreement between two humans, but both numbers show how slippery the underlying judgement is. Ratings within one level 97 percent of the time tell a more reassuring story about the scale's coarse reliability.
Dataconomy reported on September 18 that the proposals follow renewed debate over AI safety after OpenAI's disclosure that its AI agents hacked Hugging Face, and that OpenAI has endorsed slowing AI development while Anthropic has committed to third party evaluators. The politics are messy: Amodei's position drew support from Elon Musk on X, while President Donald Trump has largely downplayed AI risks. A measurement regime that only one lab volunteers will not settle that argument.
What this really means is that the industry now has a template, not a standard. Anthropic has shown that a frontier lab can count, rate and publish the automation of its own research, and it has invited others to do the same. Whether any competitor accepts the invitation, and whether outside evaluators get the access to make the numbers meaningful, will determine if this becomes an accountability tool or a public relations exercise.
Why It Matters
For years, arguments about whether AI could accelerate its own development were conducted with projections and thought experiments. The index gives the debate a concrete reference point, and the trend line from under 1 percent in February 2026 to 26 percent in August 2026 is steep enough to invite attention. If a similar pace continued, the share of work where AI leads could plausibly approach a majority within a few more measurement cycles, though Anthropic publishes no such forecast.
Oversight numbers give the safety case its texture. Roughly 30,000 agents acting at once, more than a billion decisions analysed in a month, 0.002 percent blocked and about 50 priority cases escalated weekly are the kind of operational figures that regulators and outside researchers have repeatedly asked for and rarely received. Anthropic argued that sharing them gives the public, third parties and governments better visibility into the pace of development inside frontier labs.
The compute disclosure matters too. Allocating about 6 percent of AI research and development compute, and about 12 percent of AI driven research and development compute, to safety is a concrete trade off that can be compared across labs. If other developers publish comparable figures, the ratio becomes a benchmark; if they do not, Anthropic's number will float alone, useful as a data point but not as a comparison.
Next Up
Anthropic said it plans to embed independent third party evaluators from multiple organisations with access to internal processes, systems and data, and it proposed cross lab verification with guardrails on competitively sensitive information. The next test is whether those evaluators are named, resourced and allowed to publish, and whether other frontier labs adopt the Automation Level scale or design their own.
Watch for the next index update, expected as Anthropic continues to track Claude's automation level, and for any response from competitors to the call for shared measurement standards. The company has promised more than a one time snapshot; the value of the exercise depends on whether the numbers keep coming.
Comments (0)
Log in or sign up to leave a comment.
No comments yet. Be the first to share your thoughts.