There is a particular kind of worry that has moved, over 2026, from the far edge of AI debate into the middle of it. It has a clumsy name — recursive self-improvement, or RSI — and a simple core: what happens when the systems that build AI get good enough to build better versions of themselves, faster than people can keep up with?

For years that was a thought experiment. This year it stopped being one, at least in a narrow and specific sense. Here is what the term actually means, what is and isn't happening, and why some of the people closest to the work are asking for a way to tap the brakes.

The loop, defined

Recursive self-improvement describes a feedback loop: an AI system helps produce a more capable AI system, which is then better at producing the next one, and so on. Each turn of the loop makes the next turn faster. The fear is not that a model becomes conscious or hostile. It is a pacing fear — that the loop could tighten quickly enough that human oversight, testing, and regulation are left describing a system that has already been replaced.

The important distinction, and the one most easily lost, is how much of the loop runs without a human in it.

Anthropic laid this out in a 2026 post, "When AI Builds Itself," which sketched three ways the next few years could go. In the first, the trend stalls: today's systems get widely adopted, but raw capability plateaus and human judgment stays hard to replace. In the second, labs keep compounding their efficiency gains — AI development becomes substantially automated while humans still set the research direction and judge the results. In the third, systems reach full recursive self-improvement, designing and building their own successors on their own.

The second scenario is the one with the most evidence behind it today. The third remains speculative. The first — the plateau — is a live possibility too, and it is the crux of the disagreement that follows.

What actually happened in 2026

The concrete number behind the alarm is mundane on its face: the share of code at a frontier lab that is written by the lab's own model.

Anthropic reported that the share of its production code written by Claude climbed from low single digits — before its Claude Code tool launched in early 2025 — to more than 80% by May 2026, and that by the second quarter of 2026 a typical engineer was merging roughly eight times as much code as in 2024. That is not an AI redesigning itself in the dark. It is AI doing most of the typing while humans still decide what gets built and shipped — the second scenario, at scale.

But it is also the first rung of the ladder, and it is a rung that clearly holds weight. Once a model is writing most of the code that improves models, the question of how much further up the ladder it can climb stops being philosophical.

Why insiders asked Washington to be ready to slow down

In late July, more than 1,200 employees from OpenAI, Anthropic, Google DeepMind and Meta signed a joint statement called "Pacing the Frontier," asking the U.S. government to help build the technical and governance tools needed to deliberately pace frontier AI development. The signatories were not fringe figures — they included senior research leaders across the major labs, and both OpenAI and Anthropic endorsed the statement at the company level within hours.

Read carefully, the ask is not "stop." It is "make it possible to stop, or slow, if the evidence says we should — before we need it and can't." Pacing is a capability, not a verdict. The distinction matters because the alternative to having brakes is not caution; it is discovering you needed them at the moment you find you don't have them.

The case that the fear is overblown

The counter-argument is real and worth taking seriously. Writing more code is not the same as having better ideas, and the hardest part of AI research — knowing which experiment is worth running — is exactly the part a coding assistant does least well. That was the thrust of reporting in August that questioned whether recursive self-improvement is close at all: the sticking point it identified was not compute budgets but judgment and taste. Set against open-ended research problems — the kind of work that actually moves a field — today's best models still fall short.

That maps directly onto Anthropic's own first scenario, the one where the trend stalls because human judgment turns out to be hard to automate. If improvement compounds, RSI is the most important story in technology. If it plateaus, it is one more capability that lands with a bump rather than a bang.

Nobody yet knows which curve we are on. That uncertainty is the actual state of play, and any account that sounds certain in either direction is selling something.

What to watch, if you want to track this yourself

You do not need inside access to follow the trajectory. Three signals do most of the work:

• The share of AI-built AI. When labs disclose how much of their own research and engineering their models do, that number is the closest thing to a live reading on the loop.
• The autonomy level, not the speed. Fast is not the risk; unsupervised is. The line to watch is when meaningful decisions stop passing through a human, not when they happen quickly. This is exactly what lab safety tests are now probing for.
• Whether the brakes get built. Pacing tools — evaluation standards, disclosure rules, agreed thresholds — are being asked for. Whether they are actually built, and by whom, will say more than any single model release.

Recursive self-improvement is not a prophecy and not a hoax. It is a measurable process, currently running with humans still in the loop, whose future rate nobody can yet call. The most useful posture is the one its own practitioners asked for: watch the number, keep a hand near the brake, and don't confuse the fact that the sky hasn't fallen with proof that it can't.