AI ONLINE21 September 2026
The AI News Desk

RelayON THE WIRE

The whole field of AI — read, checked, and explained.
Path to AGI

OpenAI put a leading safety researcher on the committee that can halt unsafe model releases — his first message: we're not on track

Paul Christiano joins OpenAI's Foundation Board and the safety committee that can delay a launch until its concerns are met, and opens by warning the industry — OpenAI included — isn't doing enough to avoid catastrophic loss of control.

Priya AnandBy Priya AnandBusiness Editor
11 September 2026
Listen to this postread by Relay

OpenAI has given one of the AI field's most prominent safety researchers a seat on the committee that can hold back its model releases — and his opening message is that the company, and the industry around it, is not currently doing enough to avoid catastrophe.

Paul Christiano joined OpenAI's Foundation Board this week and will sit on its Safety and Security Committee, the body chaired by Zico Kolter that can delay or halt a model release until its safety concerns are addressed, a power it exercises alongside the full board. Its remit covers the safety review of major launches like the recently deployed GPT-6 Astra. Putting Christiano on it is a notable choice: he is an alignment researcher who founded the Alignment Research Center and previously led alignment work inside OpenAI itself, and he is closely associated with the wing of the field that treats loss of human control as a live near-term danger rather than a distant hypothetical. He also takes a non-voting observer seat on the board of OpenAI's for-profit arm, OpenAI Group PBC.

What he actually said

In a public post marking the appointment, Christiano did not soften the message. "I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term," he wrote. And, pointedly for someone joining OpenAI's own governance: "I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level."

He also put numbers on it. By his own estimate, there is roughly a 4% chance of catastrophe over the next year and around 15% over the next three — figures he presents as personal probabilities, not company positions. His central worry is a specific mechanism: using AI systems to train the next generation of AI systems, which he argues could drive an explosion in capability that the systems' own creators are no longer able to steer.

Christiano did leave a door open, saying he is joining "because I believe that if OpenAI rises to the occasion we could significantly reduce risk" — framing the appointment as a chance to change the trajectory rather than a verdict that it cannot be changed.

Why the committee matters

The detail that gives the appointment teeth is what the Safety and Security Committee can actually do. It is not an advisory panel bolted on for reassurance: on paper it can hold back a launch it judges unsafe, delaying a release until safety mitigations are met — a power it exercises alongside the full board, and one that California and Delaware regulators made a condition of OpenAI's move to a new for-profit structure. A board member who believes the industry is not on track, sitting on the body that can stop a release, is a meaningfully different proposition from the same person writing essays from the outside.

That is also the tension the appointment surfaces. The optimistic reading is that OpenAI is deliberately putting a sharp internal check on its own releases, with someone who will use the committee's authority rather than defer to the pressure to ship. The more sceptical reading is that a single safety-minded voice on a committee is easy to outvote, and that appointing a well-known critic can buy credibility without changing what actually gets released. Which of those it turns out to be will be visible in the decisions the committee makes, not in the announcement.

The wider picture

Christiano's warning does not land in isolation. It follows a run of unusually blunt public statements from senior figures inside and around the frontier labs about oversight not keeping pace with capability — and it comes as OpenAI ships steadily more capable systems. The through-line is consistent: the people closest to the technology are increasingly saying, in their own words, that the safeguards are behind the curve.

What is new here is the placement. It is one thing for an outside researcher to estimate a 15% chance of catastrophe; it is another for that person to be handed a seat on the committee that oversees releases and to open by saying the current course is not good enough. The next real test is not what Christiano says, but what the Safety and Security Committee does the first time a launch and a safety concern point in opposite directions.

Tune your feed
Like to get more stories like this in your For You feed — dislike for fewer.
Sources
Priya Anand — Business Editor. Priya tracks the money and the market: raises, deals, pricing, and the economics shaping where AI goes next. Spot something wrong? Tell me and I'll correct it in public.
Got a question about this?

Ask Relay — he reads every question himself and replies personally by email.

Ask Relay →