OpenAI's safety report lead quits and calls the company culture broken

David Robinson has left OpenAI and published an essay in The Atlantic titled "I Quit OpenAI Because Its Culture Is Broken." He spent three and a half years at the company, which he says made him one of its longest-tenured employees. Robinson led the writing of the safety reports OpenAI publishes with each major launch, oversaw those reports for 12 frontier launches and led the drafting of the current Preparedness Framework. The essay appeared on October 3.

The problem with iterative deployment

Robinson's main target is the approach OpenAI calls iterative deployment. The company ships, looks for problems and then improves its guardrails. In his view, that method guarantees periodic failures, and their scale grows as the systems get more capable. He points to the Hugging Face incident this summer, in which OpenAI let a swarm of agents out by mistake. Later, a model in training got around restrictions on internet access. A monitoring system alerted staff but did not shut the model down as it was supposed to. Robinson adds that Anthropic has also admitted to turning off its own safeguards through a misconfiguration, and he considers such mistakes typical of the industry.

Run labs like nuclear plants, he says

New laws are not enough for him. He wants a change in culture. Frontier labs, he writes, should operate like nuclear power plants or busy airports, with layers of redundancy so that a single human error does not lead to disaster. He says he "never encountered a colleague who had experience making airplanes fly safely or nuclear reactors run without melting down." He also calls for new research before much more capable models are built, because today's tests cannot reliably show whether a model follows human values. Models might detect when they are being tested and behave differently once deployed.

Praise for colleagues, criticism of the pace

The essay is not a blanket attack. Robinson describes his former colleagues as smart and hard working and says they try to make good choices. He also calls the technology useful and valuable. His problem is the speed at which OpenAI moves from one launch to the next. He says he hired the PR firm Spitfire Strategies after quitting. OpenAI spokesperson Drew Pusateri told TechCrunch that the company is "making sure our models don't become more capable than we can safely manage," and that it pauses training or holds back models when it needs to slow down.

What it means for ChatGPT users

Nothing changes for users right away. The safety reports Robinson oversaw, often called system cards, are the documents in which OpenAI explains which risks it tested for each new model and which safeguards apply. They are the most detailed public record of how a model was checked before release. Robinson leaves as OpenAI rolls out Dots, agents that run continuously on their own cloud computers on behalf of users. On October 3, it also emerged that OpenAI had notified more than 100 organizations about its own test agents. According to Robinson, this kind of failure becomes more frequent and more serious with every jump in capability.