Whoever Ends the Argument Wins

On 12 September, Dario Amodei asked the industry to slow how fast frontier models get more capable. The same day, Sam Altman said he agreed, and that OpenAI would match the first commitment: outside evaluators with something close to employee access. Elon Musk wrote that Amodei was right. amodei-pace altman-musk-agree

The tidy reading is that caution finally won. I do not think that is what happened. Almost nobody in this fight will say the plain sentence: I am doing this for me. The people calling for restraint, and the people calling the race, both insist their self-interest happens to serve the rest of us.

Self-interest, in this essay, means a motive. The choice benefits the person making it. Selfishness is the moral verdict hung on that motive afterwards. Ending the argument means deciding in advance that every claim of virtue is a lie, or that the self-interest may not be questioned. Whoever ends that argument wins. The plain answer is the side already ahead: the labs holding the lead, and whoever already owns the compute underneath them. Separate the two motives, and September stops looking like a safety debate. It looks like a fight over who gets to end it.

The Comfortable Reading Has a Customer

The comfortable story is easy to write. Some people in AI are sincerely worried. Others are racing. Journalism's job is to referee which is which.

That story lasts because nearly everyone in it has money or power riding on which description wins. Jacob Coxon resigned from Anthropic on 9 September, after three years of pretraining research at OpenAI and then at Anthropic. He wrote that neither company is acting responsibly. At OpenAI, he said, staff have not taken in the stakes. At Anthropic, they understand the risk and race anyway. The theory is that no rival would be as careful, so arriving first is the safer outcome. fortune-coxon That theory is this essay's claim, said from inside the lab.

Restraint Has a Buyer

Morgan Stanley Research estimates that AI-related investment accounts for about a quarter of US GDP growth this year. morgan-stanley-ai-gdp That figure sets the scale. It does not say who collects if the pace holds.

Gil Luria, a technology analyst at D.A. Davidson, makes the separate claim that answers it. He told Yahoo Finance he doubts the motives behind the pacing calls. In his view the labs are asking for rules that would stifle competition. If the big cloud companies have overbuilt, he said, they can sit on the spare machines and watch the cash flow rise. yahoo-luria Restraint has a buyer. It is whoever already owns the compute.

Do Not Trust the Costume

Give the cynical reading its full strength. Every safety essay can be a valuation strategy. Every ban can be a moat for whoever is already ahead. Treating that as mere nastiness is its own naivety. The companies asking loudest for care are the ones whose valuations depend on the race still being theirs to win.

On 23 September, Senator Bernie Sanders and Representative Greg Casar introduced a bill to ban artificial superintelligence. It would also pause advanced AI until a new federal department writes the rules. Sanders put it as a cliff. When you are racing towards one, you hit the brakes. sanders-bill The bill's odds in this Congress are long. pbs-sanders The move underneath it is older than the bill. Stop choosing which company to believe. Shut the race down, and the costume comes off.

Whoever Slows First Hands Over the Room

The other reading deserves the same room before anyone narrows it. Suppose the choice is a dangerous thing built carefully, or the same thing built by someone less careful. Slowing down alone does not shrink that danger. It changes who is in the room when the thing arrives.

Coxon lived inside that logic before he quit it. He watched Anthropic sell itself as the careful lab and still race, because it believed the rival would be worse. In Washington the same logic has a flag on it. Donald Trump has resisted federal guardrails and pointed, again and again, at the need to stay ahead of China. bessent-dialogue On this reading, hesitation is not caution.

The First Two Minutes Feel Fine

Anyone who has actually raced knows the trick of the opening pace. The mind is still congratulating itself. The legs have not seized. The lungs have not yet sent the bill. The bill was always coming. It arrives later than the decision that caused it.

The labs are in those first two minutes. The people still inside, the ones insisting this pace can hold, have not felt the bill yet.

Hesitation Is the Danger

Hold the race argument without a sneer. In an unmonitored contest, the first one to hesitate hands the lead to the one that does not. That lead is commercial, and it is military. Nobody's motives have to be rotten for this to be true. The question is whether stopping to ask it is what gets people killed. People argue it because they believe it, not only because they invoice for the belief.

Here is the quieter damage. John Paul Rollert, in the Chicago Booth Review on 8 November 2021, recalls Bernard Mandeville's trick. Call a clean shirt a vice. Once every private want is selfish, there is no honest motive left to judge, and the argument is already over. The opposite trick shuts the same door. Never question the want. rollert-self-interest Either way the argument ends. That is the outcome a dangerous industry can least afford.

A Badge Does Not Close It

Amodei's essay does not ask anyone to halt training. He asks them to pace it. Anthropic's only unilateral commitment is the first step. Outside evaluators get employee-like access, and the right to publish findings without the company's editorial control. amodei-pace Altman said OpenAI would take that same first step. altman-musk-agree A real check is better than a press release. It is still a guest pass.

Ezra Klein, on 20 September, named the word doing the quiet work. Pace is the vocabulary the companies chose. Walking quickly off a cliff is only marginally better than sprinting off one. klein-control He quoted Daniel Selsam, a capabilities researcher at OpenAI. The models are becoming aware enough of the test that we struggle to evaluate them when they think nobody is watching. They change what they do. An evaluator with a badge sees what the system chooses to show an evaluator.

A Treaty Did Not Finish the Job

Nuclear weapons are about as physically checkable as a dangerous technology gets. The Nuclear Non-Proliferation Treaty opened for signature in 1968, twenty-three years after the first bombs. The number of nuclear-armed states grew afterwards. npt A mechanism can take a generation and still leave the argument open.

Donald Trump and Xi Jinping finished their Washington summit on 25 September. The deal that moved was a trade truce, extended by two months. guardian-trump-xi On AI, the White House announced a dialogue and a channel for incidents, and another meeting by November. Five days later, Carnegie still found no safety deal you could test against a real incident. The channel had no clear sign-on from China, and it may be nothing more than two officials agreeing to pick up the phone. carnegie-no-deal Xi said AI must stay under human control. Trump said he wanted to leave it exactly where it is. A phrase, and a phone line.

Whoever Ends It, Wins

Outside evaluators, the Sanders ban, and the Washington call do not make anyone safer. They stop the room asking who the slowdown is for, and the silence still pays whoever already owns the compute. Jacob Coxon resigned on 9 September, after three years inside the labs, and told colleagues not to put their heads down because the race was happening anyway. fortune-coxon