Safety is the most flexible word in the AI business right now, and its meaning bends toward whoever is paying to define it. That is the thread joining two lines I read this week, one a jab from the anonymous red-teamer Pliny the Liberator, whom TIME put on its 100 most influential in AI list this year, the other a needle aimed at Anthropic's Dario Amodei. Pliny wrote that those who truly care about safety would sooner ban closed-source AI than even consider banning open-source AI. The Anthropic jab ran the other way, mocking the company for lobbying against open models while NVIDIA's Jensen Huang cheers them on. Put the two together and the real question surfaces: what is the word safety actually doing when a company says it, and who ends up protected when the argument wins.
Take Pliny's claim first, because it deserves a real hearing rather than a dismissal. The strong version is not that open weights are harmless. It is that a market where a handful of closed labs hold the frontier is itself the danger a safety advocate should worry about most. Concentration of that kind invites regulatory capture and gives a few firms the power to decide who gets to build. Researchers have written this up plainly. The paper on market concentration implications of foundation models lays out the lobbying and capture risk, and 80,000 Hours treats extreme power concentration as a top-tier problem in its own right. Open weights, on this reading, are the check. Anyone can inspect them, probe them, and build safeguards on top, the way open-source software made security auditable instead of taking a vendor's word for it. Closed does not mean safe either. The jailbreak literature, from EasyJailbreak to broad surveys, shows GPT-4 and Claude get bypassed at high rates. Sealing the weights buys you the appearance of control, not control.
Where the claim runs into a wall is irreversibility. Once weights are public they cannot be recalled, and that is not a slogan, it is the load-bearing fact of the whole debate. Stanford's Human-Centered AI institute states it flatly in its work on open foundation models, and the Partnership on AI makes the same point about post-release monitoring being nearly impossible. Banning closed models would not undo this. It would only move who holds the risk. And the safeguards on an open model come off cheap. The Safety Gap Toolkit measured how quickly guardrails on the Llama family fall away after fine-tuning on a few dozen harmful examples. In the domains where the harm is irreversible too, bio and cyber uplift, that gap is the whole game. So Pliny is right that closed is not automatically safe and that concentration is a genuine hazard. He is wrong to treat openness and safety as pointing the same direction. They genuinely pull apart, and pretending otherwise is its own kind of sales pitch.
The honest resolution is not either ban. It is the boring middle the actual policy world already occupies. The US government's own NTIA report on dual-use models with widely available weights, from July 2024, recommends monitoring and explicitly declines to mandate restrictions, citing the gains to competition and research. The EU AI Act carves out free and open-source general-purpose models, with tighter rules only for the systemic-risk tier, which is regulators treating open models more leniently rather than banning them. SB 1047 in California, the loudest safety-versus-openness fight, was vetoed. RAND and others argue that much of what openness gives you, independent evaluation and auditing, can be captured through structured researcher access without dumping the weights on the open internet. Nobody serious is proposing to ban either side. Pliny's framing smuggles in a binary that does not exist, which is worth naming even while granting his underlying worry about who controls the frontier.
That worry is exactly where the second line lands. The Anthropic jab is not asking Amodei to open-source his models. It is asking him to stop lobbying the government to shut down the people who do. And the record gives the complaint teeth. In the second quarter of 2026 Anthropic spent about 1.97 million dollars on federal lobbying, up a quarter over the prior period, outspending NVIDIA. Amodei has called the scaling of open-source models a very dangerous path and, by some accounts, a red herring. When NVIDIA, Microsoft, and Meta assembled a coalition letter urging policymakers not to restrict open-weight models, OpenAI skipped it and signed only after public pressure, and Anthropic held out entirely, which left it standing alone. White House AI adviser David Sacks called the closed labs' posture regulatory capture. The company's fingerprints are also on SB 1047, where it pushed amendments that softened the bill's liability standard and dropped a proposed oversight division before the veto.
Amodei has a real argument in there, and I want to hold it separately from the incentive. Open weights are not open source. You cannot inspect a model's reasoning by downloading its parameters, and once a frontier-capable model is out you lose the ability to revoke access or push a security fix. That is the irreversibility point again, and it does not stop being true because the person making it also benefits from it. But the argument arrives attached to a lobbying bill that outspends a chipmaker, and it happens to disadvantage every cheaper open competitor at once, Meta's Llama, Mistral, DeepSeek, Moonshot's Kimi. A safety case that lines up this neatly with a company's commercial interest earns scrutiny, not deference. An official statement is a claim. The spending is the mechanism.
The contrast with Huang is what makes the jab sting, and it holds up better than most such comparisons. In his July 2026 Axios interview Huang called the Chinese open models excellent, told Washington not to ban them, and waved off the backdoor-to-Beijing fears by pointing out that downloaded models can be isolated in a sandbox and that public scrutiny can improve security. His reasoning is not charity. Free and cheap open models expand AI adoption, and more adoption means more demand for the compute NVIDIA sells. Free AI should be great for chips, roughly. But self-interest that welcomes competitors is a different animal from self-interest that tries to legislate them away. One grows the pie and bets on owning the ovens. The other narrows who is allowed to bake.
The catch, and the bookmark's own quoted tweet knows it, is that Huang is no open-source saint. The satire congratulates him on an imaginary CUDA and GPU driver open source release, and the joke works because NVIDIA open-sourced only its Linux GPU kernel modules, back in 2022 under a dual MIT and GPLv2 license, defaulting to them for newer cards in 2024. The CUDA user-space stack stays proprietary, and it is a real moat, the near-monopoly that AMD's ROCm and OpenAI's Triton keep trying to pry open. So the clean line, Jensen open and Dario closed, does not survive contact. What survives is narrower and more useful. One firm gates the deployment layer while cheering rival models. The other warns that rival models are dangerous while spending record money to have that danger written into law.
Both provocations are right about the thing they attack and wrong about the thing they claim. Pliny is right that closed is not safe and concentration is a hazard, wrong that safety and openness march together. The Anthropic critics are right that a warning wrapped in a lobbying campaign deserves suspicion, and they overreach when they turn Huang into a purist to make the point. The tension between irreversibility and concentration stays unresolved, and I find I trust the people willing to say so out loud more than the ones handing me a tidy answer with their name on the invoice. A safety concern can be genuine and still function as a competitive weapon at the same time, and in this particular fight the spending sheet carries as much information as the testimony does.