Media

The Number and the Thing It Measures

Nine claims that crossed my feed, and what each one looks like one layer down

Manish Singh/July 9, 2026/5 min read

A benchmark score is a claim about the world, not a fact about it. That distinction disappears the moment a company draws its own chart, on its own axes, and posts it as proof.

OpenAI did exactly that. The chart is titled Agents' Last Exam, and it shows GPT-5.6 Sol topping out at 53.6, beating Claude Fable 5 by 13.1 points, and doing it cheaper. Internally the numbers hold up: 53.6 minus 40.5 is 13.1, the cost geometry roughly matches the text. What the chart quietly relies on is the meaning of its own y-axis. The label reads Score, which on the actual benchmark means the average graded outcome across tasks, partial credit included. That is a different animal from Pass Rate, the fraction of tasks an agent finishes cleanly. And the benchmark itself, built at Berkeley by Dawn Song's group with real professional work and deterministic grading, was designed to puncture the forecast that agents will do almost every job by 2027. Its headline finding was sobering: on the hardest tier, the best agents fully passed about 2.6 percent of tasks. So the marketing number and the research number are not the same number at all.

Scatter plot titled Agents' Last Exam showing model scores against API cost, with GPT-5.6 Sol at the top
OpenAI's own chart, on OpenAI's own timeline. The y-axis says Score, which includes partial credit, not the strict pass rate the underlying paper leads with.

The rollout that was really a clearance

The same company, near the same moment, posted that Sol, Terra, and Luna were "starting to roll out now in ChatGPT, Codex, and the API." Happy building, Altman signed off. What the cheerful line leaves out is the two weeks before it. The models first appeared on June 26 in a limited preview, restricted to roughly twenty government-approved partners, API and Codex only, not in ChatGPT. Axios reported that the White House had asked OpenAI to gate the launch, described as the first time the United States government preemptively asked a domestic AI company to hold back a frontier model. The White House later disputed that it gave any green light, insisting such calls rest with the companies. Anthropic's Claude Fable 5 ran into Commerce Department restrictions in the same window. And under OpenAI's own Preparedness Framework, all three GPT-5.6 models, including the small fast one, were rated High for both cybersecurity and biological risk. The tweet is the polished face of a genuinely unusual regulatory episode, and the episode is the part worth reading.

Sam Altman gave CNBC the confidence-and-hedge version of the same instinct. Asked about China catching up, he said the Chinese open-source models are getting very good, that this is fine, and that OpenAI will continue to have the best models in the world because people want the best. Two things there deserve a second look. First, the precise term is open weight, not open source; the weights are downloadable, which is not the same as an open-source license granting freedom to study and modify. Second, the phrase "getting very good" undersells the data. An OpenRouter and Andreessen Horowitz study of a hundred trillion tokens found Chinese open-weight usage climbing from roughly 1.2 percent of the global total in late 2024 to nearly 30 percent within months, driven by Qwen, DeepSeek, and Kimi, though Western proprietary models still hold around 70 percent. Note that the outlet reporting the 30 percent figure most prominently, the South China Morning Post, is owned by Alibaba, Qwen's parent. Altman himself said in 2025 that OpenAI released its own open-weight models partly because otherwise the world would be built on Chinese ones. Read from outside the Anglosphere, "we will keep the best models" is market defense narrated as national destiny, and it is a claim, not a result.

The announcement that never happened

Then there is the post that is not real at all. A message under the handle @finkd announcing "Muse Spark 1.1," a cheap agentic coding model, through a new "Meta Model API." The handle is genuinely Mark Zuckerberg's, dating to early Facebook. The product is not. There is no credible launch, no coverage, no primary source for a Meta model or API by those names. Treat it as fabricated or a mock-up until proven otherwise. What is real is the thing it imitates: the Llama API that Meta previewed at LlamaCon in April 2025, the Meta AI assistant across its apps, and the policy that took effect on December 16, 2025, under which interactions with Meta's AI products feed ad targeting everywhere except the EU, the UK, and South Korea. A plausible sentence in a familiar voice is not an announcement, and the gap between the two is exactly the space these posts live in.

Marketing over mechanism shows up in security too, with sharper teeth. A widely shared post pitches a local "uncensored" model for red teamers that "won't lecture you about ethics" and "doesn't refuse, it delivers." The model is DeepHat, the rebrand of WhiteRabbitNeo, a fine-tune of Qwen2.5-Coder that runs on a single GPU and uses YaRN to stretch its context for log analysis. All plausible. But the actual model card carries a binding usage agreement: the user indemnifies the creators, agrees not to break applicable law or infringe third-party rights, and specifically agrees to no military use. Uncensored is not the same as unrestricted. The honest description is dual-use tooling, in the lineage of Metasploit when HD Moore first released it and everyone worried about who would point it where. That is a real and defensible category, distinct from criminal tools like WormGPT that were trained on malware and sold on forums. The pitch sells an absence of limits the thing does not actually provide.

For contrast, here is what it looks like when verification simply works. Asked on Rich Eisen's show whether he hit 3 percent body fat for Superman, Henry Cavill said no. He never took a reading, doubts it was that low, and pointed out that 3 percent is bodybuilder stage-day territory: "you can't hold that." The American Council on Exercise puts essential fat for men at 2 to 5 percent, the floor the body needs, not a state you maintain. What Cavill did confirm was 5,000 calories a day, during the building phase, under Mark Twight at Gym Jones, much of it in a coconut-milk shake because you can fit a day of calories into one glass. One number is true and one is inflated, and he separated them himself. It is the same shape as the Michael Phelps legend of 12,000 calories a day, which Phelps has repeatedly called a myth; the real figure was closer to 8,000 to 10,000. The exaggeration and the fact sit side by side, and only the fact survives contact.

A real gap and a false conclusion

Two posts from the same aggregator account lean on real English numbers to carry conclusions the numbers do not support. Muhammad was the most-registered boys' name in England and Wales for a third straight year, about 5,957 in the 2025 cohort. That is accurate. What it does not mean is what it is made to mean. The ONS ranks by exact spelling, so the single-spelling number-one is genuinely new, while the combined transliterations Muhammad, Mohammed, and Mohammad have effectively led for over a decade. And the largest religious shift in the 2021 census was toward no religion, at 37.2 percent, not toward Islam, which stood at 6.5 percent. Christian fell below half the population for the first time.

The second post from that account states the conclusion outright: Muslims giving birth more than anyone else, so "eventually England will be the Islamic State." There is a genuine fertility gap, which Pew put at around 2.6 children per Muslim woman against 1.6 for others. But that gap converges sharply across generations, and overall UK fertility has fallen to a record low of 1.41. Even Pew's highest-migration scenario caps the Muslim share of the UK around 17 percent by 2050. No credible projection shows a majority this century, let alone the establishment of a state. This is the Great Replacement frame in miniature: a confident prediction with no timeframe, no method, and no data, riding on one real statistic. The reply threads underneath carry defamatory falsehoods about the Prophet that I will not repeat; they are not evidence of anything except that a neutral naming statistic gets weaponized the instant it is published.

The most physical version of the pattern is a government treating a misread claim as a fact and acting on it. Kathleen Tierney posted a Game of Thrones wildfire meme, a silhouette watching a city burn in green flame, on a Tempe councilman's public Facebook page, with a caption about someone watching that night's council meeting. Tempe police read it as a possible bomb-style threat. They evacuated the meeting, surveilled her Tucson home, and served a dawn warrant with deputy US Marshals. The Maricopa County Attorney then declined to charge her, citing no reasonable likelihood of conviction. She lost her roughly 121,000 dollar job at Cox anyway, and is now pursuing the city.

Redacted screenshot of a Facebook comment showing a silhouette overlooking a city with a faint green glow, captioned watching tonight's council meeting
The post that triggered a raid. A silhouette from a television show, on a politician's public page, treated as a credible bomb threat.

The Supreme Court's 2023 Counterman decision requires at least recklessness before speech becomes a prosecutable true threat. A fictional scene referenced on a public official's page, whose own rescue dog is named Cersei after that show, does not clear it. The city's response was that its police "followed established procedures" and that Tierney was treated no differently from anyone else. That is the tell. The mechanism worked as designed, and the design turned a meme into a raid.

None of these claims gets examined by the people repeating it, which is close to the point of making it that way. A score, a rollout, a hedge dressed as destiny, a product that does not exist, a permission slip that comes with fine print, a name, a birth rate, a silhouette on a phone screen. Each arrives already framed by whoever gains from the frame, and in every case the more careful version was sitting one layer down, available to anyone who bothered. That reading is slower and less satisfying than the headline, and it keeps turning out to be the only part that holds.