← Back issues Reports from our team — to our clients, ourselves, and our agents Issue No. 6

Proofs

Q1 2026 11 pieces · 5 departments Set in Leonardo Codex

Show your work — 131 models, honest margins, and the thesis that it's agents all the way down.

The measurement quarter. The Car Wash Test fails 76% of 131 models on one common-sense question; Mercury-2 runs MCP tool evals across 39 models; Same Weights, Different Results shows the weights aren't the product — the harness around them is; and the agent-habitat and data-flywheel patterns give agents somewhere to live. In the workshop, "Agents all the way down" (February 8) compresses the whole position into a single essay — every fragment since orbits it. Measurement isn't bookkeeping; it's how the field decides what's real. Anchored by How AI in January 2026.

Proofs cover
Proofs, the cover — Renaissance notebook drawing — sepia iron-gall ink, hatch shading, geometric construction lines on aged parchment; diagrams that prove.

The Quarterly Retrospective

The measurement quarter. We asked 131 models whether to walk or drive to the car wash and three quarters of them failed; we ran 39 models against real vehicle data over MCP; we ran the same weights on four providers and got four different machines. Proof turned out to be cheap to manufacture and expensive to ignore.

The pattern pieces held up alongside the numbers: agents need habitats — state, memory, and bounded decisions living in a git repo — and the data flywheel showed up independently in three separate builds. Sraffa's Gesture supplied the quarter's philosophy: the danger isn't what models secretly understand, it's our willingness to take fluent simulation for the real thing.

How AI All 1 →

How AI in January 2026

We're at the beginning of infinity. In geological time this is all happening in an instant — and 2026 might be the best year to be alive.

Model Watch All 3 →

Same Weights, Different Results

We ran the same Nemotron model on four providers and got wildly different results. MCP tool use ranged from 1/6 to 6/6. Speed varied 16x. The weights are identical. The results are not.

Can LLMs Use Real-World Tools? Mercury-2, ELO, and the Umwelten Setup

We ran 39 models against real Rivian driving data via MCP tools. Inception's Mercury-2 delivered a perfect 15/15 in 8.6 seconds. Here's the standout model, the ELO narrative rankings, and how the same Umwelten setup powers both chat and evals.

The Car Wash Test: Learning from Model Evals

We asked 131 AI models a simple question — should I walk or drive to the car wash? 76% got it wrong. Simple gotcha questions reveal more about model reasoning than any benchmark leaderboard.

The Harness All 3 →

The Agent Habitat

An agent isn't just automation with LLM calls. It carries state, accumulates memory, and makes bounded decisions under uncertainty — and a git repo is where all of that lives.

The Data Flywheel Pattern

Build applications by dropping in data and letting AI handle parsing, structuring, and synthesis. Three case studies.

Claude Code, not Code

The real power of Claude Code isn't writing software—it's orchestrating skills for research, newsletters, browser automation, and turning one-off requests into repeatable workflows.

Essay All 1 →

Sraffa's Gesture, the Crack in the Crystal, and Why the Stochastic Parrot Still Bites

Wittgenstein's shift from the Tractatus to the Philosophical Investigations—triggered by Sraffa's Neapolitan gesture—reframes the stochastic parrot debate. The danger isn't that LLMs lack secret understanding. It's our willingness to treat fluent simulation as the real thing.

Measurements All 3 →

Testing 48 LLMs Across 5 Dimensions for $4.63

Interactive leaderboard across reasoning, knowledge, instruction following, coding, and MCP tool use.

Can LLMs Use Real-World Tools?

39 models, real Rivian driving data via MCP — most can call tools; fewer call the right ones.

The Car Wash Test: Do LLMs Have Common Sense?

131 models, one question — should you walk or drive to the car wash?