← Q1 2026
Q1 2026
Model Watch
What we know and do not know about models — opinions, briefs, and eval teasers.
-
Same Weights, Different Results
We ran the same Nemotron model on four providers and got wildly different results. MCP tool use ranged from 1/6 to 6/6. Speed varied 16x. The weights are identical. The results are not.
-
Can LLMs Use Real-World Tools? Mercury-2, ELO, and the Umwelten Setup
We ran 39 models against real Rivian driving data via MCP tools. Inception's Mercury-2 delivered a perfect 15/15 in 8.6 seconds. Here's the standout model, the ELO narrative rankings, and how the same Umwelten setup powers both chat and evals.
-
The Car Wash Test: Learning from Model Evals
We asked 131 AI models a simple question — should I walk or drive to the car wash? 76% got it wrong. Simple gotcha questions reveal more about model reasoning than any benchmark leaderboard.