Findings and field notes.
What our benchmarks found, how we run them, and notes on building AI products you can trust.
-
Benchmarks · · 4 min readAI Checkpoint: Argon matches Opus, until it sells
Week ending 16 October: Gemini 4 Argon opens up, Muse Wonder arrives, Muse Spark Open ships open weights, how long models take per task, and a three-sentence fix for models that call real news fake.
-
Model launches · · 8 min readGemini 4 argon review: Opus-level product data at a quarter of the cost, until you ask it to sell
We ran Gemini 4 Argon through six benchmarks on launch day. It matched Claude Opus 5.5 on product data at a quarter of the cost, then added five times as many unsupported claims when asked to sell.
-
Model launches · · 6 min readClaude Opus 5.5 vs Muse wonder full: Eight crafts, and Opus still wins most
We asked Claude Opus 5.5, Muse Wonder and Muse Wonder Full to make eight crafts in a web page, from a thrown bowl to an ice swan. Full closed part of the gap that standard Wonder left, with an agent and second chances. Opus still made the more convincing object in most of them.
-
Model launches · · 7 min readMuse wonder review: Good analysis, work that still needs checking
We tested standard Muse Wonder on six evaluations. Survey analysis was strong; product copy, slide layout and decisions against a clock still needed work. Full is outside this API comparison.
-
Guides · · 9 min readWe told 11 AI models what happened in 2026. Most of them thought we were making it up.
We texted 11 models real 2026 news, from the Knicks title to Allbirds becoming an AI company. Given only today's date, most hedged, one called nearly all of it fake, and one claimed to have searched sources it had no way to reach.
-
Guides · · 6 min readWe tested the anatomy Claude Code skill on everyday objects and seven AI models
Anatomy, the viral Claude Code skill, turns a one-line idea into an interactive isometric machine. We used it to explain how a toilet flushes, how a fridge works, why lifts don't fall, how a zip works and how a hand coffee grinder works, then ran the same prompt through seven AI models.
-
Benchmarks · · 4 min readAI Checkpoint: Sol’s new fast lane
Week ending 9 October: GPT-6.1 Sol Ultrafast, Haiku 5.5, Mistral Large 4, new AI interfaces, SurveyBench and a practical lesson in image tiling.
-
Guides · · 7 min readWe raced AI models to find one small thing in a huge picture. Cutting it up more than doubled what they found.
We timed LLMs and decision models finding one small target in 79 full-size scenes, from Where's Waldo pages to our own puzzles. Gemini 3.5 Flash Lite found the most; a decision model that can only answer yes or no found a target for a tenth of a cent.
-
Model launches · · 8 min readClaude Haiku 5.5 is far better than Haiku 4.5 at a tenth of the price. GPT-6 Luna still beats it at business work
We ran Claude Haiku 5.5 through our business benchmarks and 18 playground tests on launch day. It is a big step up from Haiku 4.5 and very cheap, but GPT-6 Luna, at the same price, beat it on listings, decks and ad planning.
-
Model launches · · 7 min readMistral Large 4 is fast, cheap and Europe's best model. On business work, it still makes things up.
We ran Mistral Large 4 through our four business benchmarks on launch day. It is quick, inexpensive and careful with structured data, but it added unsupported claims to most product listings and finished last on slide decks.
-
Model launches · · 5 min readGemini 3.6 Flash wins the minute. Gemini 3.5 Flash-Lite wins the second.
Google's two new fast models split BulletBench's hardest latency tests: Gemini 3.6 Flash topped the 60-second board, while Gemini 3.5 Flash-Lite nearly won Lightning without losing a single game on time.
-
Benchmarks · · 4 min readAI is now a better football pundit than the pundit. It still lost to one line of code.
We had six frontier AI models predict all 32 knockout matches of the 2026 World Cup, blind. They out-called a top BBC pundit. Then a one-line rule, always back the higher-ranked team, beat every single one of them on…
-
Benchmarks · · 3 min readSix models, one mind: What the World Cup revealed about how AIs actually think
Across 34 World Cup fixtures, six frontier AI models from five different labs picked the same winner 31 times. Then we gave them memory of the tournament and they changed zero of 96 picks. The herd is real, and it has…
-
Benchmarks · · 3 min readThe AI that won our World Cup benchmark didn't predict best. It read the rules.
GLM-5.2 won World Cup Bench by two points while spending the fewest reasoning tokens and posting an ordinary hit rate. It found the one free lunch in our scoring rules and ate it 34 times. A story about Goodhart's law…
-
Guides · · 5 min readHow to continue a Claude Code conversation in Codex after hitting your usage limit
Ran out of Claude Code allowance halfway through a task? Here is how to move the conversation into Codex and continue without losing the decisions behind the code.
-
Benchmarks · · 5 min readGPT-5.6 Luna was meant to be fast. It still lost to the clock.
Luna arrived as the fastest GPT-5.6. Across 96 BulletBench games, it lost 71 on time. The result exposes the model the market still has not built: genuinely smart, sub-second and predictable.
-
Benchmarks · · 7 min readBulletBench: We made 23 AI models play speed chess, and the clock was the judge
Every leaderboard measures how smart a model is. None measure how smart it is per second. So we built one: AI models play speed chess on a real clock, and slow genius loses on time.
-
Benchmarks · · 3 min readSonnet 5 gave the sharpest marketing analysis we've ever tested. It also lost money every single time.
A ROASBench case study in why fluent reasoning isn't the same as good decisions, and why for this model more thinking made it worse.
-
Benchmarks · · 4 min readClaude vs Gemini vs GPT in a 12-month marketing simulation
ROAS Bench is one of the clearest examples of where frontier models diverge in practice: not on prose quality, but on economic compounding.
-
Benchmarks · · 6 min read
How we built ROASBench to feel like real growth work
ROASBench was built from operator experience, not benchmark theatre. We designed it to feel like real performance marketing: stateful, constrained, path-dependent, and economically unforgiving.
-
Guides · · 10 min readLiteLLM alternatives for 2026
Looking for a LiteLLM alternative? First decide whether you need a Python library for switching LLM providers or an AI gateway for routing, fallbacks and control. Here are the options for each.
-
Benchmarks · · 4 min read
Why most LLMs still can't run growth
ROAS Bench shows that growth is not a copywriting problem. It is a compounding systems problem, and most models still break the system faster than they improve it.
-
Benchmarks · · 3 min read
The AI marketing benchmark that punishes plausible-sounding strategy
Most LLMs can sound like a competent growth lead for one turn. ROAS Bench is interesting because it makes them live with the consequences for twelve months.
-
Benchmarks · · 8 min readGemini Embedding 2 just launched - so we benchmarked it
Google launched gemini-embedding-2-preview on 10 March 2026 as its first multimodal embedding model, with one shared embedding space for text, images, video, audio, and PDFs. Google specifically positions it for…
-
Benchmarks · · 4 min readThe great AI gifting showdown: Which model should you trust for Christmas shopping?
It’s that time of year again. You’re out and about, the clock is ticking, and you still haven't found the perfect gift for your partner, your roommate, or that difficult-to-shop-for in-law. Naturally, many of us are…
-
Model launches · · 3 min readGoogle Gemini 3 review: The benchmarks actually match the hype
So, on Tuesday Google launched Gemini 3. The hype was massive leading up to this, and honestly? It is justified. It is really, really good. Trying to explain how good is difficult without getting bogged down in…
-
Model launches · · 4 min readGPT-5.1 first look: Smarter, warmer… but not a breakthrough
2025’s flagship model season kicked off on 12 November with the unexpected arrival of GPT-5.1, with OpenAI getting their release out before Gemini 3. While we’re still waiting for API access (and therefore can’t run proper…
-
Guides · · 2 min readHow to prepare for Gemini 3 + GPT 5.1
Here we go again: new-flagship season. Google’s Gemini 3 has been peeking through A/B tests in AI Studio and docs watchers have noticed model lifecycle shuffles, while OpenAI is lining up a GPT-5.1 family (base…