We Raced AI Models to Find One Small Thing in a Huge Picture. Cutting It Up More Than Doubled What They Found.

Give an AI model a 4,000-pixel picture and ask it to find one small thing, and it mostly fails: it shrinks the picture before looking, and the thing disappears. Cut the picture into tiles, ask about every tile at once, and the same model finds more than twice as many. We raced LLMs and decision models through 79 full-size scenes to see who finds the target, how fast, and at what cost.

Watch them race

Each lane is a real run, replayed on its real clock. The box above the lanes says what the models are looking for, and the line under each name says what it is doing at that moment: checking tiles, comparing its best claims, or zooming in. Blue cells are questions still waiting for an answer, dark cells came back "not here", and amber cells claimed "found it". At the end, a green ring means the answer was right and a red cross means it was wrong; the white box is where the target really is. The fastest run on each scene is first. Pick another scene, hide models below, or save the race as a GIF.

Where's Waldo? (Where's Wally? in the UK) pages © Martin Handford, published by Walker Books and Candlewick Press. Shown here, from public research datasets, for research and commentary on how AI models search pictures; not endorsed by the rights holders. The letter and shape puzzles are ours.

The short version

• Gemini 3.5 Flash Lite found the most: 66 of 79 (84%) with 512-pixel tiles, at $0.0155 a scene. Smaller tiles found one more, at nearly four times the cost and almost twice the time.
• Cutting the picture up is what works. Shown each whole scene at once (in a pilot run earlier the same day), the same model found 26 of 79.
• The decision models answered fastest but found fewer: GPT-6 Luna Decisions and pplx-decider v1.1 each found 31 of 79, in a median 3.4 and 3.9 seconds a scene.
• On the same search, the decision models are fastest and cheapest. Run through the same yes-or-no zoom, they found 39% in under 4 seconds a scene. Only Claude Haiku 5.5 found more (48%), taking 16 seconds; Gemini 3.5 Flash Lite found 33% and GPT-6 Luna 22%. The LLMs' real edge is that they can point.
• pplx-decider v1.1 is by far the cheapest per find: $0.0004 a scene, about a tenth of a cent per target found, less than a third of the cheapest LLM run.
• Waldo is hard for everyone. The best run found him on 19 of 29 pages; Cloudflare's two decision models found one target between them in 158 tries.

How the models searched, and why LLMs and decision models search differently

Every model got the same scenes and the same target, and had to give one point; it counts if the point lands on the target (its box, plus 10% of its size). How a model can get to that point depends on what kind of answer it can give.

LLMs can say where

An LLM writes text, so it can answer "where?" directly: "yes, about a third of the way across and near the bottom". That lets it find the target in one step per piece of the picture.

• The whole picture at once. One question about the whole scene. Cheap, but the model shrinks a big picture before looking, and a small target turns into a few pixels.
• Tiles. The scene cut into overlapping squares (256, 512 or 1,024 pixels) and every square asked at the same moment: "is the target here, and if so where?" Because the questions run in parallel, 70 tiles take not much longer than the slowest one. The answer is the most confident "it's here".
• Tiles, then a double-check. Several tiles often say "it's here, 100% sure", and only one can be right. So one more question shows close-ups of the top four claims side by side and asks which is the target. This is what the LLM lanes in the race do.

Decision models can only say yes or no

A decision model doesn't write anything. It answers a typed question (yes or no, pick one, score) with a probability, which makes it quick and cheap, but it has no way to say "bottom left". So it has to narrow down instead:

• Yes or no on each tile, then zoom in. Every 512-pixel tile is asked "does this show the target?" at the same moment. The likeliest tile is cut into nine overlapping pieces and those are asked again, then the likeliest of those, three rounds in all, until a piece is about the size of the target. The answer is the middle of the last round, weighted by each piece's probability.

The probabilities are a real advantage: where LLMs claim "100% sure" for several wrong tiles, a decision model's numbers can be compared, so there is no tie to break. The cost is the zooming: three more rounds of questions one after another, each on a smaller and smaller crop, and the last crops can show only part of the target.

Comparing like with like

Each kind first got the strategy that suits what it can answer. To compare them on exactly the same search, we also ran the LLMs through the decision models' zoom: the same crops and the same yes-or-no question, with the LLM giving its probability of yes.

On that common ground the decision models are the quick, cheap searchers, but not the most accurate. GPT-6 Luna Decisions and pplx-decider v1.1 each found the target in 31 of 79 scenes (39%), in a median 3.4 and 3.9 seconds, at $0.0024 and $0.0004 a scene. Claude Haiku 5.5, zooming, found 38 (48%) but took 16.2 seconds a scene at $0.0061; Gemini 3.5 Flash Lite found 26 (33%) in 10.3 seconds at $0.0236, and GPT-6 Luna 17 (22%) in 13.5 seconds. The same LLMs did far better when they could point: Flash Lite found 66 (84%) with tiles and a double-check. So asking yes or no suits a decision model, and an LLM should be asked where.

Results: found, time and cost

How to read this table: "Found" is the share of all 79 scenes where the model pointed at the target; the next three columns split it by set (29 Waldo pages, 30 V* photos, 20 of our own puzzles). Time is the median a scene, from first question to final answer. Cost per find is the run's total cost divided by the targets it found.

The LLM rows come from a tile-size test in which the three models ran at the same time, which slows each of them a little; the decision models ran one at a time. The second round, now running, reruns every LLM alone, adds 12 more current vision models, and runs the LLMs through the decision models' zoom as well, so the two kinds can be compared on exactly the same search.

LLMs against decision models

The decision models are quick and, in pplx-decider's case, remarkably cheap. Their problem isn't finding the right area: pplx-decider picked the tile containing the target in 60 of 79 scenes, and GPT-6 Luna Decisions in 50. They lose it in the last zoom rounds, where a 48-pixel piece shows only part of a letter or a face, and the final point misses by a median of 9 pixels (pplx-decider) to 18 (GPT-6 Luna Decisions). An LLM can simply say where in the tile the target is.

pplx-decider also has a long tail: on three of our scenes a question took close to a minute, so its slowest scenes ran past 50 seconds. GPT-6 Luna Decisions never did.

Cloudflare's Clef and Clef Flash found one target between them, out of 79 scenes each. Shown a crop with the target and one without, they gave the same probability either way, so the zoom never got started in the right place. They were also the most expensive decision models per scene.

We also tried TypeSafe's Jev, a decision model that reads text only, as the judge in the verify step, choosing between Flash Lite's own few-word descriptions of each claim. It made things worse (55 found against 58 with no verify step): the descriptions were often identical.

What cutting the picture up buys

For Gemini 3.5 Flash Lite, asking about 512-pixel tiles instead of the whole scene took finds from 26 to 66 out of 79, for a little more waiting (the tiles go at once) and about 35 times the cost: $0.0155 a scene instead of $0.0004. (The whole-scene run was a pilot earlier the same day, with a prompt that differed only in not asking for a short description.) Smaller is not always better: GPT-6 Luna found most with the largest tiles (42 with 1,024 pixels, 27 with 256), and Claude Haiku 5.5 found about the same at every size. The verify step helped Flash Lite (66 against 60 without it) and hurt Haiku (55 against 62).

Waldo himself

The 29 Where's Waldo pages were the hardest set by far. Waldo is a few dozen pixels on pages up to 4,000 pixels wide, often next to people in red and white stripes, and the scans are uneven. The best run, Gemini 3.5 Flash Lite on 512-pixel tiles, found him 19 times (18 on 256-pixel tiles); Claude Haiku 5.5 found him 12 times, and the best decision model, pplx-decider, 9. Pick a Waldo scene in the race above to watch each model hunt for him. The datasets mark only his head, so a point on his head or upper body counts as finding him (we first scored the head alone, and corrected that after seeing models point at his jumper).

How we tested

• Scenes. 29 Where's Waldo pages (1,280 to 4,000 pixels) with Waldo's head marked, from the public HereIsWally, Hey-Waldo and Kaggle waldo-yolo research sets; every box checked by eye. 30 V* Bench photos (2,000 to 3,300 pixels, targets 8 to 50 pixels). 20 scenes we generated at 4,000 by 3,000 pixels.
• Models and settings. Gemini models through Google's API, the rest through OpenRouter, each lab's current models, at their fastest reasoning setting where there is one (named in the table). Decision models through OpenRouter's Decisions API.
• Rules. One point per scene; a hit if it falls in the target's box grown by 10% of its larger side. For Waldo the box is his head and upper body: the datasets' head box, widened by half and extended down by one head height, checked by eye against every near miss. A question unanswered after 60 seconds counts as a miss. Tiles overlap by 15%.
• Time runs from the first question to the final answer and includes sending the tiles, a few megabytes a scene at 512 pixels, from one office connection; faster or slower connections will shift it.
• What this doesn't show. One run per configuration, on 79 scenes: differences of a few finds are within chance. It measures finding one known target, not describing a scene.

Quick answers

Model pages: Gemini 3.5 Flash-Lite · Claude Haiku 5.5 · GPT-6 Luna