Research

Gemini 4 Hydrogen got faster: we reran our speed chess test

Test Gemini 4 Hydrogen on your prompt → It runs beside the models you could use instead, with every answer checked and costed.

Two dot charts comparing Gemini 4 Hydrogen on 13 and 16 November. Scores rose at every setting (Bullet default 557 to 889) and median seconds per move fell at every setting (Bullet default 3.39 to 2.30).
BulletBench, Gemini 4 Hydrogen through OpenRouter, 8 games per setting, 13 and 16 November 2026. On 13 November both 10+1 runs stopped after losing their first four games on time.

Google released Gemini 4 Hydrogen on 12 November as the fast member of the Gemini 4 family, saying it finishes tasks 2.5 times faster than Gemini 3.8 Flash. When we ran our speed chess test the next day, it didn't look fast. In 10-second games it lost its first four games on time at both settings we tried, and in 60-second games it scored 557.

We heard it had got quicker since, so on 16 November we reran exactly the same settings. It is quicker. Every setting answered faster, and the extra time on the clock turned into better chess.

Same test, three days apart

BulletBench plays a model against Stockfish at fixed strengths with a chess clock running, so a model has to be quick as well as good. Lightning gives each side 10 seconds plus 1 second a move; Bullet gives 60 seconds for the whole game. Each setting plays 8 games.

SettingScore, 13 NovScore, 16 NovMedian move, 13 → 16 NovLost on time, 16 Nov
Lightning, defaulttoo slow147 (0–381)1.85 s → 1.50 s7 of 8
Lightning, low reasoningtoo slow557 (370–727)1.88 s → 1.39 s5 of 8
Bullet, default557 (370–727)889 (593–1,229)3.39 s → 2.30 s4 of 8
Bullet, low reasoning557 (370–727)839 (541–1,130)2.63 s → 1.71 s4 of 8

Ranges are 95% intervals. "Too slow" means the run stopped after four straight losses on time, which is how BulletBench treats a model that can't keep up with the clock.

The Bullet result is the big change. At 889, Hydrogen's default setting is now 4th of 71 configurations, behind Gemini 3.8 Flash at low reasoning (1,068), GPT-6.1 Sol on OpenAI's Ultrafast tier (928) and Gemini 3.1 Flash-Lite at minimal reasoning (912). The intervals overlap, so treat those three as close. Its games also went on longer: 166 moves across the 8 games against 123 on 13 November.

It is still clock-bound. It lost at least half its games on time at every setting, and in Lightning only the low reasoning setting scored respectably (557, 22nd of 48). Gemini Decision 1, Google's non-LLM decision model, still does better there at 755.

Why it got faster

We don't know. Google hasn't announced a change to Hydrogen, and the Gemini API changelog has no speed note for it after the 12 November launch entry. On 15 November a Google engineer, Melvin Johnson, asked for feedback on the Gemini 4 models and, to a request to make Carbon faster, replied "Making it faster." Launch-week traffic easing off would also do it. OpenRouter showed no speed figures for Hydrogen at launch; on 16 November its five standard Google routes showed a median 1.7 to 2.7 seconds to first token and 127 to 143 tokens a second.

A speed test on one day is a snapshot of the serving, not of the model. We plan to rerun Hydrogen, and Argon and Carbon too, since both lost every 10-second game on time at launch.

Cheapest model that passes

On our benchmarks that define a pass for each task: what one passing answer cost in our run, failed attempts included, and whether a cheaper model passed.

  • CatalogBench · cost per 1,000 accepted listings
    Gemini 4 Hydrogen passed 125 of 168 attempts, with no estimate: failed-call costs incomplete. Cheapest: GPT-6 Luna at $0.70 (114 of 168 attempts).
  • DeckBench · cost per presentable deck
    Gemini 4 Hydrogen: no presentable decks (0 of 6 tasks). Cheapest: GPT-6.1 Terra at $0.13 (3 of 6 tasks).
  • SurveyBench · cost per sound survey task
    Gemini 4 Hydrogen: $3.03 (1 of 9 tasks), 11 of 12. Cheapest: GPT-6 Luna at $0.03 (3 of 9 tasks). It passed at least as often.

Cost per attempt divided by the share that passed, at the run date's prices, excluding judging. Each model at its cheapest setting per passing answer. With a handful of tasks, a pass or two either way is within chance. No estimate where a run had missing outputs, because their cost may not have been recorded.

Quick answers

Did Gemini 4 Hydrogen get faster?

In our tests, yes. Its median time per chess move fell by 19 to 35% between 13 and 16 November at the same settings, through the same OpenRouter route. Google has not announced a change.

How fast is Gemini 4 Hydrogen now?

A median 1.4 to 1.5 seconds a move in 10-second games and 1.7 to 2.3 seconds in 60-second games, on 16 November. OpenRouter's own figures that day showed 127 to 143 tokens a second on Google's five standard routes.

Is Gemini 4 Hydrogen good at speed chess now?

In 60-second games it rated 889 at its default setting, 4th of 71 configurations, behind Gemini 3.8 Flash at low reasoning (1,068). It still lost half its games on time. In 10-second games it rated 557 at low reasoning, 22nd of 48.

Release dates: new AI models released this week

Will Gemini 4 Hydrogen work for your product?

Benchmarks aren't your task. Paste the prompt your product uses and run it beside the models you could use instead, with every answer checked and costed.

New here? Your email starts a free trial. No credit card needed.

More research
All posts →