Gemini 4 Hydrogen got faster: we reran our speed chess test
Test Gemini 4 Hydrogen on your prompt → It runs beside the models you could use instead, with every answer checked and costed.

Google released Gemini 4 Hydrogen on 12 November as the fast member of the Gemini 4 family, saying it finishes tasks 2.5 times faster than Gemini 3.8 Flash. When we ran our speed chess test the next day, it didn't look fast. In 10-second games it lost its first four games on time at both settings we tried, and in 60-second games it scored 557.
We heard it had got quicker since, so on 16 November we reran exactly the same settings. It is quicker. Every setting answered faster, and the extra time on the clock turned into better chess.
Same test, three days apart
BulletBench plays a model against Stockfish at fixed strengths with a chess clock running, so a model has to be quick as well as good. Lightning gives each side 10 seconds plus 1 second a move; Bullet gives 60 seconds for the whole game. Each setting plays 8 games.
| Setting | Score, 13 Nov | Score, 16 Nov | Median move, 13 → 16 Nov | Lost on time, 16 Nov |
|---|---|---|---|---|
| Lightning, default | too slow | 147 (0–381) | 1.85 s → 1.50 s | 7 of 8 |
| Lightning, low reasoning | too slow | 557 (370–727) | 1.88 s → 1.39 s | 5 of 8 |
| Bullet, default | 557 (370–727) | 889 (593–1,229) | 3.39 s → 2.30 s | 4 of 8 |
| Bullet, low reasoning | 557 (370–727) | 839 (541–1,130) | 2.63 s → 1.71 s | 4 of 8 |
Ranges are 95% intervals. "Too slow" means the run stopped after four straight losses on time, which is how BulletBench treats a model that can't keep up with the clock.
The Bullet result is the big change. At 889, Hydrogen's default setting is now 4th of 71 configurations, behind Gemini 3.8 Flash at low reasoning (1,068), GPT-6.1 Sol on OpenAI's Ultrafast tier (928) and Gemini 3.1 Flash-Lite at minimal reasoning (912). The intervals overlap, so treat those three as close. Its games also went on longer: 166 moves across the 8 games against 123 on 13 November.
It is still clock-bound. It lost at least half its games on time at every setting, and in Lightning only the low reasoning setting scored respectably (557, 22nd of 48). Gemini Decision 1, Google's non-LLM decision model, still does better there at 755.
Why it got faster
We don't know. Google hasn't announced a change to Hydrogen, and the Gemini API changelog has no speed note for it after the 12 November launch entry. On 15 November a Google engineer, Melvin Johnson, asked for feedback on the Gemini 4 models and, to a request to make Carbon faster, replied "Making it faster." Launch-week traffic easing off would also do it. OpenRouter showed no speed figures for Hydrogen at launch; on 16 November its five standard Google routes showed a median 1.7 to 2.7 seconds to first token and 127 to 143 tokens a second.
A speed test on one day is a snapshot of the serving, not of the model. We plan to rerun Hydrogen, and Argon and Carbon too, since both lost every 10-second game on time at launch.
Cheapest model that passes
On our benchmarks that define a pass for each task: what one passing answer cost in our run, failed attempts included, and whether a cheaper model passed.
- CatalogBench · cost per 1,000 accepted listings
Gemini 4 Hydrogen passed 125 of 168 attempts, with no estimate: failed-call costs incomplete. Cheapest: GPT-6 Luna at $0.70 (114 of 168 attempts). - DeckBench · cost per presentable deck
Gemini 4 Hydrogen: no presentable decks (0 of 6 tasks). Cheapest: GPT-6.1 Terra at $0.13 (3 of 6 tasks). - SurveyBench · cost per sound survey task
Gemini 4 Hydrogen: $3.03 (1 of 9 tasks), 11 of 12. Cheapest: GPT-6 Luna at $0.03 (3 of 9 tasks). It passed at least as often.
Cost per attempt divided by the share that passed, at the run date's prices, excluding judging. Each model at its cheapest setting per passing answer. With a handful of tasks, a pass or two either way is within chance. No estimate where a run had missing outputs, because their cost may not have been recorded.
Quick answers
Did Gemini 4 Hydrogen get faster?
In our tests, yes. Its median time per chess move fell by 19 to 35% between 13 and 16 November at the same settings, through the same OpenRouter route. Google has not announced a change.
How fast is Gemini 4 Hydrogen now?
A median 1.4 to 1.5 seconds a move in 10-second games and 1.7 to 2.3 seconds in 60-second games, on 16 November. OpenRouter's own figures that day showed 127 to 143 tokens a second on Google's five standard routes.
Is Gemini 4 Hydrogen good at speed chess now?
In 60-second games it rated 889 at its default setting, 4th of 71 configurations, behind Gemini 3.8 Flash at low reasoning (1,068). It still lost half its games on time. In 10-second games it rated 557 at low reasoning, 22nd of 48.
Release dates: new AI models released this week
Will Gemini 4 Hydrogen work for your product?
Benchmarks aren't your task. Paste the prompt your product uses and run it beside the models you could use instead, with every answer checked and costed.
New here? Your email starts a free trial. No credit card needed.