OpenHands Index

Name: OpenHands Index results
Creator: OpenHands Index
License: https://www.apache.org/licenses/LICENSE-2.0

Writing tests that reproduce reported bugs.

Data older than 90 days: last updated 30 Jun 2026

Results dated: 30 Jun 2026
Models: 34
Unit: % resolved
Licence: Apache License 2.0

Testing (SWT-Bench): Kimi K2.6

Top 15 of 34 results · % resolved, higher is better. Choose a model to highlight it.Clear highlight

1 Claude Fable 5Anthropic 91.9%
2 Claude Opus 4.8Anthropic 84.3%
3 GPT-5.5OpenAI 83.4%
4 MiniMax-M3MiniMax 81.1%
5 Claude Opus 4.7Anthropic 80.8%
6 Claude Opus 4.6Anthropic 78.8%
7 Claude Opus 4.5Anthropic 78.5%
8 Gemini 3.5 FlashGoogle 76.2%
9 Gemini 3.1 Pro PreviewGoogle 75.1%
10 GPT-5.2OpenAI 73.2%
11 Gemini 3 Flash PreviewGoogle 70.7%
12 Kimi K2.6Moonshot AI 70.4%
12 GPT-5.4OpenAI 70.4%
14 GLM-5.1Z.ai 70.2%
15 MiniMax-M2.7MiniMax 69.1%

Full results

OpenHands Index: testing (SWT-Bench), % resolved, higher is better
#	Model	Testing (SWT-Bench) % resolved, higher is better	Price $ per million tokens, in / out
1	Claude Fable 5Anthropic	91.9%	$10 / $50
2	Claude Opus 4.8Anthropic	84.3%	$5 / $25
3	GPT-5.5OpenAI	83.4%	$5 / $30
4	MiniMax-M3MiniMax	81.1%	$0.30 / $1.20
5	Claude Opus 4.7Anthropic	80.8%	$5 / $25
6	Claude Opus 4.6Anthropic	78.8%	$5 / $25
7	Claude Opus 4.5Anthropic	78.5%	$5 / $25
8	Gemini 3.5 FlashGoogle	76.2%	$1.50 / $9
9	Gemini 3.1 Pro PreviewGoogle	75.1%	$2 / $12
10	GPT-5.2OpenAI	73.2%	$1.75 / $14
11	Gemini 3 Flash PreviewGoogle	70.7%	$0.50 / $3
12	Kimi K2.6Moonshot AI	70.4%	$0.95 / $4
12	GPT-5.4OpenAI	70.4%	$2.50 / $15
14	GLM-5.1Z.ai	70.2%	$1.38 / $4.40
15	MiniMax-M2.7MiniMax	69.1%	$0.30 / $1.20
16	Claude Sonnet 4.5Anthropic	68.8%	$3 / $15
17	Gemini 3 Pro PreviewGoogle	68.6%	–
18	DeepSeek-V4-Pro (0423)DeepSeek	68.1%	$1.42 / $2.83
18	MiniMax-M2.5MiniMax	68.1%	$0.30 / $1.20
20	GPT-5.2-CodexOpenAI	67.0%	$1.75 / $14
21	Qwen3.6-PlusAlibaba	62.1%	$0.33 / $1.95
22	Kimi K2.5Moonshot AI	61.9%	$0.57 / $2.85
23	MiniMax-M2.1MiniMax	61.4%	$0.30 / $1.20
24	Claude Sonnet 4.6Anthropic	54.0%	$3 / $15
25	DeepSeek-V3.2 (reasoning on)DeepSeek	53.6%	$0.30 / $0.96
26	GLM-4.7Z.ai	49.4%	$0.54 / $1.98
27	Kimi K2 ThinkingMoonshot AI	47.3%	$0.60 / $2.50
27	GLM-5Z.ai	47.3%	$0.95 / $2.55
29	Qwen3-Coder-NextAlibaba	45.7%	$0.18 / $0.90
29	Nemotron 3 Super 120B A12BNVIDIA	45.7%	$0.085 / $0.40
31	Qwen3.5-FlashAlibaba	38.3%	$0.065 / $0.26
32	Qwen3-Coder-480B-A35BAlibaba	34.9%	$0.35 / $1.50
33	Trinity Large ThinkingArcee AI	33.3%	$0.25 / $0.80
34	Nemotron 3 Nano 30B A3BNVIDIA	7.6%	$0.05 / $0.20

Swipe the table sideways for more columns.

Ranks follow the score as shown, so equal numbers share a rank. Results as published by OpenHands Index; we do not re-run them.

Progress over time

Testing (SWT-Bench), plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's; 1 model without a known release date is left off. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models. The 6 labs with the best scores today; 4 others not drawn.

AnthropicOpenAIMiniMaxGoogleMoonshot AIZ.ai

What it measures

Writing tests that reproduce reported bugs.

What it does not measure

Not the model alone: the score is for the model inside the OpenHands agent, at the SDK version shown.

Source

OpenHands Index · Methodology

OpenHands Index by the OpenHands contributors, Apache License 2.0.