Android Bench 2.0
AI-assisted software engineering has seen the emergence of several benchmarks to measure the capabilities of LLMs. Android developers face specific challenges that aren't covered by existing benchmarks, so we created one that focuses on a north star of high quality Android development.
Long-horizon task results
Pass rate
28.0%
Avg completion rate
82.2%
Avg latency
7.9 h
Avg cost
$375.7
Per task results
Pass rate
22.7%
Avg completion rate
82.4%
Avg latency
22.2 h
Avg cost
$492.6
Per task results
Pass rate
19.3%
Avg completion rate
74.3%
Avg latency
8.6 h
Avg cost
$235.8
Per task results
Pass rate
16.7%
Avg completion rate
77.8%
Avg latency
27.0 h
Avg cost
$861.4
Per task results
Pass rate
14.0%
Avg completion rate
74.3%
Avg latency
47.2 h
Avg cost
$260.2
Per task results
Pass rate
12.0%
Avg completion rate
72.8%
Avg latency
66.2 h
Avg cost
$418.3
Per task results
Pass rate
8.0%
Avg completion rate
47.4%
Avg latency
12.1 h
Avg cost
$34.5
Per task results
Pass rate
7.3%
Avg completion rate
50.1%
Avg latency
9.9 h
Avg cost
$26.0
Per task results
Pass rate
6.7%
Avg completion rate
59.8%
Avg latency
14.7 h
Avg cost
$283.7
Per task results
Pass rate
4.7%
Avg completion rate
53.4%
Avg latency
5.1 h
Avg cost
$53.2
Per task results
1.0 task results
Track the latest AI model benchmarks, newly introduced agent architectures, and continuous performance evaluations on the platform. Stay updated with our routine methodology updates and release logs.
-
New updates • Sep 17thLaunched long-horizon tasks
-
New updates • Sep 17thNow benchmarking models using specialized agents
-
New agent results • Sep 17th
antigravity-sdk results now availableGemini 3.6 Flash, Gemini 3.7 Flash
-
New agent results • Sep 17th
pi results now availableGemini 3.6 Flash
-
New agent results • Sep 17th
codex results now availableGPT 5.6 Luna, GPT 5.6 Sol
-
New models • Aug 11thClaude Opus 5
-
New models • Aug 11thGPT 5.6 Sol, GPT 5.6 Luna, GPT 5.6 Terra
-
New models • Aug 11thKimi K3
-
New models • Aug 11thQwen3.8 Max
-
New models • Aug 11thGemini 3.6 Flash, Gemini 3.5 Flash Lite
-
Archived models • Aug 11thGemma 4 26B A4B IT