Google updates Android Bench leaderboard for AI coding tools

In short: Google updated Android Bench, a public test that compares AI tools on Android app development tasks, and it now includes more models and new cost and speed metrics.

What happened

Google has refreshed Android Bench, a benchmark (a standardized test) it uses to measure how well different AI models help with Android app development. The test includes 100 tasks that cover common developer work, like writing code and fixing issues.

The updated leaderboard adds eight newer models, including Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, and Qwen 3.7 Max. Google also says it is now tracking more than just accuracy, and it includes cost and efficiency, plus “open-weight” models (models where the core files can be downloaded and run by others, like having a recipe instead of only being able to order the dish).

In the new results, Gemini 3.1 Pro sits in fifth place. It trails models such as GPT 5.4, Claude Sonnet 5, and Claude Fable 5. Google’s chart shows Claude Fable 5 leading with 84.5 percent accuracy.

Cost and time also vary widely. The article notes that some top scorers can cost more than $130 in tokens (the paid units of text the model reads and writes) to run the full benchmark. Gemini 3.5 Flash showed a high cost in this test because it took about 28 hours to finish.

Why it matters

Many developers now use AI coding helpers, but results can be uneven, like different students doing better on different parts of the same exam. A public scoreboard that includes both accuracy and cost can help teams choose tools based on their budget and time, not just raw performance.

Source: Arstechnica