← Back to home
Benchmark · PinchBench
AI Coding Agent Leaderboard
PinchBench success rates for AI coding agents, cross-referenced with ModelPriceLab stored listed-price records so you can compare benchmark performance and public prices.
57 models· 15 Code & DevOps tasks· Price overlay·
Success Rate Leaderboard
Success rate = percentage of standardized OpenClaw agent tasks completed successfully. Graded via automated checks and LLM judge.
#
Model
Success Rate
Run Cost
Input / 1M
Output / 1M
1
#1
🦞Xiaomi MiMo-V2.5 Pro
xiaomi/mimo-v2.5-pro
Success
98.0%
Run Cost
11.39Input / 1M
$0.43Output / 1M
$0.872
#2
🦀Xiaomi MiMo-V2.5
xiaomi/mimo-v2.5
Success
97.9%
Run Cost
5.44Input / 1M
$0.14Output / 1M
$0.283
#3
🦐MiniMax M2.7
minimax/minimax-m2.7
Success
96.2%
Run Cost
2.08Input / 1M
$0.21Output / 1M
$0.844
#4
Claude Opus 4.7
anthropic/claude-opus-4.7
Success
95.9%
Run Cost
38.04Input / 1M
$2.50Output / 1M
$12.505
#5
ByteDance Seed 2.0 Lite
bytedance-seed/seed-2.0-lite
Success
94.9%
Run Cost
1.74Input / 1M
$0.25Output / 1M
$2.006
#6
GLM 5V Turbo
z-ai/glm-5v-turbo
Success
94.9%
Run Cost
12.81Input / 1M
$1.20Output / 1M
$4.007
#7
Claude Opus 4.8 Fast
anthropic/claude-opus-4.8-fast
Success
94.5%
Run Cost
159.60Input / 1M
$2.50Output / 1M
$12.508
#8
Claude Sonnet 4.6
anthropic/claude-sonnet-4.6
Success
94.5%
Run Cost
15.18Input / 1M
$1.50Output / 1M
$7.509
#9
DeepSeek-V4 Flash
deepseek/deepseek-v4-flash
Success
94.0%
Run Cost
0.88Input / 1M
$0.15Output / 1M
$0.6010
#10
Gemma 4 26B
google/gemma-4-26b-a4b-it
Success
93.9%
Run Cost
0.45Input / 1M
$0.07Output / 1M
$0.2311
#11
Qwen3.7 Max
qwen/qwen3.7-max
Success
93.8%
Run Cost
19.43Input / 1M
$1.48Output / 1M
$4.4212
#12
Grok 4.3
x-ai/grok-4.3
Success
93.7%
Run Cost
13.00Input / 1M
$1.25Output / 1M
$2.5013
#13
Gemma 4 31B
google/gemma-4-31b-it
Success
93.6%
Run Cost
1.03Input / 1M
$0.09Output / 1M
$0.3414
#14
Qwen3.6 Plus
qwen/qwen3.6-plus
Success
93.6%
Run Cost
6.97Input / 1M
$0.33Output / 1M
$1.9515
#15
Nemotron 3.5 Lightning 30B
nvidia/nemotron-3.5-lightning-30b-a3b
Success
93.4%
Run Cost
-Input / 1M
$0Output / 1M
$016
#16
Mistral Devstral 2512
mistralai/devstral-2512
Success
93.4%
Run Cost
2.30Input / 1M
$0.40Output / 1M
$2.0017
#17
Claude Opus 4.8
anthropic/claude-opus-4.8
Success
92.8%
Run Cost
80.10Input / 1M
$2.50Output / 1M
$12.5018
#18
GLM 5.1
z-ai/glm-5.1
Success
92.6%
Run Cost
-Input / 1M
$1.40Output / 1M
$4.4019
#19
DeepSeek-V4 Pro
deepseek/deepseek-v4-pro
Success
92.3%
Run Cost
-Input / 1M
$0.66Output / 1M
$1.9820
#20
GPT-5.5
openai/gpt-5.5
Success
92.3%
Run Cost
22.17Input / 1M
$2.50Output / 1M
$15.0021
#21
Gemini 3 Flash Preview
google/gemini-3-flash-preview
Success
92.3%
Run Cost
3.09Input / 1M
$0.50Output / 1M
$3.0022
#22
Nemotron 3 Ultra 550B
nvidia/nemotron-3-ultra-550b-a55b
Success
92.0%
Run Cost
-Input / 1M
$0.50Output / 1M
$2.2023
#23
Gemini 3.1 Flash Lite
google/gemini-3.1-flash-lite
Success
91.3%
Run Cost
3.79Input / 1M
$0.25Output / 1M
$1.5024
#24
GPT-5.6 Sol
openai/gpt-5.6-sol
Success
90.9%
Run Cost
65.60Input / 1M
$4.00Output / 1M
$20.0025
#25
Qwen3.6 Flash
qwen/qwen3.6-flash
Success
89.9%
Run Cost
12.59Input / 1M
$0.19Output / 1M
$1.1326
#26
Claude Haiku 4.5
anthropic/claude-haiku-4.5
Success
89.8%
Run Cost
1.69Input / 1M
$1.00Output / 1M
$5.0027
#27
Gemini 3.5 Flash
google/gemini-3.5-flash
Success
89.8%
Run Cost
25.84Input / 1M
$1.50Output / 1M
$9.0028
#28
Gemini 3.1 Pro Preview
google/gemini-3.1-pro-preview
Success
89.6%
Run Cost
15.84Input / 1M
$2.00Output / 1M
$12.0029
#29
Grok 4.5
x-ai/grok-4.5
Success
89.5%
Run Cost
6.15Input / 1M
$2.00Output / 1M
$6.0030
#30
Grok 4.20
x-ai/grok-4.20
Success
89.4%
Run Cost
16.68Input / 1M
$1.25Output / 1M
$2.5031
#31
Step 3.5 Flash
stepfun/step-3.5-flash
Success
89.3%
Run Cost
0.50Input / 1M
$0.10Output / 1M
$0.3032
#32
Sakana Fugu Ultra
sakana/fugu-ultra
Success
89.0%
Run Cost
86.11Input / 1M
$5.00Output / 1M
$30.0033
#33
GLM 5 Turbo
z-ai/glm-5-turbo
Success
88.9%
Run Cost
5.89Input / 1M
$1.20Output / 1M
$4.0034
#34
GPT-5.4
openai/gpt-5.4
Success
88.3%
Run Cost
10.38Input / 1M
$1.25Output / 1M
$7.5035
#35
Kimi K2.7 Code
moonshotai/kimi-k2.7-code
Success
88.1%
Run Cost
10.32Input / 1M
$0.67Output / 1M
$3.3536
#36
GPT-5.4 Mini
openai/gpt-5.4-mini
Success
87.8%
Run Cost
3.40Input / 1M
$0.38Output / 1M
$2.2537
#37
Trinity Large Thinking
arcee-ai/trinity-large-thinking
Success
87.0%
Run Cost
1.29Input / 1M
$0.25Output / 1M
$0.8038
#38
GPT-5.6 Luna
openai/gpt-5.6-luna
Success
86.7%
Run Cost
14.36Input / 1M
$0.10Output / 1M
$0.6039
#39
Aion 3.0
aion-labs/aion-3.0
Success
85.8%
Run Cost
30.26Input / 1M
$3.00Output / 1M
$6.0040
#40
Grok Build 0.1
x-ai/grok-build-0.1
Success
85.8%
Run Cost
19.03Input / 1M
$1.00Output / 1M
$2.0041
#41
Kimi K2.5
moonshotai/kimi-k2.5
Success
83.4%
Run Cost
2.77Input / 1M
$0.45Output / 1M
$2.2542
#42
Mistral Small 2603
mistralai/mistral-small-2603
Success
82.8%
Run Cost
2.17Input / 1M
$0.15Output / 1M
$0.6043
#43
GLM 5.2
z-ai/glm-5.2
Success
81.9%
Run Cost
18.20Input / 1M
$0.02Output / 1M
$16.0044
#44
Mistral Large 2512
mistralai/mistral-large-2512
Success
81.8%
Run Cost
0.48Input / 1M
$0.50Output / 1M
$1.5045
#45
Ling 2.6 1T
inclusionai/ling-2.6-1t
Success
81.4%
Run Cost
7.00Input / 1M
$0.02Output / 1M
$0.0646
#46
GPT-5.4 Nano
openai/gpt-5.4-nano
Success
78.6%
Run Cost
0.78Input / 1M
$0.20Output / 1M
$1.2547
#47
GPT-5.6 Terra
openai/gpt-5.6-terra
Success
76.0%
Run Cost
24.13Input / 1M
$1.00Output / 1M
$6.0048
#48
Trinity Large Preview
arcee-ai/trinity-large-preview
Success
72.2%
Run Cost
1.90Input / 1M
$0.25Output / 1M
$0.8049
#49
Nemotron 3 Super 120B
nvidia/nemotron-3-super-120b-a12b
Success
67.0%
Run Cost
-Input / 1M
$0.08Output / 1M
$0.4550
#50
GPT OSS 120B
openai/gpt-oss-120b
Success
64.7%
Run Cost
0.30Input / 1M
$0.04Output / 1M
$0.1751
#51
Amazon Nova 2 Lite V1
amazon/nova-2-lite-v1
Success
50.8%
Run Cost
0.84Input / 1M
$0.30Output / 1M
$2.5052
#52
GPT OSS 20B
openai/gpt-oss-20b
Success
50.0%
Run Cost
0.31Input / 1M
$0.02Output / 1M
$0.0953
#53
GPT-5.5 Pro
openai/gpt-5.5-pro
Success
48.9%
Run Cost
153.44Input / 1M
$15.00Output / 1M
$90.0054
#54
Claude Fable 5
anthropic/claude-fable-5
Success
37.8%
Run Cost
114.16Input / 1M
$10.00Output / 1M
$50.0055
#55
Claude Sonnet 4
anthropic/claude-sonnet-4
Success
22.3%
Run Cost
21.19Input / 1M
$3.00Output / 1M
$15.0056
#56
Llama 3.1 70B Instruct
meta-llama/llama-3.1-70b-instruct
Success
19.0%
Run Cost
5.00Input / 1M
$0.40Output / 1M
$0.4057
#57
Llama 4 Scout
meta-llama/llama-4-scout
Success
8.4%
Run Cost
0.24Input / 1M
$0.10Output / 1M
$0.30Benchmark data from PinchBench (pinchbench.com), refreshed daily; latest run 2026-10-05. Price data from ModelPriceLab. For informational purposes only.
About PinchBench
PinchBench is an open-source benchmarking system that evaluates LLMs as OpenClaw coding agents across 15 standardized Code & DevOps tasks. Unlike traditional LLM benchmarks, PinchBench focuses on tool usage, multi-step reasoning, handling ambiguous instructions, and practical outcomes.
Tool Usage
Can the model call tools with correct parameters
Multi-step Reasoning
Can it chain actions to complete complex tasks
Real-world Messiness
Can it handle ambiguous and incomplete information
Practical Outcomes
Did it actually create the file, send the email
23 Benchmark Tasks
Covering calendar creation, code writing, document summarization, email triage, market research, and more. Each task is graded via automated checks, LLM judge, or a hybrid approach.
✅Sanity Check
Automated📅Calendar Event
Automated📈Stock Research
Automated✍️Blog Writing
LLM Judge🌤️Weather Script
Automated📄Doc Summary
LLM Judge🎤Conference Research
LLM Judge✉️Email Drafting
LLM Judge🧠Memory Retrieval
Automated📁File Structure
Automated🔄API Workflow
Hybrid🔌Skill Install
Automated🔍Skill Search
Automated🎨Image Generation
Hybrid🤖Humanize AI Text
LLM Judge📊Research Summary
LLM Judge📬Email Triage
Hybrid🔎Email Search
Hybrid🏢Market Research
Hybrid📑CSV/Excel Analysis
Hybrid👶ELI5 PDF Summary
LLM Judge📖Report Comprehension
Automated💾Knowledge Persistence
HybridKey Takeaways
Two Perfect Scores
Claude Opus 4.6 and Inception Mercury 2 tie at 100% success; Claude Opus 4.8 Fast follows at 94.5%.
Anthropic Owns the Top
All 6 Claude models are ranked; 5 of the top 13 spots are Anthropic, from Opus 4.6 (100%) to Haiku 4.5 (90.4%).
57x Cost Spread
DeepSeek-V4 Flash scores 91.5% at $1.44 per run, while top-tier models cost $47–$165. Compare prices before assuming a premium.
Better model decisions come from seeing benchmark scores and API pricing together
Explore the full price matrix, scenario leaderboards, and solution library on ModelPriceLab to find the optimal combination for your use case.