Quote boards, one exam at a time

Honest ranking is the same exam. These tables are WorkChi pins of public LiveBench items, scored with official generators and official scorers. They are not livebench.ai Overall, not Artificial Analysis Intelligence, and the coding table is not site Coding 83.9.

Snapshot 2026-08-17 · methodology · business-task leaderboard

Isolated protocol · n=48

HS-48

Same 48-item LiveBench HS pin, official-gen yaml/floor applied (temp 1, ≥64k). Default effort — not Max/xHigh.
Headline is item-pooled pass rate. Language / reasoning / data analysis are category slices, not a second ranking.
Not livebench.ai Overall. Ties at n=48 are common; mathif and coding-v2 break the cheap-model cluster.
A second t0/4k block exists on disk. It is not this exam and is not shown here.

#	Model	Pass %	Pass	Language	Reasoning	Data	p50	Cost
1	Claude Opus 5 anthropic/claude-opus-5	77.1	37/48	81.3	93.8	56.3	9819ms	$1.52
=	Claude Sonnet 5 anthropic/claude-sonnet-5	77.1	37/48	81.3	93.8	56.3	1890ms	$1.44
3	Grok 4.6 x-ai/grok-4.6	72.9	35/48	81.3	93.8	43.8	39.8s	$0.10
4	Gemini 3.6 Flash google/gemini-3.6-flash	70.8	34/48	81.3	93.8	37.5	1605ms	$1.08
=	MiniMax M3 minimax/minimax-m3	70.8	34/48	81.3	87.5	43.8	21.9s	$0.33
6	fusion:field fusion:field	68.8	33/48	62.5	81.3	62.5	2105ms	$1.40
=	Gemma 4 31B google/gemma-4-31b-it	68.8	33/48	75.0	81.3	50.0	228ms	$0.02
=	GPT-5.6 Sol openai/gpt-5.6-sol	68.8	33/48	75.0	87.5	43.8	11.3s	$1.07
=	GPT-5.6 Terra openai/gpt-5.6-terra	68.8	33/48	81.3	87.5	37.5	7401ms	$0.26
10	fusion:lightning fusion:lightning	66.7	32/48	81.3	68.8	50.0	1051ms	—
11	MiniMax M2.7 minimax/minimax-m2.7	62.5	30/48	43.8	93.8	50.0	87.1s	$0.34
12	DeepSeek V4 Flash deepseek/deepseek-v4-flash	60.4	29/48	75.0	56.3	50.0	347ms	$0.02
=	GLM 5.2 z-ai/glm-5.2	60.4	29/48	62.5	75.0	43.8	12.2s	$0.83
14	Gemini 3.5 Flash-Lite google/gemini-3.5-flash-lite	52.1	25/48	62.5	50.0	43.8	435ms	$0.12

Isolated protocol · n=32

Math + instruction following

32-item pin: mathematics + instruction following. Ranked by cat-mean (task-mean, then category-mean).
effort=high is not livebench.ai Max / xHigh. Not official LiveBench Overall.
Do not average this table with HS-48 or coding-v2.

#	Model	Cat-mean	Pass	Math	IF	p50	Cost
1	Claude Opus 5 anthropic/claude-opus-5	90.0	27/32	88.1	91.9	1650ms	$1.27
2	Gemini 3.6 Flash google/gemini-3.6-flash	87.6	28/32	75.2	100.0	1467ms	$0.85
3	Gemini 3.7 Flash google/gemini-3.7-flash	87.4	27/32	74.8	100.0	2209ms	$0.23
4	fusion:lightning fusion:lightning	86.7	25/32	73.5	100.0	2479ms	$0.60
5	GPT-5.6 Sol openai/gpt-5.6-sol	83.6	25/32	74.6	92.6	581ms	$1.21
6	GPT-5.6 Terra openai/gpt-5.6-terra	83.5	24/32	74.6	92.3	561ms	$0.25
7	Gemini 3.5 Flash-Lite google/gemini-3.5-flash-lite	82.8	25/32	65.5	100.0	1209ms	$0.61
8	MiniMax M2.7 minimax/minimax-m2.7	81.9	25/32	67.6	96.1	42.7s	$0.27
9	GLM 5.2 z-ai/glm-5.2	81.5	25/32	75.3	87.8	472ms	$0.33
10	DeepSeek V4 Flash deepseek/deepseek-v4-flash	79.0	24/32	62.2	95.8	1220ms	$0.05
11	Gemma 4 31B google/gemma-4-31b-it	75.1	22/32	66.6	83.6	422ms	$0.14
12	Grok 4.6 x-ai/grok-4.6	74.8	24/32	57.2	92.3	53.6s	$0.09
13	Claude Sonnet 5 anthropic/claude-sonnet-5	74.6	22/32	73.6	75.7	1934ms	$1.37
14	Ling 3.0 Flash inclusionai/ling-3.0-flash	74.6	24/32	49.1	100.0	600ms	$0.04
15	MiniMax M3 minimax/minimax-m3	73.8	20/32	71.9	75.8	23.5s	$0.25
16	fusion:field fusion:field	69.3	18/32	58.2	80.4	60.4s	$2.26

Isolated protocol · n=32

Coding v2

32-item public 2024 pin. Official tests in Docker (runc + caps), not string-match and not gVisor.
Every public HF livebench/coding row has livebench_removal_date=2025-04-02. This is not livebench.ai site Coding (83.9).
effort=high is not site Max/xHigh. Opus and Gemini 3.6/3.7 saturate this pin (32/32) — the exam cannot rank that pair.
Do not pool with coding-v1 or leftover/residual probes.

#	Model	Cat-mean	Pass	LCB	Completion	p50	Cost
1	Claude Opus 5 anthropic/claude-opus-5	100.0	32/32	100.0	100.0	1736ms	$1.44
=	Gemini 3.6 Flash google/gemini-3.6-flash	100.0	32/32	100.0	100.0	1440ms	$0.57
=	Gemini 3.7 Flash google/gemini-3.7-flash	100.0	32/32	100.0	100.0	1806ms	$0.19
4	fusion:lightning fusion:lightning	96.9	31/32	100.0	93.8	2270ms	$0.52
5	GPT-5.6 Sol openai/gpt-5.6-sol	93.8	30/32	93.8	93.8	494ms	$0.67
6	fusion:field fusion:field	90.6	29/32	93.8	87.5	48.9s	$2.58
=	GPT-5.6 Terra openai/gpt-5.6-terra	90.6	29/32	93.8	87.5	489ms	$0.18
8	Grok 4.6 x-ai/grok-4.6	87.5	28/32	93.8	81.3	45.8s	$0.07
9	GLM 5.2 z-ai/glm-5.2	84.4	27/32	87.5	81.3	755ms	$1.61
10	Claude Sonnet 5 anthropic/claude-sonnet-5	78.1	25/32	93.8	62.5	1899ms	$0.42
11	MiniMax M3 minimax/minimax-m3	75.0	24/32	81.3	68.8	48.6s	$0.27
12	Gemma 4 31B google/gemma-4-31b-it	71.9	23/32	93.8	50.0	350ms	$0.16
13	DeepSeek V4 Flash deepseek/deepseek-v4-flash	68.8	22/32	93.8	43.8	983ms	$0.03
14	Gemini 3.5 Flash-Lite google/gemini-3.5-flash-lite	62.5	20/32	87.5	37.5	1151ms	$0.39
15	Ling 3.0 Flash inclusionai/ling-3.0-flash	53.1	17/32	75.0	31.3	685ms	$0.02
=	MiniMax M2.7 minimax/minimax-m2.7	53.1	17/32	81.3	25.0	27.4s	$0.11

What these numbers are

HS-48 is language + reasoning + data analysis (no math, no coding, no IF), default effort. Math+IF and coding-v2 use effort=high, which is still not livebench.ai Max / xHigh. Coding-v2 is a 2024 public slice (removal_date=2025-04-02); Opus and Gemini 3.6/3.7 all score 100 on it. Rank that pair on math+IF, not coding-v2.

Fusion rows are ensembles, not single models. Serving is mixed: MiniMax and xAI first-party when available, everyone else OpenRouter. Do not average a row with a different protocol string.

A separate residual coding pin (n=3, not shown as a ranking table) keeps the leftover items Opus and Gemini 3.7 both fail. On that exam Sol scores 1/3; Opus, Gemini 3.6, and Gemini 3.7 score 0/3. n is too small to quote a ranking. It is not coding-v2 and not site Coding.

Loading…

LiveBench subsets

Quote boards, one exam at a time

Snapshot 2026-08-17 · methodology · business-task leaderboard

Isolated protocol · n=48

HS-48

Same 48-item LiveBench HS pin, official-gen yaml/floor applied (temp 1, ≥64k). Default effort — not Max/xHigh.
Headline is item-pooled pass rate. Language / reasoning / data analysis are category slices, not a second ranking.
Not livebench.ai Overall. Ties at n=48 are common; mathif and coding-v2 break the cheap-model cluster.
A second t0/4k block exists on disk. It is not this exam and is not shown here.

#	Model	Pass %	Pass	Language	Reasoning	Data	p50	Cost
1	Claude Opus 5 anthropic/claude-opus-5	77.1	37/48	81.3	93.8	56.3	9819ms	$1.52
=	Claude Sonnet 5 anthropic/claude-sonnet-5	77.1	37/48	81.3	93.8	56.3	1890ms	$1.44
3	Grok 4.6 x-ai/grok-4.6	72.9	35/48	81.3	93.8	43.8	39.8s	$0.10
4	Gemini 3.6 Flash google/gemini-3.6-flash	70.8	34/48	81.3	93.8	37.5	1605ms	$1.08
=	MiniMax M3 minimax/minimax-m3	70.8	34/48	81.3	87.5	43.8	21.9s	$0.33
6	fusion:field fusion:field	68.8	33/48	62.5	81.3	62.5	2105ms	$1.40
=	Gemma 4 31B google/gemma-4-31b-it	68.8	33/48	75.0	81.3	50.0	228ms	$0.02
=	GPT-5.6 Sol openai/gpt-5.6-sol	68.8	33/48	75.0	87.5	43.8	11.3s	$1.07
=	GPT-5.6 Terra openai/gpt-5.6-terra	68.8	33/48	81.3	87.5	37.5	7401ms	$0.26
10	fusion:lightning fusion:lightning	66.7	32/48	81.3	68.8	50.0	1051ms	—
11	MiniMax M2.7 minimax/minimax-m2.7	62.5	30/48	43.8	93.8	50.0	87.1s	$0.34
12	DeepSeek V4 Flash deepseek/deepseek-v4-flash	60.4	29/48	75.0	56.3	50.0	347ms	$0.02
=	GLM 5.2 z-ai/glm-5.2	60.4	29/48	62.5	75.0	43.8	12.2s	$0.83
14	Gemini 3.5 Flash-Lite google/gemini-3.5-flash-lite	52.1	25/48	62.5	50.0	43.8	435ms	$0.12

Isolated protocol · n=32

Math + instruction following

32-item pin: mathematics + instruction following. Ranked by cat-mean (task-mean, then category-mean).
effort=high is not livebench.ai Max / xHigh. Not official LiveBench Overall.
Do not average this table with HS-48 or coding-v2.

#	Model	Cat-mean	Pass	Math	IF	p50	Cost
1	Claude Opus 5 anthropic/claude-opus-5	90.0	27/32	88.1	91.9	1650ms	$1.27
2	Gemini 3.6 Flash google/gemini-3.6-flash	87.6	28/32	75.2	100.0	1467ms	$0.85
3	Gemini 3.7 Flash google/gemini-3.7-flash	87.4	27/32	74.8	100.0	2209ms	$0.23
4	fusion:lightning fusion:lightning	86.7	25/32	73.5	100.0	2479ms	$0.60
5	GPT-5.6 Sol openai/gpt-5.6-sol	83.6	25/32	74.6	92.6	581ms	$1.21
6	GPT-5.6 Terra openai/gpt-5.6-terra	83.5	24/32	74.6	92.3	561ms	$0.25
7	Gemini 3.5 Flash-Lite google/gemini-3.5-flash-lite	82.8	25/32	65.5	100.0	1209ms	$0.61
8	MiniMax M2.7 minimax/minimax-m2.7	81.9	25/32	67.6	96.1	42.7s	$0.27
9	GLM 5.2 z-ai/glm-5.2	81.5	25/32	75.3	87.8	472ms	$0.33
10	DeepSeek V4 Flash deepseek/deepseek-v4-flash	79.0	24/32	62.2	95.8	1220ms	$0.05
11	Gemma 4 31B google/gemma-4-31b-it	75.1	22/32	66.6	83.6	422ms	$0.14
12	Grok 4.6 x-ai/grok-4.6	74.8	24/32	57.2	92.3	53.6s	$0.09
13	Claude Sonnet 5 anthropic/claude-sonnet-5	74.6	22/32	73.6	75.7	1934ms	$1.37
14	Ling 3.0 Flash inclusionai/ling-3.0-flash	74.6	24/32	49.1	100.0	600ms	$0.04
15	MiniMax M3 minimax/minimax-m3	73.8	20/32	71.9	75.8	23.5s	$0.25
16	fusion:field fusion:field	69.3	18/32	58.2	80.4	60.4s	$2.26

Isolated protocol · n=32

Coding v2

32-item public 2024 pin. Official tests in Docker (runc + caps), not string-match and not gVisor.
Every public HF livebench/coding row has livebench_removal_date=2025-04-02. This is not livebench.ai site Coding (83.9).
effort=high is not site Max/xHigh. Opus and Gemini 3.6/3.7 saturate this pin (32/32) — the exam cannot rank that pair.
Do not pool with coding-v1 or leftover/residual probes.

#	Model	Cat-mean	Pass	LCB	Completion	p50	Cost
1	Claude Opus 5 anthropic/claude-opus-5	100.0	32/32	100.0	100.0	1736ms	$1.44
=	Gemini 3.6 Flash google/gemini-3.6-flash	100.0	32/32	100.0	100.0	1440ms	$0.57
=	Gemini 3.7 Flash google/gemini-3.7-flash	100.0	32/32	100.0	100.0	1806ms	$0.19
4	fusion:lightning fusion:lightning	96.9	31/32	100.0	93.8	2270ms	$0.52
5	GPT-5.6 Sol openai/gpt-5.6-sol	93.8	30/32	93.8	93.8	494ms	$0.67
6	fusion:field fusion:field	90.6	29/32	93.8	87.5	48.9s	$2.58
=	GPT-5.6 Terra openai/gpt-5.6-terra	90.6	29/32	93.8	87.5	489ms	$0.18
8	Grok 4.6 x-ai/grok-4.6	87.5	28/32	93.8	81.3	45.8s	$0.07
9	GLM 5.2 z-ai/glm-5.2	84.4	27/32	87.5	81.3	755ms	$1.61
10	Claude Sonnet 5 anthropic/claude-sonnet-5	78.1	25/32	93.8	62.5	1899ms	$0.42
11	MiniMax M3 minimax/minimax-m3	75.0	24/32	81.3	68.8	48.6s	$0.27
12	Gemma 4 31B google/gemma-4-31b-it	71.9	23/32	93.8	50.0	350ms	$0.16
13	DeepSeek V4 Flash deepseek/deepseek-v4-flash	68.8	22/32	93.8	43.8	983ms	$0.03
14	Gemini 3.5 Flash-Lite google/gemini-3.5-flash-lite	62.5	20/32	87.5	37.5	1151ms	$0.39
15	Ling 3.0 Flash inclusionai/ling-3.0-flash	53.1	17/32	75.0	31.3	685ms	$0.02
=	MiniMax M2.7 minimax/minimax-m2.7	53.1	17/32	81.3	25.0	27.4s	$0.11

What these numbers are

Fusion rows are ensembles, not single models. Serving is mixed: MiniMax and xAI first-party when available, everyone else OpenRouter. Do not average a row with a different protocol string.