Created bymario-alka.dePowered bynoob2claw.detricoma.de
🏆
Rankings

LLM Leaderboard

Real measurements of every tested language model on real, named hardware – rated by raw speed (performance in tokens per second, prefill and time to first token) and by practical task quality in complete agent and chat runs (harness). Pick a benchmark type below or filter by model, maker and hardware to see exactly what is tested and how the results are produced.

⚡ Performance (tok/s)🤖 Harness quality👥 Concurrency🖥️ real hardware
Reset
Metric:
Size
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
219,59 tok/s TG
Prefill 2.273 · TTFT 11.329 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen 9 7945HX with Radeon Graphics · 61 GB RAMllama.cppgodclawIQ4_XS
2Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
157,50 tok/s TG
Prefill 1.715 · TTFT 6.606 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen 9 7945HX with Radeon Graphics · 61 GB RAMllama.cppgodclawIQ4_XS
3Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
138,75 tok/s TG
Prefill 1.439 · TTFT 1.508 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen 9 7945HX with Radeon Graphics · 61 GB RAMllama.cppgodclawIQ4_XS
4Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
190,25 tok/s TG
Prefill 1.859 · TTFT 13.867 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen 9 7945HX with Radeon Graphics · 61 GB RAMllama.cppgodclawIQ4_XS
5Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
156,26 tok/s TG
Prefill 1.343 · TTFT 8.459 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen 9 7945HX with Radeon Graphics · 61 GB RAMllama.cppgodclawIQ4_XS
6Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
120,01 tok/s TG
Prefill 1.155 · TTFT 1.875 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen 9 7945HX with Radeon Graphics · 61 GB RAMllama.cppgodclawIQ4_XS
7Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,14 tok/s TG
Prefill 118 · TTFT 246.235 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
8Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
23,19 tok/s TG
Prefill 102 · TTFT 136.966 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
9Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,20 tok/s TG
Prefill 98 · TTFT 19.630 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
10Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
30,60 tok/s TG
Prefill 119 · TTFT 251.486 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
11Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
22,55 tok/s TG
Prefill 103 · TTFT 138.591 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
12NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
15,60 tok/s TG
Prefill 71 · TTFT 293.317 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
13NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
16,81 tok/s TG
Prefill 67 · TTFT 215.255 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
14NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,87 tok/s TG
Prefill 61 · TTFT 35.354 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
15Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
23,62 tok/s TG
Prefill 80 · TTFT 331.679 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
16Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,05 tok/s TG
Prefill 68 · TTFT 189.332 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
17Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
7,24 tok/s TG
Prefill 64 · TTFT 30.093 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
18DeepSeek-V4-Flash-284B-A13B284BDeepSeek Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,13 tok/s TG
Prefill 69 · TTFT 186.352 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
19DeepSeek-V4-Flash-284B-A13B284BDeepSeek Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,95 tok/s TG
Prefill 72 · TTFT 174.547 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
20DeepSeek-V4-Flash-284B-A13B284BDeepSeek Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,29 tok/s TG
Prefill 65 · TTFT 35.268 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
21Llama-3_1-Nemotron-Ultra-253B-v1253BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,68 tok/s TG
Prefill 48 · TTFT 183.808 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
22Llama-3_1-Nemotron-Ultra-253B-v1253BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,63 tok/s TG
Prefill 51 · TTFT 181.415 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
23Llama-3_1-Nemotron-Ultra-253B-v1253BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
0,67 tok/s TG
Prefill 35 · TTFT 60.617 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
24Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
309,02 tok/s TG
Prefill 1.120 · TTFT 42.243 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
25Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
184,38 tok/s TG
Prefill 940 · TTFT 16.001 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
26Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
69,03 tok/s TG
Prefill 856 · TTFT 2.253 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
27Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
883,39 tok/s TG
Prefill 5.377 · TTFT 12.467 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
28Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
468,02 tok/s TG
Prefill 4.949 · TTFT 4.084 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
29Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
129,00 tok/s TG
Prefill 3.247 · TTFT 594 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
30minimax-m2230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
13,28 tok/s TG
Prefill 120 · TTFT 263.391 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawAWQ
31minimax-m2230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
16,95 tok/s TG
Prefill 113 · TTFT 190.982 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawAWQ
32minimax-m2230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
10,53 tok/s TG
Prefill 67 · TTFT 34.091 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawAWQ
33Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
316,28 tok/s TG
Prefill 1.496 · TTFT 42.551 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
34Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
205,50 tok/s TG
Prefill 1.351 · TTFT 15.992 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
35Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
88,50 tok/s TG
Prefill 1.209 · TTFT 2.272 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
36Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,17 tok/s TG
Prefill 1.479 · TTFT 42.546 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
37Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
203,41 tok/s TG
Prefill 1.372 · TTFT 16.012 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
38Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
88,20 tok/s TG
Prefill 1.206 · TTFT 2.277 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
39Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,94 tok/s TG
Prefill 1.446 · TTFT 42.781 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
40Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
204,44 tok/s TG
Prefill 1.300 · TTFT 16.378 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
41Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
88,39 tok/s TG
Prefill 1.195 · TTFT 2.302 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
42Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,14 tok/s TG
Prefill 150 · TTFT 68.889 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
43Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,16 tok/s TG
Prefill 149 · TTFT 69.107 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
44Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,71 tok/s TG
Prefill 95 · TTFT 22.448 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
45Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,54 tok/s TG
Prefill 143 · TTFT 70.040 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
46Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,47 tok/s TG
Prefill 143 · TTFT 69.846 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
47Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,73 tok/s TG
Prefill 95 · TTFT 22.362 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
48NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
32,59 tok/s TG
Prefill 122 · TTFT 237.566 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
49NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,79 tok/s TG
Prefill 109 · TTFT 133.175 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
50NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
12,12 tok/s TG
Prefill 98 · TTFT 21.966 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
51Mistral-Small-4-119B-2603119BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
62,26 tok/s TG
Prefill 284 · TTFT 238.161 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
52Mistral-Small-4-119B-2603119BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
39,36 tok/s TG
Prefill 221 · TTFT 103.372 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
53Mistral-Small-4-119B-2603119BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
16,70 tok/s TG
Prefill 135 · TTFT 20.068 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
54Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,77 tok/s TG
Prefill 156 · TTFT 56.974 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
55Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,78 tok/s TG
Prefill 173 · TTFT 53.622 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
56Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,52 tok/s TG
Prefill 148 · TTFT 16.554 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
57Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
25,18 tok/s TG
Prefill 261 · TTFT 255.650 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
58Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
14,12 tok/s TG
Prefill 239 · TTFT 121.204 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
59Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,41 tok/s TG
Prefill 229 · TTFT 10.683 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
60Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,39 tok/s TG
Prefill 172 · TTFT 265.241 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
61Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
14,39 tok/s TG
Prefill 171 · TTFT 124.363 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
62Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,69 tok/s TG
Prefill 161 · TTFT 13.385 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
63Seed-OSS-36B-Instruct36BByteDance Seed Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
39,21 tok/s TG
Prefill 438 · TTFT 205.680 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
64Seed-OSS-36B-Instruct36BByteDance Seed Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
22,83 tok/s TG
Prefill 421 · TTFT 75.552 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
65Seed-OSS-36B-Instruct36BByteDance Seed Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
7,49 tok/s TG
Prefill 409 · TTFT 5.396 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
66Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
142,87 tok/s TG
Prefill 498 · TTFT 91.483 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
67Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
83,72 tok/s TG
Prefill 448 · TTFT 33.704 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
68Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,85 tok/s TG
Prefill 375 · TTFT 5.208 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
69Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
142,20 tok/s TG
Prefill 503 · TTFT 90.967 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
70Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
81,40 tok/s TG
Prefill 445 · TTFT 34.520 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
71Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,95 tok/s TG
Prefill 421 · TTFT 4.580 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
72Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
140,11 tok/s TG
Prefill 524 · TTFT 90.775 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
73Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
82,00 tok/s TG
Prefill 451 · TTFT 34.179 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
74Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
30,30 tok/s TG
Prefill 431 · TTFT 4.485 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
75Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
45,31 tok/s TG
Prefill 644 · TTFT 217.710 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
76Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
25,71 tok/s TG
Prefill 662 · TTFT 64.391 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
77Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,46 tok/s TG
Prefill 453 · TTFT 5.425 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
78Qwen2.5-32B-Instruct32.8BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,95 tok/s TG
Prefill 698 · TTFT 216.065 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawAWQ
79Qwen2.5-32B-Instruct32.8BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
25,69 tok/s TG
Prefill 709 · TTFT 63.754 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawAWQ
80Qwen2.5-32B-Instruct32.8BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,41 tok/s TG
Prefill 449 · TTFT 5.486 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawAWQ
81OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,93 tok/s TG
Prefill 707 · TTFT 215.096 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
82OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
25,80 tok/s TG
Prefill 706 · TTFT 63.420 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
83OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,46 tok/s TG
Prefill 449 · TTFT 5.444 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
84Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
153,96 tok/s TG
Prefill 1.395 · TTFT 74.722 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
85Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
116,57 tok/s TG
Prefill 1.336 · TTFT 23.791 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
86Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
63,07 tok/s TG
Prefill 856 · TTFT 2.843 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
87Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
155,85 tok/s TG
Prefill 1.036 · TTFT 78.057 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
88Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
117,91 tok/s TG
Prefill 941 · TTFT 26.171 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
89Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
63,10 tok/s TG
Prefill 865 · TTFT 2.812 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
90Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
157,00 tok/s TG
Prefill 1.473 · TTFT 72.295 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
91Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
118,55 tok/s TG
Prefill 1.537 · TTFT 22.290 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
92Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
63,81 tok/s TG
Prefill 958 · TTFT 2.541 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
93Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
155,77 tok/s TG
Prefill 1.107 · TTFT 76.761 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
94Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
118,04 tok/s TG
Prefill 1.033 · TTFT 25.146 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
95Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
63,75 tok/s TG
Prefill 902 · TTFT 2.698 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
96Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
152,65 tok/s TG
Prefill 1.613 · TTFT 73.900 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
97Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
115,20 tok/s TG
Prefill 1.424 · TTFT 23.184 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
98Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
61,65 tok/s TG
Prefill 951 · TTFT 2.560 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
99NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
263,34 tok/s TG
Prefill 784 · TTFT 55.800 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
100NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
163,13 tok/s TG
Prefill 683 · TTFT 22.415 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
101NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
68,05 tok/s TG
Prefill 598 · TTFT 3.605 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
102Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
262,44 tok/s TG
Prefill 822 · TTFT 55.319 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
103Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
162,55 tok/s TG
Prefill 715 · TTFT 21.986 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
104Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
68,96 tok/s TG
Prefill 623 · TTFT 3.485 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
105Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
262,45 tok/s TG
Prefill 811 · TTFT 55.388 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
106Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
163,13 tok/s TG
Prefill 684 · TTFT 22.434 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
107Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
68,85 tok/s TG
Prefill 604 · TTFT 3.568 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
108Qwen3.6-27B27BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
57,21 tok/s TG
Prefill 470 · TTFT 180.290 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
109Qwen3.6-27B27BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
30,55 tok/s TG
Prefill 442 · TTFT 56.170 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
110Qwen3.6-27B27BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,83 tok/s TG
Prefill 430 · TTFT 4.494 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
111Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.009,04 tok/s TG
Prefill 4.090 · TTFT 12.734 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
112Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
585,92 tok/s TG
Prefill 3.535 · TTFT 4.487 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
113Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
189,07 tok/s TG
Prefill 3.110 · TTFT 638 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
114Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,50 tok/s TG
Prefill 2.121 · TTFT 85.480 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
115Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
59,83 tok/s TG
Prefill 1.470 · TTFT 24.544 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
116Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,15 tok/s TG
Prefill 1.266 · TTFT 2.131 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
117Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
62,88 tok/s TG
Prefill 1.114 · TTFT 148.957 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
118Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
34,87 tok/s TG
Prefill 1.172 · TTFT 41.913 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
119Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
12,98 tok/s TG
Prefill 695 · TTFT 3.957 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
120Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
590,01 tok/s TG
Prefill 5.490 · TTFT 17.999 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
121Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
312,21 tok/s TG
Prefill 5.010 · TTFT 5.594 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
122Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
84,82 tok/s TG
Prefill 3.484 · TTFT 649 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
123gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
677,50 tok/s TG
Prefill 2.978 · TTFT 18.549 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
124gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
360,92 tok/s TG
Prefill 2.636 · TTFT 6.319 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
125gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
92,78 tok/s TG
Prefill 2.595 · TTFT 761 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_XL
126Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
913,39 tok/s TG
Prefill 8.119 · TTFT 11.247 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
127Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
489,71 tok/s TG
Prefill 8.631 · TTFT 3.252 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
128Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
140,90 tok/s TG
Prefill 2.783 · TTFT 1.005 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
129Qwen3-Coder-Next79.7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
106,05 tok/s TG
Prefill 361 · TTFT 137.879 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
130Qwen3-Coder-Next79.7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
64,58 tok/s TG
Prefill 300 · TTFT 56.560 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
131Qwen3-Coder-Next79.7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
24,83 tok/s TG
Prefill 267 · TTFT 9.108 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
132North-Mini-Code-1.030.5BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
168,98 tok/s TG
Prefill 1.029 · TTFT 69.564 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
133North-Mini-Code-1.030.5BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
118,36 tok/s TG
Prefill 895 · TTFT 23.355 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
134North-Mini-Code-1.030.5BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
59,05 tok/s TG
Prefill 745 · TTFT 2.651 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawUD-Q4_K_M
135MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
15,16 tok/s TG
Prefill 152 · TTFT 287.291 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawBF16-INT4-AWQ
136MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
22,54 tok/s TG
Prefill 135 · TTFT 163.713 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawBF16-INT4-AWQ
137MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
11,45 tok/s TG
Prefill 84 · TTFT 27.256 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawBF16-INT4-AWQ
138MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
13,22 tok/s TG
Prefill 153 · TTFT 277.094 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
139MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,85 tok/s TG
Prefill 141 · TTFT 157.949 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
140MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
11,86 tok/s TG
Prefill 87 · TTFT 26.343 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
141Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
159,03 tok/s TG
Prefill 110 · TTFT 25.408 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
142Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
81,44 tok/s TG
Prefill 97 · TTFT 10.098 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
143Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,57 tok/s TG
Prefill 29 · TTFT 286 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
144Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
316,35 tok/s TG
Prefill 1.470 · TTFT 42.801 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
145Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
205,02 tok/s TG
Prefill 1.348 · TTFT 16.034 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
146Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,47 tok/s TG
Prefill 1.427 · TTFT 42.816 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
147Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
205,02 tok/s TG
Prefill 1.327 · TTFT 16.066 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
148Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
319,02 tok/s TG
Prefill 1.498 · TTFT 42.331 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
149Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
204,31 tok/s TG
Prefill 1.375 · TTFT 15.932 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
150Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
103,26 tok/s TG
Prefill 2.824 · TTFT 64.279 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
151Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
57,81 tok/s TG
Prefill 2.629 · TTFT 20.620 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
152Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,14 tok/s TG
Prefill 1.510 · TTFT 2.317 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
153gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.743,96 tok/s TG
Prefill 13.641 · TTFT 5.790 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
154gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
969,62 tok/s TG
Prefill 12.437 · TTFT 1.852 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
155gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
276,28 tok/s TG
Prefill 9.272 · TTFT 213 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
156Laguna-S-2.1117.6BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
42,33 tok/s TG
Prefill 153 · TTFT 257.193 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
157Laguna-S-2.1117.6BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
32,70 tok/s TG
Prefill 134 · TTFT 130.995 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
158Laguna-S-2.1117.6BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
12,62 tok/s TG
Prefill 130 · TTFT 21.118 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawINT4
159Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,93 tok/s TG
Prefill 58 · TTFT 151.267 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawNVFP4
160Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,81 tok/s TG
Prefill 55 · TTFT 158.303 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawNVFP4
161Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,86 tok/s TG
Prefill 66 · TTFT 40.934 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawNVFP4
162Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,28 tok/s TG
Prefill 53 · TTFT 164.092 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
163Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,80 tok/s TG
Prefill 59 · TTFT 150.829 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
164Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,84 tok/s TG
Prefill 66 · TTFT 41.139 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
165Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,20 tok/s TG
Prefill 57 · TTFT 151.603 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawFP8
166Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,93 tok/s TG
Prefill 59 · TTFT 150.457 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawFP8
167Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,98 tok/s TG
Prefill 61 · TTFT 44.934 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawFP8
168Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,50 tok/s TG
Prefill 57 · TTFT 152.613 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
169Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
7,00 tok/s TG
Prefill 59 · TTFT 151.350 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
170Laguna-M.1225.8BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,80 tok/s TG
Prefill 65 · TTFT 41.504 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
171gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.210,29 tok/s TG
Prefill 9.745 · TTFT 8.399 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
172gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
710,36 tok/s TG
Prefill 8.954 · TTFT 2.695 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
173gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
242,32 tok/s TG
Prefill 6.971 · TTFT 284 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
174gpt-oss-120b120BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
60,05 tok/s TG
Prefill 224 · TTFT 126.906 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
175gpt-oss-120b120BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
53,10 tok/s TG
Prefill 190 · TTFT 71.280 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
176gpt-oss-120b120BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,92 tok/s TG
Prefill 170 · TTFT 11.622 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
177GLM-4.5-Air106BZ.ai (Zhipu) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,60 tok/s TG
Prefill 108 · TTFT 79.520 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
178GLM-4.5-Air106BZ.ai (Zhipu) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,34 tok/s TG
Prefill 107 · TTFT 80.161 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
179GLM-4.5-Air106BZ.ai (Zhipu) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,87 tok/s TG
Prefill 112 · TTFT 19.038 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
180Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
103,94 tok/s TG
Prefill 2.848 · TTFT 64.620 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
181Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
57,59 tok/s TG
Prefill 2.617 · TTFT 21.158 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
182command-a-reasoning-08-2025111BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,57 tok/s TG
Prefill 98 · TTFT 86.763 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
183command-a-reasoning-08-2025111BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,67 tok/s TG
Prefill 99 · TTFT 85.963 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
184command-a-reasoning-08-2025111BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,16 tok/s TG
Prefill 128 · TTFT 21.272 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
185NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.196,98 tok/s TG
Prefill 6.074 · TTFT 10.280 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
186NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
676,42 tok/s TG
Prefill 5.367 · TTFT 3.579 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
187NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
233,69 tok/s TG
Prefill 2.376 · TTFT 950 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
188gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.082,39 tok/s TG
Prefill 6.941 · TTFT 9.659 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
189gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
592,13 tok/s TG
Prefill 5.662 · TTFT 3.385 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
190gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
165,04 tok/s TG
Prefill 2.897 · TTFT 711 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
191gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.743,14 tok/s TG
Prefill 12.148 · TTFT 5.881 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
192gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
971,35 tok/s TG
Prefill 9.983 · TTFT 2.015 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
193Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
41,87 tok/s TG
Prefill 1.695 · TTFT 20.797 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
194Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,62 tok/s TG
Prefill 1.223 · TTFT 11.695 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
195Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
14,82 tok/s TG
Prefill 509 · TTFT 4.908 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
196Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,19 tok/s TG
Prefill 1.424 · TTFT 20.551 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
197Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,61 tok/s TG
Prefill 1.219 · TTFT 11.738 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
198Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
15,11 tok/s TG
Prefill 622 · TTFT 3.910 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
199Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,73 tok/s TG
Prefill 816 · TTFT 35.129 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
200Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
18,94 tok/s TG
Prefill 516 · TTFT 20.783 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
201Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,61 tok/s TG
Prefill 342 · TTFT 5.738 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
202Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
30,59 tok/s TG
Prefill 819 · TTFT 35.012 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
203Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,05 tok/s TG
Prefill 517 · TTFT 20.708 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
204Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,63 tok/s TG
Prefill 345 · TTFT 5.672 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
205Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,41 tok/s TG
Prefill 1.525 · TTFT 19.257 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
206Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
22,64 tok/s TG
Prefill 1.305 · TTFT 10.957 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
207Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
16,34 tok/s TG
Prefill 672 · TTFT 3.622 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
208Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
497,64 tok/s TG
Prefill 8.340 · TTFT 5.360 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
209Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,66 tok/s TG
Prefill 5.742 · TTFT 2.122 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
210Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
85,15 tok/s TG
Prefill 3.218 · TTFT 666 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclawAWQ
211gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
24,73 tok/s TG
Prefill 660 · TTFT 34.336 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
212gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,13 tok/s TG
Prefill 488 · TTFT 20.357 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
213gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
7,29 tok/s TG
Prefill 446 · TTFT 4.523 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
214gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
486,20 tok/s TG
Prefill 4.345 · TTFT 5.863 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
215gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
315,66 tok/s TG
Prefill 3.728 · TTFT 2.722 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
216gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
75,04 tok/s TG
Prefill 3.021 · TTFT 659 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
217gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.022,79 tok/s TG
Prefill 18.744 · TTFT 2.204 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
218gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
616,01 tok/s TG
Prefill 13.778 · TTFT 803 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
219gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
144,58 tok/s TG
Prefill 7.078 · TTFT 281 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
220gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.608,46 tok/s TG
Prefill 31.452 · TTFT 1.633 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
221gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
973,54 tok/s TG
Prefill 23.599 · TTFT 460 ms
5×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw
222gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
228,18 tok/s TG
Prefill 11.075 · TTFT 179 ms
1×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMvLLMgodclaw