Created bymario-alka.dePowered bynoob2claw.detricoma.de
🏆
Rankings

LLM Leaderboard

Real measurements of every tested language model on real, named hardware – rated by raw speed (performance in tokens per second, prefill and time to first token) and by practical task quality in complete agent and chat runs (harness). Pick a benchmark type below or filter by model, maker and hardware to see exactly what is tested and how the results are produced.

⚡ Performance (tok/s)🤖 Harness quality👥 Concurrency🖥️ real hardware
Reset
Metric:
Size
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,20 tok/s TG
Prefill 261 · TTFT 8.641 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
2Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,93 tok/s TG
Prefill 290 · TTFT 8.335 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
3Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,90 tok/s TG
Prefill 266 · TTFT 8.275 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
4Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,69 tok/s TG
Prefill 427 · TTFT 8.728 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
5Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,65 tok/s TG
Prefill 325 · TTFT 8.859 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
6Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
17,33 tok/s TG
Prefill 621 · TTFT 3.879 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
7Mamba-Codestral-7B-v0.17BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,25 tok/s TG
Prefill 327 · TTFT 8.424 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
8Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,85 tok/s TG
Prefill 276 · TTFT 9.964 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
9Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,73 tok/s TG
Prefill 245 · TTFT 10.366 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
10Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,47 tok/s TG
Prefill 226 · TTFT 10.329 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
11Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,89 tok/s TG
Prefill 241 · TTFT 11.388 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0
12Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
623,04 tok/s TG
Prefill 6.697 · TTFT 15.390 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
13Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
646,94 tok/s TG
Prefill 10.489 · TTFT 10.971 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
14Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
615,23 tok/s TG
Prefill 13.720 · TTFT 797 ms
5×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclawAWQ
15Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
709,24 tok/s TG
Prefill 11.744 · TTFT 2.430 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclawAWQ
16Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
728,31 tok/s TG
Prefill 10.081 · TTFT 2.619 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclawAWQ
17Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.055,44 tok/s TG
Prefill 10.341 · TTFT 9.706 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
18Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,80 tok/s TG
Prefill 136 · TTFT 225.066 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
19Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,46 tok/s TG
Prefill 96 · TTFT 123.967 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
20Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
0,75 tok/s TG
Prefill 72 · TTFT 29.692 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
21Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,14 tok/s TG
Prefill 145 · TTFT 210.728 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
22Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,70 tok/s TG
Prefill 124 · TTFT 113.211 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
23Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
0,78 tok/s TG
Prefill 70 · TTFT 30.322 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
24Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
162,67 tok/s TG
Prefill 4.605 · TTFT 6.779 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
25Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
104,62 tok/s TG
Prefill 3.528 · TTFT 3.829 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
26Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
41,51 tok/s TG
Prefill 1.815 · TTFT 1.180 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
27Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
161,82 tok/s TG
Prefill 5.708 · TTFT 7.791 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
28Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
126,19 tok/s TG
Prefill 4.087 · TTFT 4.824 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
29Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
45,56 tok/s TG
Prefill 2.801 · TTFT 1.230 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
30Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
170,04 tok/s TG
Prefill 3.644 · TTFT 7.723 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
31Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
111,39 tok/s TG
Prefill 2.569 · TTFT 4.744 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
32Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
45,30 tok/s TG
Prefill 1.770 · TTFT 1.272 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
33Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
170,07 tok/s TG
Prefill 4.885 · TTFT 6.913 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
34Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,92 tok/s TG
Prefill 3.632 · TTFT 3.888 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
35Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
45,80 tok/s TG
Prefill 1.823 · TTFT 1.259 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
36Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
154,06 tok/s TG
Prefill 4.443 · TTFT 9.034 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
37Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
102,38 tok/s TG
Prefill 3.496 · TTFT 4.754 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
38Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
40,39 tok/s TG
Prefill 1.837 · TTFT 1.479 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
39Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
204,85 tok/s TG
Prefill 5.517 · TTFT 5.147 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
40Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
134,94 tok/s TG
Prefill 4.042 · TTFT 3.043 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
41Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
54,41 tok/s TG
Prefill 2.690 · TTFT 841 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
42Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
153,12 tok/s TG
Prefill 4.348 · TTFT 7.017 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
43Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
86,87 tok/s TG
Prefill 3.428 · TTFT 3.916 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
44Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
38,51 tok/s TG
Prefill 1.549 · TTFT 1.390 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
45Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
157,87 tok/s TG
Prefill 8.091 · TTFT 1.495 ms
5×NVIDIA GeForce RTX 4090AMD Ryzen 3 3100 4-Core Processor · 61 GB RAMllama.cppopenclaw_cliQ8_0
46Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
50,31 tok/s TG
Prefill 5.478 · TTFT 413 ms
1×NVIDIA GeForce RTX 4090AMD Ryzen 3 3100 4-Core Processor · 61 GB RAMllama.cppopenclaw_cliQ8_0
47Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
265,94 tok/s TG
Prefill 8.432 · TTFT 3.619 ms
10×NVIDIA GeForce RTX 4090AMD Ryzen 3 3100 4-Core Processor · 61 GB RAMllama.cppopenclaw_cliQ4_K_M
48Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
186,91 tok/s TG
Prefill 6.423 · TTFT 2.079 ms
5×NVIDIA GeForce RTX 4090AMD Ryzen 3 3100 4-Core Processor · 61 GB RAMllama.cppopenclaw_cliQ4_K_M
49Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,02 tok/s TG
Prefill 3.516 · TTFT 610 ms
1×NVIDIA GeForce RTX 4090AMD Ryzen 3 3100 4-Core Processor · 61 GB RAMllama.cppopenclaw_cliQ4_K_M
50Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,20 tok/s TG
Prefill 3.616 · TTFT 593 ms
1×NVIDIA GeForce RTX 4090AMD Ryzen 3 3100 4-Core Processor · 61 GB RAMllama.cppopenclaw_cliQ4_K_M
51Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
151,50 tok/s TG
Prefill 3.183 · TTFT 10.690 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
52Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
99,04 tok/s TG
Prefill 2.389 · TTFT 6.000 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
53Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
46,65 tok/s TG
Prefill 1.245 · TTFT 1.845 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
54Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
220,09 tok/s TG
Prefill 4.004 · TTFT 7.026 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0
55Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,54 tok/s TG
Prefill 2.918 · TTFT 4.227 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0
56Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,19 tok/s TG
Prefill 1.906 · TTFT 1.189 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0
57Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
226,87 tok/s TG
Prefill 3.972 · TTFT 7.112 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
58Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
153,79 tok/s TG
Prefill 2.885 · TTFT 4.280 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
59Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
67,54 tok/s TG
Prefill 2.060 · TTFT 1.099 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
60Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
155,02 tok/s TG
Prefill 2.329 · TTFT 12.088 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
61Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
101,60 tok/s TG
Prefill 1.687 · TTFT 7.279 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
62Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
46,82 tok/s TG
Prefill 1.231 · TTFT 1.830 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
63Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
156,79 tok/s TG
Prefill 4.188 · TTFT 11.390 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
64Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
107,29 tok/s TG
Prefill 2.668 · TTFT 7.432 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
65Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
46,30 tok/s TG
Prefill 1.883 · TTFT 1.835 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
66Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,92 tok/s TG
Prefill 2.977 · TTFT 10.664 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
67Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
98,13 tok/s TG
Prefill 2.200 · TTFT 6.014 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
68Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
45,06 tok/s TG
Prefill 1.195 · TTFT 1.788 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
69Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
134,10 tok/s TG
Prefill 1.552 · TTFT 19.988 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
70Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
91,65 tok/s TG
Prefill 2.058 · TTFT 7.712 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
71Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,45 tok/s TG
Prefill 1.174 · TTFT 2.313 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
72Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
141,77 tok/s TG
Prefill 3.307 · TTFT 10.899 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
73Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
87,68 tok/s TG
Prefill 2.867 · TTFT 6.226 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
74Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
32,37 tok/s TG
Prefill 1.631 · TTFT 1.662 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
75Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
135,40 tok/s TG
Prefill 2.502 · TTFT 11.327 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
76Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,60 tok/s TG
Prefill 1.791 · TTFT 6.870 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
77Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
33,61 tok/s TG
Prefill 1.301 · TTFT 1.735 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
78Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,30 tok/s TG
Prefill 1.554 · TTFT 19.578 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
79Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
74,57 tok/s TG
Prefill 2.319 · TTFT 7.067 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
80Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
33,45 tok/s TG
Prefill 1.304 · TTFT 2.089 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
81Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
193,37 tok/s TG
Prefill 4.316 · TTFT 6.560 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
82Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
118,88 tok/s TG
Prefill 3.109 · TTFT 3.967 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
83Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
50,43 tok/s TG
Prefill 2.188 · TTFT 1.034 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
84Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
142,79 tok/s TG
Prefill 4.374 · TTFT 10.633 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
85Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
87,06 tok/s TG
Prefill 3.482 · TTFT 6.548 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
86Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
33,37 tok/s TG
Prefill 2.019 · TTFT 1.676 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
87Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
133,08 tok/s TG
Prefill 3.434 · TTFT 9.975 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
88Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,06 tok/s TG
Prefill 2.578 · TTFT 5.626 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
89Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
33,50 tok/s TG
Prefill 1.317 · TTFT 1.755 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
90Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
52,41 tok/s TG
Prefill 1.260 · TTFT 19.903 ms
10×NVIDIA Tesla V100 PCIe 32GBAMD Ryzen 3 PRO 3200GE w/ Radeon Vega Graphics · 61 GB RAMllama.cppgodclawQ8_0
91Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
52,78 tok/s TG
Prefill 1.962 · TTFT 6.490 ms
5×NVIDIA Tesla V100 PCIe 32GBAMD Ryzen 3 PRO 3200GE w/ Radeon Vega Graphics · 61 GB RAMllama.cppgodclawQ8_0
92Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
35,13 tok/s TG
Prefill 2.320 · TTFT 975 ms
1×NVIDIA Tesla V100 PCIe 32GBAMD Ryzen 3 PRO 3200GE w/ Radeon Vega Graphics · 61 GB RAMllama.cppgodclawQ8_0
93Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
127,71 tok/s TG
Prefill 1.369 · TTFT 22.194 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
94Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
61,43 tok/s TG
Prefill 1.440 · TTFT 10.139 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
95Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,54 tok/s TG
Prefill 806 · TTFT 3.373 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
96Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
141,93 tok/s TG
Prefill 1.385 · TTFT 19.489 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
97Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
81,21 tok/s TG
Prefill 1.020 · TTFT 11.554 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
98Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,04 tok/s TG
Prefill 604 · TTFT 3.801 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
99Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
152,96 tok/s TG
Prefill 891 · TTFT 26.016 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
100Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
84,60 tok/s TG
Prefill 796 · TTFT 14.144 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
101Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,02 tok/s TG
Prefill 598 · TTFT 3.766 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
102Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
143,29 tok/s TG
Prefill 1.967 · TTFT 20.264 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
103Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
91,82 tok/s TG
Prefill 1.642 · TTFT 12.961 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
104Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,76 tok/s TG
Prefill 945 · TTFT 3.652 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
105Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
138,44 tok/s TG
Prefill 1.562 · TTFT 19.650 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
106Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,75 tok/s TG
Prefill 1.013 · TTFT 11.630 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
107Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,69 tok/s TG
Prefill 600 · TTFT 3.579 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
108Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
127,28 tok/s TG
Prefill 1.097 · TTFT 28.208 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
109Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,52 tok/s TG
Prefill 1.067 · TTFT 13.620 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
110Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,55 tok/s TG
Prefill 607 · TTFT 4.493 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
111Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
223,59 tok/s TG
Prefill 2.319 · TTFT 10.040 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
112Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
100,43 tok/s TG
Prefill 2.076 · TTFT 5.460 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
113Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
50,09 tok/s TG
Prefill 1.557 · TTFT 1.455 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
114Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
89,30 tok/s TG
Prefill 4.368 · TTFT 9.029 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMvLLMgodclawAWQ
115Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
47,24 tok/s TG
Prefill 3.416 · TTFT 5.086 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMvLLMgodclawAWQ
116Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
10,00 tok/s TG
Prefill 1.583 · TTFT 1.597 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMvLLMgodclawAWQ
117Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
271,15 tok/s TG
Prefill 1.523 · TTFT 12.453 ms
10×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
118Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
126,92 tok/s TG
Prefill 1.616 · TTFT 7.001 ms
5×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
119Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
43,40 tok/s TG
Prefill 1.066 · TTFT 2.179 ms
1×2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
120Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
272,78 tok/s TG
Prefill 1.270 · TTFT 14.950 ms
10×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
121Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
188,33 tok/s TG
Prefill 1.184 · TTFT 17.577 ms
10×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
122Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
196,80 tok/s TG
Prefill 3.639 · TTFT 17.840 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
123Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
104,97 tok/s TG
Prefill 3.091 · TTFT 4.029 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
124Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
42,51 tok/s TG
Prefill 1.809 · TTFT 1.187 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
125Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
214,87 tok/s TG
Prefill 3.898 · TTFT 18.332 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
126Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,55 tok/s TG
Prefill 3.087 · TTFT 3.813 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
127Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,40 tok/s TG
Prefill 1.773 · TTFT 1.295 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
128Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
192,56 tok/s TG
Prefill 2.801 · TTFT 18.904 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
129Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
112,45 tok/s TG
Prefill 2.423 · TTFT 4.657 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
130Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,41 tok/s TG
Prefill 1.819 · TTFT 1.240 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
131Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
205,47 tok/s TG
Prefill 5.429 · TTFT 14.556 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
132Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
124,85 tok/s TG
Prefill 3.953 · TTFT 4.557 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
133Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
43,95 tok/s TG
Prefill 2.859 · TTFT 1.207 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
134Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
536,72 tok/s TG
Prefill 9.810 · TTFT 3.224 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
135Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
329,05 tok/s TG
Prefill 8.161 · TTFT 1.778 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
136Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,96 tok/s TG
Prefill 3.773 · TTFT 568 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
137Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
534,84 tok/s TG
Prefill 10.455 · TTFT 3.204 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
138Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
327,23 tok/s TG
Prefill 8.158 · TTFT 1.764 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
139Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,48 tok/s TG
Prefill 3.787 · TTFT 566 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
140Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
180,91 tok/s TG
Prefill 4.113 · TTFT 19.821 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
141Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
107,24 tok/s TG
Prefill 3.566 · TTFT 4.706 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
142Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,46 tok/s TG
Prefill 1.769 · TTFT 1.542 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
143Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
210,73 tok/s TG
Prefill 4.210 · TTFT 15.755 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
144Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
81,86 tok/s TG
Prefill 3.277 · TTFT 3.594 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
145Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,16 tok/s TG
Prefill 1.921 · TTFT 1.196 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
146Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
207,60 tok/s TG
Prefill 2.947 · TTFT 16.548 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
147Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
83,01 tok/s TG
Prefill 2.572 · TTFT 4.388 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
148Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,20 tok/s TG
Prefill 1.906 · TTFT 1.183 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
149Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
198,40 tok/s TG
Prefill 5.762 · TTFT 12.931 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
150Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
112,20 tok/s TG
Prefill 5.159 · TTFT 4.120 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
151Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,03 tok/s TG
Prefill 3.034 · TTFT 1.137 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
152Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
197,49 tok/s TG
Prefill 4.479 · TTFT 15.705 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
153Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
97,58 tok/s TG
Prefill 3.294 · TTFT 3.610 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
154Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
26,46 tok/s TG
Prefill 1.916 · TTFT 1.120 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
155Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
394,57 tok/s TG
Prefill 11.196 · TTFT 3.006 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
156Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
227,30 tok/s TG
Prefill 8.019 · TTFT 1.690 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
157Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
51,79 tok/s TG
Prefill 4.021 · TTFT 533 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
158Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
181,88 tok/s TG
Prefill 4.310 · TTFT 15.943 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
159Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,13 tok/s TG
Prefill 3.562 · TTFT 4.610 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
160Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,30 tok/s TG
Prefill 1.859 · TTFT 1.467 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
161Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
330,66 tok/s TG
Prefill 4.317 · TTFT 12.310 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
162Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
179,41 tok/s TG
Prefill 4.191 · TTFT 2.702 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
163Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
67,52 tok/s TG
Prefill 2.833 · TTFT 799 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
164Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
735,14 tok/s TG
Prefill 16.886 · TTFT 1.945 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
165Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
466,27 tok/s TG
Prefill 13.248 · TTFT 1.078 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
166Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
117,98 tok/s TG
Prefill 5.926 · TTFT 361 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
167Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
99,29 tok/s TG
Prefill 9.684 · TTFT 3.553 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
168Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
51,37 tok/s TG
Prefill 7.945 · TTFT 1.984 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
169Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
10,63 tok/s TG
Prefill 3.580 · TTFT 655 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
170Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
300,82 tok/s TG
Prefill 4.868 · TTFT 10.813 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
171Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
132,76 tok/s TG
Prefill 4.413 · TTFT 2.568 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
172Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
43,96 tok/s TG
Prefill 3.170 · TTFT 714 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
173Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,63 tok/s TG
Prefill 582 · TTFT 3.807 ms
1×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
174Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
63,76 tok/s TG
Prefill 821 · TTFT 12.234 ms
5×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
175Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,98 tok/s TG
Prefill 604 · TTFT 3.558 ms
1×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
176Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
87,91 tok/s TG
Prefill 1.493 · TTFT 6.317 ms
5×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
177Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,80 tok/s TG
Prefill 1.555 · TTFT 1.457 ms
1×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ4_K_M
178Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
98,68 tok/s TG
Prefill 1.242 · TTFT 8.170 ms
5×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
179Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,20 tok/s TG
Prefill 1.206 · TTFT 1.877 ms
1×AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAMllama.cppgodclawQ8_0
180Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
194,44 tok/s TG
Prefill 2.527 · TTFT 19.500 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
181Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
113,19 tok/s TG
Prefill 2.335 · TTFT 4.870 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
182Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,45 tok/s TG
Prefill 1.820 · TTFT 1.238 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
183Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
207,02 tok/s TG
Prefill 2.897 · TTFT 16.703 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
184Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
83,06 tok/s TG
Prefill 2.594 · TTFT 4.350 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
185Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,20 tok/s TG
Prefill 1.896 · TTFT 1.189 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
186Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
333,85 tok/s TG
Prefill 4.298 · TTFT 12.227 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
187Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
181,01 tok/s TG
Prefill 4.232 · TTFT 2.679 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
188Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
67,52 tok/s TG
Prefill 3.057 · TTFT 740 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
189Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
313,09 tok/s TG
Prefill 4.977 · TTFT 10.633 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
190Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
132,91 tok/s TG
Prefill 4.455 · TTFT 2.545 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
191Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
43,99 tok/s TG
Prefill 3.163 · TTFT 715 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
192Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
197,53 tok/s TG
Prefill 4.241 · TTFT 18.028 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
193Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,45 tok/s TG
Prefill 3.019 · TTFT 3.678 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
194Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
42,52 tok/s TG
Prefill 1.818 · TTFT 1.180 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
195Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
202,30 tok/s TG
Prefill 4.589 · TTFT 17.666 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
196Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,65 tok/s TG
Prefill 3.146 · TTFT 3.814 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
197Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,39 tok/s TG
Prefill 1.817 · TTFT 1.263 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
198Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
181,70 tok/s TG
Prefill 5.490 · TTFT 14.312 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
199Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
125,38 tok/s TG
Prefill 4.946 · TTFT 4.287 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
200Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,01 tok/s TG
Prefill 2.864 · TTFT 1.204 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
201Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
492,58 tok/s TG
Prefill 9.647 · TTFT 3.456 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
202Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
306,77 tok/s TG
Prefill 7.576 · TTFT 1.901 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
203Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
75,27 tok/s TG
Prefill 3.394 · TTFT 631 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
204Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
492,95 tok/s TG
Prefill 9.614 · TTFT 3.459 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
205Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
306,07 tok/s TG
Prefill 7.668 · TTFT 1.898 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
206Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
75,04 tok/s TG
Prefill 3.417 · TTFT 628 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
207Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
182,92 tok/s TG
Prefill 4.049 · TTFT 17.792 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
208Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,27 tok/s TG
Prefill 3.206 · TTFT 4.680 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
209Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
44,54 tok/s TG
Prefill 1.774 · TTFT 1.537 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
210Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
211,54 tok/s TG
Prefill 4.003 · TTFT 15.863 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
211Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
81,95 tok/s TG
Prefill 3.318 · TTFT 3.548 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
212Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,18 tok/s TG
Prefill 1.937 · TTFT 1.185 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
213Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
190,07 tok/s TG
Prefill 5.648 · TTFT 12.906 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
214Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
112,88 tok/s TG
Prefill 4.206 · TTFT 4.283 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
215Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,03 tok/s TG
Prefill 3.042 · TTFT 1.134 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
216Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
198,71 tok/s TG
Prefill 4.358 · TTFT 15.452 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
217Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
97,66 tok/s TG
Prefill 3.328 · TTFT 3.570 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
218Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
26,46 tok/s TG
Prefill 1.921 · TTFT 1.117 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
219Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
371,52 tok/s TG
Prefill 10.318 · TTFT 3.250 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
220Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
215,90 tok/s TG
Prefill 8.081 · TTFT 1.805 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
221Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,83 tok/s TG
Prefill 3.624 · TTFT 593 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
222Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
187,04 tok/s TG
Prefill 4.177 · TTFT 15.498 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
223Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,07 tok/s TG
Prefill 3.492 · TTFT 4.570 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
224Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,31 tok/s TG
Prefill 1.876 · TTFT 1.453 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
225Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
655,08 tok/s TG
Prefill 14.863 · TTFT 2.211 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
226Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
423,25 tok/s TG
Prefill 11.522 · TTFT 1.241 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
227Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,68 tok/s TG
Prefill 5.117 · TTFT 419 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
228Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
98,26 tok/s TG
Prefill 9.468 · TTFT 3.791 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
229Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
51,15 tok/s TG
Prefill 7.311 · TTFT 2.155 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
230Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
10,64 tok/s TG
Prefill 3.337 · TTFT 704 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
231Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
277,83 tok/s TG
Prefill 5.287 · TTFT 5.932 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
232Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
168,80 tok/s TG
Prefill 4.187 · TTFT 3.262 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
233Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
40,46 tok/s TG
Prefill 2.095 · TTFT 1.025 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
234Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
278,10 tok/s TG
Prefill 5.436 · TTFT 5.880 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
235Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
168,52 tok/s TG
Prefill 4.154 · TTFT 3.278 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
236Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
40,37 tok/s TG
Prefill 2.099 · TTFT 1.022 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
237Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
197,27 tok/s TG
Prefill 5.765 · TTFT 5.461 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
238Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
112,18 tok/s TG
Prefill 4.404 · TTFT 3.055 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
239Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
25,18 tok/s TG
Prefill 2.281 · TTFT 942 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
240Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
401,40 tok/s TG
Prefill 9.462 · TTFT 3.339 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
241Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
253,96 tok/s TG
Prefill 7.169 · TTFT 1.866 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
242Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
63,44 tok/s TG
Prefill 3.611 · TTFT 594 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
243Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
46,72 tok/s TG
Prefill 5.554 · TTFT 6.192 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
244Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
24,14 tok/s TG
Prefill 4.232 · TTFT 3.475 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
245Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
5,01 tok/s TG
Prefill 2.028 · TTFT 1.155 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
246Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
122,95 tok/s TG
Prefill 3.286 · TTFT 8.287 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
247Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
77,55 tok/s TG
Prefill 2.405 · TTFT 4.874 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
248Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
23,78 tok/s TG
Prefill 1.882 · TTFT 1.220 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
249Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
126,89 tok/s TG
Prefill 2.125 · TTFT 10.798 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
250Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,65 tok/s TG
Prefill 1.940 · TTFT 5.826 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0