Created bymario-alka.dePowered bynoob2claw.detricoma.de
🏆
Ranglisten

LLM Leaderboard

Reale Messergebnisse aller getesteten Sprachmodelle auf echter, benannter Hardware – bewertet nach roher Geschwindigkeit (Performance in Token pro Sekunde, Prefill und Zeit bis zum ersten Token) und nach praktischer Aufgaben-Qualität in vollständigen Agenten- und Chat-Abläufen (Harness). Wähle unten einen Benchmark-Typ oder filtere nach Modell, Hersteller und Hardware, um genau zu sehen, was jeweils geprüft wird und wie die Ergebnisse zustande kommen.

⚡ Performance (tok/s)🤖 Harness-Qualität👥 Concurrency🖥️ echte Hardware
Reset
Messwert:
Größe
GPUs
#Modell / HerstellerMesswerteParallelGPU / CPU / RAMRuntime
1Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
317,00 tok/s TG
Prefill 14.849 · TTFT 2.492 ms
10×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 126 GB RAMvLLMgodclaw
2Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
183,35 tok/s TG
Prefill 11.565 · TTFT 1.219 ms
5×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 126 GB RAMvLLMgodclaw
3Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
42,39 tok/s TG
Prefill 5.909 · TTFT 412 ms
1×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 126 GB RAMvLLMgodclaw
4Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
242,41 tok/s TG
Prefill 4.265 · TTFT 17.947 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 3945WX 12-CoresstrataIQ2_XS
5Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
139,55 tok/s TG
Prefill 3.687 · TTFT 3.821 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 3945WX 12-CoresstrataIQ2_XS
6Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
115,48 tok/s TG
Prefill 1.754 · TTFT 1.124 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 3945WX 12-CoresstrataIQ2_XS
7Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
1.171,75 tok/s TG
Prefill 2.132 · TTFT 45.396 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 3945WX 12-CoresstrataIQ2_XS
8Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
589,46 tok/s TG
Prefill 2.120 · TTFT 21.049 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 3945WX 12-CoresstrataIQ2_XS
9Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
125,24 tok/s TG
Prefill 1.458 · TTFT 1.399 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 3945WX 12-CoresstrataIQ2_XS
10Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
79,19 tok/s TG
Prefill 670 · TTFT 33.965 ms
10×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 126 GB RAMllama.cppgodclawUD-Q4_K_XL
11Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
73,30 tok/s TG
Prefill 487 · TTFT 20.813 ms
5×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 126 GB RAMllama.cppgodclawUD-Q4_K_XL
12Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
38,84 tok/s TG
Prefill 258 · TTFT 10.963 ms
1×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 126 GB RAMllama.cppgodclawUD-Q4_K_XL
13Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
67,46 tok/s TG
Prefill 568 · TTFT 39.939 ms
10×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 110 GB RAMllama.cppgodclaw
14Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
55,85 tok/s TG
Prefill 415 · TTFT 24.406 ms
5×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 110 GB RAMllama.cppgodclaw
15Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
35,58 tok/s TG
Prefill 220 · TTFT 12.963 ms
1×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 110 GB RAMllama.cppgodclaw
16Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
201,32 tok/s TG
Prefill 2.330 · TTFT 11.380 ms
10×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 110 GB RAMllama.cppgodclaw
17Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
190,45 tok/s TG
Prefill 1.680 · TTFT 6.722 ms
5×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 110 GB RAMllama.cppgodclaw
18Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
90,29 tok/s TG
Prefill 1.380 · TTFT 1.577 ms
1×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 110 GB RAMllama.cppgodclaw
19Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
197,94 tok/s TG
Prefill 2.288 · TTFT 11.557 ms
10×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMllama.cppgodclawQ4_K_M
20Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
190,09 tok/s TG
Prefill 1.655 · TTFT 6.827 ms
5×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMllama.cppgodclawQ4_K_M
21Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
89,66 tok/s TG
Prefill 1.349 · TTFT 1.612 ms
1×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMllama.cppgodclawQ4_K_M
22Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
71,73 tok/s TG
Prefill 560 · TTFT 40.628 ms
10×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMllama.cppgodclawUD-Q4_K_XL
23Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
60,89 tok/s TG
Prefill 409 · TTFT 24.788 ms
5×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMllama.cppgodclawUD-Q4_K_XL
24Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
36,99 tok/s TG
Prefill 221 · TTFT 12.961 ms
1×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMllama.cppgodclawUD-Q4_K_XL
25Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
388,05 tok/s TG
Prefill 16.198 · TTFT 2.053 ms
10×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMvLLMgodclawINT4
26Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
343,90 tok/s TG
Prefill 9.629 · TTFT 1.973 ms
5×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMvLLMgodclawINT4
27Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
99,60 tok/s TG
Prefill 4.438 · TTFT 606 ms
1×2x NVIDIA RTX A6000 iAMD EPYC 7203P 8-Core Processor · 252 GB RAMvLLMgodclawINT4
28Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
194,68 tok/s TG
Prefill 1.934 · TTFT 12.719 ms
10×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
29Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
184,78 tok/s TG
Prefill 1.567 · TTFT 7.199 ms
5×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
30Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
87,74 tok/s TG
Prefill 1.275 · TTFT 1.916 ms
1×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
31Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
35,44 tok/s TG
Prefill 249 · TTFT 90.371 ms
10×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawUD-Q4_K_XL
32Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
32,77 tok/s TG
Prefill 160 · TTFT 63.351 ms
5×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawUD-Q4_K_XL
33Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
22,67 tok/s TG
Prefill 124 · TTFT 18.211 ms
1×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawUD-Q4_K_XL
34Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
25,97 tok/s TG
Prefill 204 · TTFT 110.341 ms
10×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_XL
35Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
19,36 tok/s TG
Prefill 158 · TTFT 64.302 ms
5×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_XL
36Qwen3.8-Flash-Next180BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
8,95 tok/s TG
Prefill 94 · TTFT 24.951 ms
1×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_XL
37Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
92,62 tok/s TG
Prefill 1.185 · TTFT 24.873 ms
10×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
38Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
73,50 tok/s TG
Prefill 900 · TTFT 14.075 ms
5×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
39Ling-3.0-flash127.5BinclusionAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
30,41 tok/s TG
Prefill 713 · TTFT 3.416 ms
1×2x NVIDIA RTX A6000AMD EPYC 7203P 8-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
40Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
626,41 tok/s TG
Prefill 28.681 · TTFT 928 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
41Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
418,21 tok/s TG
Prefill 21.951 · TTFT 535 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
42Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
152,55 tok/s TG
Prefill 8.176 · TTFT 245 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
43Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
469,10 tok/s TG
Prefill 5.207 · TTFT 8.770 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
44Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
209,15 tok/s TG
Prefill 5.053 · TTFT 1.909 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
45Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
125,55 tok/s TG
Prefill 3.700 · TTFT 521 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
46Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
600,91 tok/s TG
Prefill 5.037 · TTFT 7.660 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
47Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
256,43 tok/s TG
Prefill 4.992 · TTFT 1.933 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
48Ornith-1.5-35B-A3B35BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
159,15 tok/s TG
Prefill 3.734 · TTFT 517 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
49Ornith-1.5-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
424,22 tok/s TG
Prefill 4.823 · TTFT 8.833 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
50Ornith-1.5-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
178,14 tok/s TG
Prefill 4.756 · TTFT 2.026 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
51Ornith-1.5-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
70,63 tok/s TG
Prefill 3.632 · TTFT 531 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
52Ornith-1.5-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
472,21 tok/s TG
Prefill 4.673 · TTFT 9.026 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
53Ornith-1.5-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
238,93 tok/s TG
Prefill 4.648 · TTFT 2.074 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
54Ornith-1.5-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
105,97 tok/s TG
Prefill 3.502 · TTFT 550 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
55Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
153,61 tok/s TG
Prefill 1.063 · TTFT 74.931 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
56Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
79,80 tok/s TG
Prefill 936 · TTFT 25.459 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
57Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
21,83 tok/s TG
Prefill 830 · TTFT 2.957 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
58Qwen3-Coder-Next79.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
514,52 tok/s TG
Prefill 22.477 · TTFT 1.541 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
59Qwen3-Coder-Next79.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
365,08 tok/s TG
Prefill 17.147 · TTFT 895 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
60Qwen3-Coder-Next79.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
146,76 tok/s TG
Prefill 6.990 · TTFT 350 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
61Nemotron-H-47B-Reasoning-128K47BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
88,14 tok/s TG
Prefill 506 · TTFT 123.878 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
62Nemotron-H-47B-Reasoning-128K47BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
51,74 tok/s TG
Prefill 518 · TTFT 40.780 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
63Nemotron-H-47B-Reasoning-128K47BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
21,58 tok/s TG
Prefill 531 · TTFT 4.053 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
64Laguna-S-2.1117.6BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
356,42 tok/s TG
Prefill 16.357 · TTFT 2.480 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawINT4
65Laguna-S-2.1117.6BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
250,44 tok/s TG
Prefill 13.079 · TTFT 1.402 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawINT4
66Laguna-S-2.1117.6BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
109,52 tok/s TG
Prefill 5.943 · TTFT 462 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawINT4
67Qwen3.6-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
679,07 tok/s TG
Prefill 5.945 · TTFT 15.921 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
68Qwen3.6-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
374,19 tok/s TG
Prefill 5.776 · TTFT 4.793 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
69Qwen3.6-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
126,71 tok/s TG
Prefill 3.748 · TTFT 515 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
70command-a-reasoning-08-2025111BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
95,47 tok/s TG
Prefill 3.802 · TTFT 14.501 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
71command-a-reasoning-08-2025111BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
64,67 tok/s TG
Prefill 2.956 · TTFT 8.171 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
72command-a-reasoning-08-2025111BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
17,11 tok/s TG
Prefill 1.404 · TTFT 2.484 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
73GLM-4.5-Air106BZ.ai (Zhipu) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
276,46 tok/s TG
Prefill 11.804 · TTFT 2.833 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
74GLM-4.5-Air106BZ.ai (Zhipu) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
197,32 tok/s TG
Prefill 9.348 · TTFT 1.552 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
75GLM-4.5-Air106BZ.ai (Zhipu) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
90,64 tok/s TG
Prefill 3.471 · TTFT 630 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
76Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
838,55 tok/s TG
Prefill 6.016 · TTFT 13.397 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
77Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
460,94 tok/s TG
Prefill 5.733 · TTFT 4.159 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
78Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
153,16 tok/s TG
Prefill 3.678 · TTFT 525 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
79Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
200,28 tok/s TG
Prefill 1.419 · TTFT 55.883 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
80Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
104,97 tok/s TG
Prefill 1.343 · TTFT 18.020 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
81Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,90 tok/s TG
Prefill 1.033 · TTFT 2.135 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
82NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
755,10 tok/s TG
Prefill 6.964 · TTFT 14.586 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
83NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
448,68 tok/s TG
Prefill 6.418 · TTFT 4.484 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
84NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
174,14 tok/s TG
Prefill 4.417 · TTFT 488 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
85Nemotron-Cascade-2-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
723,68 tok/s TG
Prefill 7.027 · TTFT 14.700 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
86Nemotron-Cascade-2-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
435,36 tok/s TG
Prefill 6.438 · TTFT 4.605 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
87Nemotron-Cascade-2-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
171,10 tok/s TG
Prefill 4.389 · TTFT 495 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
88Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
806,94 tok/s TG
Prefill 7.252 · TTFT 13.902 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
89Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
463,93 tok/s TG
Prefill 6.756 · TTFT 4.404 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
90Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
152,50 tok/s TG
Prefill 4.956 · TTFT 555 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
91OpenReasoning-Nemotron-32B32BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
138,38 tok/s TG
Prefill 2.698 · TTFT 27.790 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
92OpenReasoning-Nemotron-32B32BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
77,23 tok/s TG
Prefill 1.954 · TTFT 6.520 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
93OpenReasoning-Nemotron-32B32BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
31,24 tok/s TG
Prefill 1.232 · TTFT 1.990 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
94gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
128,36 tok/s TG
Prefill 621 · TTFT 48.995 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
95gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
68,74 tok/s TG
Prefill 843 · TTFT 15.945 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
96gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,44 tok/s TG
Prefill 1.083 · TTFT 1.829 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
97North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
592,34 tok/s TG
Prefill 6.822 · TTFT 6.994 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
98North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
296,02 tok/s TG
Prefill 6.678 · TTFT 1.485 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
99North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
164,00 tok/s TG
Prefill 4.752 · TTFT 416 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
100Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
536,28 tok/s TG
Prefill 13.681 · TTFT 6.638 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
101Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
308,78 tok/s TG
Prefill 11.322 · TTFT 1.153 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
102Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
167,54 tok/s TG
Prefill 5.771 · TTFT 422 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
103Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
543,09 tok/s TG
Prefill 12.760 · TTFT 6.794 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
104Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
305,47 tok/s TG
Prefill 11.465 · TTFT 1.138 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
105Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
170,28 tok/s TG
Prefill 5.820 · TTFT 419 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
106Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
569,77 tok/s TG
Prefill 12.905 · TTFT 6.584 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
107Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
313,02 tok/s TG
Prefill 10.707 · TTFT 1.215 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
108Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
170,31 tok/s TG
Prefill 5.670 · TTFT 431 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
109Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
562,82 tok/s TG
Prefill 13.297 · TTFT 6.226 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
110Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
298,32 tok/s TG
Prefill 11.008 · TTFT 1.167 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
111Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
173,16 tok/s TG
Prefill 5.723 · TTFT 425 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
112Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
197,25 tok/s TG
Prefill 5.040 · TTFT 7.497 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
113Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
111,56 tok/s TG
Prefill 3.912 · TTFT 4.256 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
114Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
25,25 tok/s TG
Prefill 1.890 · TTFT 1.299 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
115Nemotron-H-47B-Reasoning-128K47BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
38,08 tok/s TG
Prefill 288 · TTFT 117.541 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
116Nemotron-H-47B-Reasoning-128K47BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
23,08 tok/s TG
Prefill 299 · TTFT 39.857 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
117Nemotron-H-47B-Reasoning-128K47BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
13,25 tok/s TG
Prefill 547 · TTFT 3.937 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
118Qwen3.6-27B27BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
161,50 tok/s TG
Prefill 1.437 · TTFT 28.025 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
119Qwen3.6-27B27BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
83,32 tok/s TG
Prefill 1.423 · TTFT 6.782 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
120Qwen3.6-27B27BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
36,26 tok/s TG
Prefill 1.170 · TTFT 1.652 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
121Qwen3-Coder-Next79.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
629,83 tok/s TG
Prefill 26.016 · TTFT 1.347 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
122Qwen3-Coder-Next79.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
410,37 tok/s TG
Prefill 19.698 · TTFT 777 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
123Qwen3-Coder-Next79.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
133,13 tok/s TG
Prefill 8.147 · TTFT 302 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
124Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
464,35 tok/s TG
Prefill 6.208 · TTFT 9.108 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
125Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
246,65 tok/s TG
Prefill 5.984 · TTFT 1.807 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
126Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
159,92 tok/s TG
Prefill 4.351 · TTFT 495 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
127Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
196,80 tok/s TG
Prefill 3.639 · TTFT 17.840 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
128Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
104,97 tok/s TG
Prefill 3.091 · TTFT 4.029 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
129Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
42,51 tok/s TG
Prefill 1.809 · TTFT 1.187 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
130Mistral-Small-3.1-24B-Instruct-250324BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
214,87 tok/s TG
Prefill 3.898 · TTFT 18.332 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
131Mistral-Small-3.1-24B-Instruct-250324BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
110,55 tok/s TG
Prefill 3.087 · TTFT 3.813 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
132Mistral-Small-3.1-24B-Instruct-250324BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
44,40 tok/s TG
Prefill 1.773 · TTFT 1.295 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
133Magistral-Small-250924BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
192,56 tok/s TG
Prefill 2.801 · TTFT 18.904 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
134Magistral-Small-250924BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
112,45 tok/s TG
Prefill 2.423 · TTFT 4.657 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
135Magistral-Small-250924BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
44,41 tok/s TG
Prefill 1.819 · TTFT 1.240 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
136Devstral-Small-250724BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
205,47 tok/s TG
Prefill 5.429 · TTFT 14.556 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
137Devstral-Small-250724BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
124,85 tok/s TG
Prefill 3.953 · TTFT 4.557 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
138Devstral-Small-250724BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
43,95 tok/s TG
Prefill 2.859 · TTFT 1.207 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
139Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
545,99 tok/s TG
Prefill 5.127 · TTFT 7.988 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
140Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
230,83 tok/s TG
Prefill 5.074 · TTFT 1.900 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
141Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
139,69 tok/s TG
Prefill 3.788 · TTFT 509 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
142Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
127,70 tok/s TG
Prefill 1.465 · TTFT 29.103 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
143Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
65,24 tok/s TG
Prefill 1.360 · TTFT 8.120 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
144Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
17,60 tok/s TG
Prefill 1.064 · TTFT 2.077 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
145Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
470,73 tok/s TG
Prefill 5.225 · TTFT 8.748 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
146Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
209,39 tok/s TG
Prefill 5.052 · TTFT 1.909 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
147Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
126,70 tok/s TG
Prefill 3.739 · TTFT 517 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
148Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
248,32 tok/s TG
Prefill 6.596 · TTFT 4.963 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
149Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
143,08 tok/s TG
Prefill 5.398 · TTFT 2.776 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
150Seed-OSS-36B-Instruct36BByteDance Seed PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
33,16 tok/s TG
Prefill 2.495 · TTFT 886 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
151Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
658,68 tok/s TG
Prefill 27.910 · TTFT 964 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
152Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
450,20 tok/s TG
Prefill 21.244 · TTFT 558 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
153Qwen3.5-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
172,47 tok/s TG
Prefill 8.214 · TTFT 241 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
154Qwen3.6-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
664,59 tok/s TG
Prefill 27.704 · TTFT 965 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
155Qwen3.6-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
453,61 tok/s TG
Prefill 21.156 · TTFT 557 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
156Qwen3.6-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
172,52 tok/s TG
Prefill 8.354 · TTFT 237 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
157Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
464,52 tok/s TG
Prefill 5.032 · TTFT 8.966 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
158Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
206,88 tok/s TG
Prefill 5.021 · TTFT 1.920 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
159Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
125,74 tok/s TG
Prefill 3.740 · TTFT 516 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
160Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
477,43 tok/s TG
Prefill 7.813 · TTFT 8.277 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
161Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
237,96 tok/s TG
Prefill 6.516 · TTFT 2.127 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
162Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
126,89 tok/s TG
Prefill 5.003 · TTFT 550 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
163Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
712,04 tok/s TG
Prefill 35.143 · TTFT 1.137 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
164Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
462,23 tok/s TG
Prefill 24.899 · TTFT 645 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
165Laguna-XS-2.133.4BPoolside PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
147,92 tok/s TG
Prefill 11.709 · TTFT 234 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
166gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
236,03 tok/s TG
Prefill 5.472 · TTFT 5.845 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
167gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
143,75 tok/s TG
Prefill 4.220 · TTFT 3.249 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
168gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
34,84 tok/s TG
Prefill 1.988 · TTFT 997 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
169Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
136,50 tok/s TG
Prefill 2.927 · TTFT 24.302 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
170Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
56,90 tok/s TG
Prefill 2.282 · TTFT 5.952 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
171Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
11,39 tok/s TG
Prefill 1.288 · TTFT 1.907 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
172OpenReasoning-Nemotron-32B32BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
143,98 tok/s TG
Prefill 2.767 · TTFT 24.248 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
173OpenReasoning-Nemotron-32B32BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
45,16 tok/s TG
Prefill 2.426 · TTFT 5.640 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
174OpenReasoning-Nemotron-32B32BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
19,26 tok/s TG
Prefill 1.308 · TTFT 1.876 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
175gpt-oss-20b20BOpenAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
685,28 tok/s TG
Prefill 9.018 · TTFT 5.356 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
176gpt-oss-20b20BOpenAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
324,88 tok/s TG
Prefill 8.394 · TTFT 1.177 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
177gpt-oss-20b20BOpenAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
194,38 tok/s TG
Prefill 5.661 · TTFT 349 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
178Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
262,60 tok/s TG
Prefill 7.019 · TTFT 5.005 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
179Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
153,33 tok/s TG
Prefill 5.312 · TTFT 2.861 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
180Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
35,99 tok/s TG
Prefill 2.848 · TTFT 865 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
181Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
425,61 tok/s TG
Prefill 14.121 · TTFT 8.163 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
182Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
213,02 tok/s TG
Prefill 10.916 · TTFT 1.253 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
183Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
128,73 tok/s TG
Prefill 5.762 · TTFT 423 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
184Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
515,26 tok/s TG
Prefill 31.910 · TTFT 1.107 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
185Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
367,00 tok/s TG
Prefill 24.948 · TTFT 625 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
186Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
156,20 tok/s TG
Prefill 10.821 · TTFT 225 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
187NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
403,66 tok/s TG
Prefill 6.688 · TTFT 9.919 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
188NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
216,68 tok/s TG
Prefill 6.212 · TTFT 1.740 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
189NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
145,08 tok/s TG
Prefill 4.478 · TTFT 481 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
190Nemotron-Cascade-2-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
391,02 tok/s TG
Prefill 6.784 · TTFT 9.603 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
191Nemotron-Cascade-2-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
216,78 tok/s TG
Prefill 6.257 · TTFT 1.743 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
192Nemotron-Cascade-2-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
145,15 tok/s TG
Prefill 4.443 · TTFT 489 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
193Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
406,79 tok/s TG
Prefill 6.308 · TTFT 9.948 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
194Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
217,44 tok/s TG
Prefill 6.099 · TTFT 1.773 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
195Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
145,19 tok/s TG
Prefill 4.389 · TTFT 491 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
196gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
251,07 tok/s TG
Prefill 7.025 · TTFT 4.495 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
197gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
152,19 tok/s TG
Prefill 5.535 · TTFT 2.477 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
198gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
36,87 tok/s TG
Prefill 2.641 · TTFT 751 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
199gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
127,17 tok/s TG
Prefill 646 · TTFT 44.975 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
200gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
52,93 tok/s TG
Prefill 870 · TTFT 15.730 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
201gemma-4-31B-it31BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
18,77 tok/s TG
Prefill 1.107 · TTFT 1.793 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
202Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
408,85 tok/s TG
Prefill 12.729 · TTFT 8.436 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
203Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
228,83 tok/s TG
Prefill 11.012 · TTFT 1.280 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
204Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
128,49 tok/s TG
Prefill 4.330 · TTFT 571 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
205Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
432,06 tok/s TG
Prefill 10.018 · TTFT 8.513 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
206Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
227,89 tok/s TG
Prefill 11.046 · TTFT 1.300 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
207Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
130,25 tok/s TG
Prefill 3.773 · TTFT 942 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
208Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
402,65 tok/s TG
Prefill 13.733 · TTFT 8.247 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
209Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
217,88 tok/s TG
Prefill 12.212 · TTFT 1.286 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
210Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
130,41 tok/s TG
Prefill 5.467 · TTFT 446 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
211Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
411,72 tok/s TG
Prefill 12.964 · TTFT 8.114 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
212Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
223,10 tok/s TG
Prefill 11.207 · TTFT 1.266 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
213Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
130,54 tok/s TG
Prefill 5.835 · TTFT 417 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
214North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
452,00 tok/s TG
Prefill 6.870 · TTFT 8.426 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
215North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
228,12 tok/s TG
Prefill 6.796 · TTFT 1.459 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
216North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
126,71 tok/s TG
Prefill 4.735 · TTFT 418 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
217Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
547,72 tok/s TG
Prefill 30.727 · TTFT 1.142 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
218Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
386,05 tok/s TG
Prefill 23.773 · TTFT 648 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
219Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
161,14 tok/s TG
Prefill 10.675 · TTFT 228 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
220Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
536,72 tok/s TG
Prefill 9.810 · TTFT 3.224 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
221Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
329,05 tok/s TG
Prefill 8.161 · TTFT 1.778 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
222Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
79,96 tok/s TG
Prefill 3.773 · TTFT 568 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
223Magistral-Small-250924BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
534,84 tok/s TG
Prefill 10.455 · TTFT 3.204 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
224Magistral-Small-250924BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
327,23 tok/s TG
Prefill 8.158 · TTFT 1.764 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
225Magistral-Small-250924BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
79,48 tok/s TG
Prefill 3.787 · TTFT 566 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
226North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
518,08 tok/s TG
Prefill 33.281 · TTFT 870 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
227North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
366,96 tok/s TG
Prefill 25.394 · TTFT 501 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
228North-Mini-Code-1.030.5BCohere PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
106,68 tok/s TG
Prefill 9.979 · TTFT 200 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
229Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
582,71 tok/s TG
Prefill 31.647 · TTFT 1.123 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
230Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
404,55 tok/s TG
Prefill 24.684 · TTFT 634 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
231Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
162,32 tok/s TG
Prefill 10.839 · TTFT 225 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
232Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
554,20 tok/s TG
Prefill 31.733 · TTFT 1.123 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
233Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
381,56 tok/s TG
Prefill 24.720 · TTFT 634 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
234Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
164,17 tok/s TG
Prefill 10.792 · TTFT 226 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
235Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
573,56 tok/s TG
Prefill 31.043 · TTFT 1.133 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
236Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
393,13 tok/s TG
Prefill 24.244 · TTFT 640 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
237Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
164,21 tok/s TG
Prefill 10.646 · TTFT 229 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
238Qwen3.8-27B27.8BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
319,01 tok/s TG
Prefill 7.048 · TTFT 4.342 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
239Qwen3.8-27B27.8BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
177,07 tok/s TG
Prefill 5.520 · TTFT 2.452 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
240Qwen3.8-27B27.8BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
42,53 tok/s TG
Prefill 2.454 · TTFT 804 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
241Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
457,45 tok/s TG
Prefill 5.564 · TTFT 8.488 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
242Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
221,70 tok/s TG
Prefill 5.542 · TTFT 1.796 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
243Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
111,16 tok/s TG
Prefill 4.336 · TTFT 457 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
244Qwen3.6-27B27BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
142,60 tok/s TG
Prefill 1.507 · TTFT 27.036 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
245Qwen3.6-27B27BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
61,94 tok/s TG
Prefill 1.488 · TTFT 6.487 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
246Qwen3.6-27B27BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
23,27 tok/s TG
Prefill 1.219 · TTFT 1.586 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
247Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
690,88 tok/s TG
Prefill 29.513 · TTFT 1.000 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
248Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
447,47 tok/s TG
Prefill 22.711 · TTFT 564 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
249Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
153,31 tok/s TG
Prefill 9.773 · TTFT 202 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
250Gemma-4-26B-A4B-it26BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
406,14 tok/s TG
Prefill 5.518 · TTFT 9.572 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0