Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
ll
Modellserver / Inferenz-Engine

llama.cpp

Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.

85Benchmarks
38Modelle
Beste Generation
147,8 tok/s
gpt-oss-20b
∅ Generation
47,5 tok/s
Schnitt aus 85 Läufen
Bester Prefill
5.195 tok/s
Prompt-Verarbeitung
Bester TTFT
445 ms
Time to First Token

Top-Modelle auf llama.cpp

Bester Generierungs-Durchsatz je Modell (tok/s).

gpt-oss-20b
147,8 tok/s
Nemotron-Cascade-2-30B-A3B
107,9 tok/s
Nemotron-3-Nano-Omni-30B-A3B-Reasoning
102,9 tok/s
gpt-oss-120b
99,8 tok/s
Qwen3-30B-A3B-Instruct-2507
97,6 tok/s
NVIDIA-Nemotron-3-Nano-4B
93,4 tok/s
Qwen3-30B-A3B-Thinking-2507
91,4 tok/s
Qwen3-VL-30B-A3B-Instruct
89,8 tok/s
Qwen3-Coder-30B-A3B-Instruct
88,2 tok/s
North-Mini-Code-1.0
87,3 tok/s

Auf welcher Hardware laeuft llama.cpp am schnellsten?

Bester Durchsatz je Grafikkarte mit diesem Modellserver.

AMD Radeon AI PRO R9700
147,8 tok/s
AMD Radeon Graphics
42,2 tok/s
Performanceprofil

Durchsatz mit llama.cpp

Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

CPUnach Prozessor

18413995,050,76,309441.8872.831Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 7955WX 16-Cores - 147,8 tok/s Generation, 2.290 tok/s Prefill, TTFT 1.516 ms (71 Laufe)AMD Ryzen Threadrippe...AMD RYZEN AI MAX+ 395 w/ Radeon 8060S - 42,2 tok/s Generation, 46 tok/s Prefill, TTFT 758 ms (2 Laufe)AMD RYZEN AI MAX+ 395...
AMD Ryzen Threadripper PRO 7955WX 16-Cores 147,8 tok/sAMD RYZEN AI MAX+ 395 w/ Radeon 8060S 42,2 tok/s

GPUnach Grafikkarte

18413995,050,76,309441.8872.831Prefill (tok/s)Generation (tok/s)AMD Radeon AI PRO R9700 - 147,8 tok/s Generation, 2.290 tok/s Prefill, TTFT 1.516 ms (71 Laufe)AMD Radeon AI PRO R97...AMD Radeon Graphics - 42,2 tok/s Generation, 46 tok/s Prefill, TTFT 758 ms (2 Laufe)AMD Radeon Graphics
AMD Radeon AI PRO R9700 147,8 tok/sAMD Radeon Graphics 42,2 tok/s

MODnach Modell

19014294,947,40,001.6753.3495.024Prefill (tok/s)Generation (tok/s)gpt-oss-20b - 147,8 tok/s Generation, 4.062 tok/s Prefill, TTFT 488 ms (3 Laufe)gpt-oss-20bNemotron-Cascade-2-30B-A3B - 107,9 tok/s Generation, 3.141 tok/s Prefill, TTFT 690 ms (1 Lauf)Nemotron-Cascade-2-30...Nemotron-3-Nano-Omni-30B-A3B-Reasoning - 102,9 tok/s Generation, 3.111 tok/s Prefill, TTFT 692 ms (1 Lauf)Nemotron-3-Nano-Omni-...gpt-oss-120b - 99,8 tok/s Generation, 1.920 tok/s Prefill, TTFT 1.022 ms (1 Lauf)gpt-oss-120bQwen3-30B-A3B-Instruct-2507 - 97,6 tok/s Generation, 3.918 tok/s Prefill, TTFT 645 ms (3 Laufe)Qwen3-30B-A3B-Instruc...NVIDIA-Nemotron-3-Nano-4B - 93,4 tok/s Generation, 2.630 tok/s Prefill, TTFT 822 ms (1 Lauf)NVIDIA-Nemotron-3-Nan...Qwen3-30B-A3B-Thinking-2507 - 91,4 tok/s Generation, 3.773 tok/s Prefill, TTFT 708 ms (3 Laufe)Qwen3-30B-A3B-Thinkin...Qwen3-VL-30B-A3B-Instruct - 89,8 tok/s Generation, 3.879 tok/s Prefill, TTFT 668 ms (3 Laufe)Qwen3-VL-30B-A3B-Inst...Qwen3-Coder-30B-A3B-Instruct - 88,2 tok/s Generation, 3.655 tok/s Prefill, TTFT 1.019 ms (3 Laufe)Qwen3-Coder-30B-A3B-I...North-Mini-Code-1.0 - 87,3 tok/s Generation, 3.234 tok/s Prefill, TTFT 648 ms (1 Lauf)North-Mini-Code-1.0Qwen3.6-35B-A3B - 79,4 tok/s Generation, 1.958 tok/s Prefill, TTFT 1.127 ms (3 Laufe)Qwen3.6-35B-A3BQwen-AgentWorld-35B-A3B - 78,5 tok/s Generation, 2.425 tok/s Prefill, TTFT 886 ms (3 Laufe)Qwen-AgentWorld-35B-A...Qwen3.5-35B-A3B - 78,5 tok/s Generation, 2.565 tok/s Prefill, TTFT 814 ms (3 Laufe)Qwen3.5-35B-A3BNVIDIA-Nemotron-3-Nano-30B-A3B - 78,3 tok/s Generation, 2.525 tok/s Prefill, TTFT 867 ms (1 Lauf)NVIDIA-Nemotron-3-Nan...Qwen3-Coder-Next - 62,4 tok/s Generation, 2.276 tok/s Prefill, TTFT 1.068 ms (2 Laufe)Qwen3-Coder-NextMistral-Small-4-119B-2603 - 58,5 tok/s Generation, 2.053 tok/s Prefill, TTFT 1.304 ms (1 Lauf)Mistral-Small-4-119B-...Ministral-3-14B-Reasoning-2512 - 51,0 tok/s Generation, 3.092 tok/s Prefill, TTFT 775 ms (3 Laufe)Ministral-3-14B-Reaso...Gemma-4-26B-A4B-it - 42,2 tok/s Generation, 71 tok/s Prefill, TTFT 609 ms (1 Lauf)Gemma-4-26B-A4B-itDevstral-Small-2507 - 33,9 tok/s Generation, 1.726 tok/s Prefill, TTFT 2.112 ms (6 Laufe)Devstral-Small-2507Mistral-Small-3.1-24B-Instruct-2503 - 33,5 tok/s Generation, 1.625 tok/s Prefill, TTFT 1.626 ms (6 Laufe)Mistral-Small-3.1-24B...Codestral-22B-v0.1 - 33,5 tok/s Generation, 1.489 tok/s Prefill, TTFT 2.492 ms (4 Laufe)Codestral-22B-v0.1Magistral-Small-2509 - 33,3 tok/s Generation, 1.607 tok/s Prefill, TTFT 1.568 ms (6 Laufe)Magistral-Small-2509GLM-4.5-Air - 32,9 tok/s Generation, 982 tok/s Prefill, TTFT 3.379 ms (1 Lauf)GLM-4.5-AirDevstral-Small-2-24B-Instruct-2512 - 29,4 tok/s Generation, 1.041 tok/s Prefill, TTFT 1.921 ms (4 Laufe)Devstral-Small-2-24B-...OpenReasoning-Nemotron-32B - 23,3 tok/s Generation, 1.196 tok/s Prefill, TTFT 2.115 ms (1 Lauf)OpenReasoning-Nemotro...Qwen2.5-Coder-32B-Instruct - 21,2 tok/s Generation, 1.059 tok/s Prefill, TTFT 3.266 ms (6 Laufe)Qwen2.5-Coder-32B-Ins...Seed-OSS-36B-Instruct - 19,1 tok/s Generation, 1.108 tok/s Prefill, TTFT 2.083 ms (1 Lauf)Seed-OSS-36B-InstructLlama-3_3-Nemotron-Super-49B-v1_5 - 12,3 tok/s Generation, 823 tok/s Prefill, TTFT 2.292 ms (1 Lauf)Llama-3_3-Nemotron-Su...
gpt-oss-20b 147,8 tok/sNemotron-Cascade-2-30B-A3B 107,9 tok/sNemotron-3-Nano-Omni-30B-A3B-Reasoning 102,9 tok/sgpt-oss-120b 99,8 tok/sQwen3-30B-A3B-Instruct-2507 97,6 tok/sNVIDIA-Nemotron-3-Nano-4B 93,4 tok/sQwen3-30B-A3B-Thinking-2507 91,4 tok/sQwen3-VL-30B-A3B-Instruct 89,8 tok/sQwen3-Coder-30B-A3B-Instruct 88,2 tok/sNorth-Mini-Code-1.0 87,3 tok/sQwen3.6-35B-A3B 79,4 tok/sQwen-AgentWorld-35B-A3B 78,5 tok/sQwen3.5-35B-A3B 78,5 tok/sNVIDIA-Nemotron-3-Nano-30B-A3B 78,3 tok/sQwen3-Coder-Next 62,4 tok/sMistral-Small-4-119B-2603 58,5 tok/sMinistral-3-14B-Reasoning-2512 51,0 tok/sGemma-4-26B-A4B-it 42,2 tok/sDevstral-Small-2507 33,9 tok/sMistral-Small-3.1-24B-Instruct-2503 33,5 tok/sCodestral-22B-v0.1 33,5 tok/sMagistral-Small-2509 33,3 tok/sGLM-4.5-Air 32,9 tok/sDevstral-Small-2-24B-Instruct-2512 29,4 tok/sOpenReasoning-Nemotron-32B 23,3 tok/sQwen2.5-Coder-32B-Instruct 21,2 tok/sSeed-OSS-36B-Instruct 19,1 tok/sLlama-3_3-Nemotron-Super-49B-v1_5 12,3 tok/s
Alle Messungen

80 Benchmark-Läufe

Jeder veroeffentlichte Lauf mit llama.cpp.

← Alle Engines
#Modell / HerstellerTypMesswerteGPU / CPU / RAMRuntimeDatum
1gpt-oss-20bOpenAITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
147,82 tok/s
Prefill 3.534 · TTFT 555 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
2gpt-oss-20bOpenAITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
143,90 tok/s
Prefill 4.229 · TTFT 464 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
3gpt-oss-20bOpenAITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
137,66 tok/s
Prefill 4.422 · TTFT 445 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
4Nemotron-Cascade-2-30B-A3BNVIDIATimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
107,87 tok/s
Prefill 3.141 · TTFT 690 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
5Nemotron-3-Nano-Omni-30B-A3B-ReasoningNVIDIATimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
102,89 tok/s
Prefill 3.111 · TTFT 692 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
6gpt-oss-120bOpenAITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
99,77 tok/s
Prefill 1.920 · TTFT 1.022 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
7Qwen3-30B-A3B-Instruct-2507Qwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
97,61 tok/s
Prefill 2.827 · TTFT 862 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
8NVIDIA-Nemotron-3-Nano-4BNVIDIATimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
93,42 tok/s
Prefill 2.630 · TTFT 822 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
9Qwen3-30B-A3B-Thinking-2507Qwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
91,39 tok/s
Prefill 4.034 · TTFT 603 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
10Qwen3-30B-A3B-Instruct-2507Qwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
90,98 tok/s
Prefill 4.283 · TTFT 569 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
11Qwen3-VL-30B-A3B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
89,84 tok/s
Prefill 4.031 · TTFT 605 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
12Qwen3-30B-A3B-Thinking-2507Qwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
88,58 tok/s
Prefill 4.831 · TTFT 503 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
13Qwen3-Coder-30B-A3B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
88,22 tok/s
Prefill 5.195 · TTFT 470 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
14Qwen3-30B-A3B-Thinking-2507Qwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
87,84 tok/s
Prefill 2.455 · TTFT 1.018 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
15North-Mini-Code-1.0CohereTimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
87,33 tok/s
Prefill 3.234 · TTFT 648 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
16Devstral-Small-2-24B-Instruct-2512Mistral AITool Usage Standard 1.0Harnessbenchmark
87,1 %
2.597/2.980 Pkt · 16/16 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_016.07.2026Details →
17Qwen3-VL-30B-A3B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
86,76 tok/s
Prefill 4.914 · TTFT 497 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
18Qwen3-VL-30B-A3B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
86,67 tok/s
Prefill 2.692 · TTFT 903 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
19Qwen3-Coder-30B-A3B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
82,49 tok/s
Prefill 3.953 · TTFT 587 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
20Qwen3.6-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
79,36 tok/s
Prefill 2.286 · TTFT 843 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
21Qwen3-30B-A3B-Instruct-2507Qwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
78,90 tok/s
Prefill 4.644 · TTFT 504 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
22Qwen-AgentWorld-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
78,54 tok/s
Prefill 2.337 · TTFT 875 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
23Qwen3.5-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
78,46 tok/s
Prefill 2.327 · TTFT 829 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
24NVIDIA-Nemotron-3-Nano-30B-A3BNVIDIATimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
78,25 tok/s
Prefill 2.525 · TTFT 867 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
25Gemma-4-26B-A4B-itGoogleTool Usage Standard 1.0Harnessbenchmark
76,1 %
2.268/2.980 Pkt · 16/16 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_016.07.2026Details →
26Devstral-Small-2-24B-Instruct-2512Mistral AIGodBrain MCP Deep 40 - Wissensbasis-Lebenszyklus 1.0Harnessbenchmark
75,7 %
5.738/7.580 Pkt · 40/40 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_017.07.2026Details →
27Qwen3-Coder-30B-A3B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
75,66 tok/s
Prefill 1.817 · TTFT 2.001 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
28Qwen-AgentWorld-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
75,30 tok/s
Prefill 2.873 · TTFT 669 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
29Qwen3.6-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
68,58 tok/s
Prefill 1.297 · TTFT 1.648 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
30Devstral-Small-2-24B-Instruct-2512Mistral AITool Usage Extrem 40 - Tech-Radar Mission 1.0Harnessbenchmark
68,6 %
5.629/8.210 Pkt · 40/40 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_017.07.2026Details →
31Qwen-AgentWorld-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
62,70 tok/s
Prefill 2.064 · TTFT 1.114 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
32Qwen3-Coder-NextQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
62,39 tok/s
Prefill 2.142 · TTFT 1.132 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
33Qwen3-Coder-NextQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
60,99 tok/s
Prefill 2.410 · TTFT 1.005 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
34Qwen3.5-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
60,88 tok/s
Prefill 2.087 · TTFT 996 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
35Qwen3.6-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
59,79 tok/s
Prefill 2.291 · TTFT 889 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
36Mistral-Small-4-119B-2603Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
58,52 tok/s
Prefill 2.053 · TTFT 1.304 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
37Ministral-3-14B-Reasoning-2512Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
51,00 tok/s
Prefill 3.205 · TTFT 716 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
38Ministral-3-14B-Reasoning-2512Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
50,36 tok/s
Prefill 3.889 · TTFT 593 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
39Ministral-3-14B-Reasoning-2512Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
47,59 tok/s
Prefill 2.183 · TTFT 1.016 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
40Qwen3.5-35B-A3BQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
42,63 tok/s
Prefill 3.281 · TTFT 617 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
41Gemma-4-26B-A4B-itGooglePerformancetest Small 1.0Performancebenchmark
42,22 tok/s
Prefill 71 · TTFT 609 ms
AMD Radeon Graphics32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_022.07.2026Details →
42Devstral-Small-2507Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
33,91 tok/s
Prefill 0 · TTFT 1.523 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
43Mistral-Small-3.1-24B-Instruct-2503Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
33,48 tok/s
Prefill 1.390 · TTFT 1.474 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
44Codestral-22B-v0.1Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
33,47 tok/s
Prefill 810 · TTFT 3.810 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
45Magistral-Small-2509Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
33,32 tok/s
Prefill 2.001 · TTFT 1.001 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
46Codestral-22B-v0.1Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
33,12 tok/s
Prefill 2.688 · TTFT 893 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
47Mistral-Small-3.1-24B-Instruct-2503Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
33,10 tok/s
Prefill 1.265 · TTFT 1.767 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
48Mistral-Small-3.1-24B-Instruct-2503Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
32,98 tok/s
Prefill 2.592 · TTFT 790 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
49GLM-4.5-AirZ.ai (Zhipu)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
32,88 tok/s
Prefill 982 · TTFT 3.379 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
50Devstral-Small-2507Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
32,07 tok/s
Prefill 2.888 · TTFT 1.129 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
51Devstral-Small-2507Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
31,41 tok/s
Prefill 1.961 · TTFT 1.394 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
52Mistral-Small-3.1-24B-Instruct-2503Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
31,27 tok/s
Prefill 1.563 · TTFT 1.699 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
53Devstral-Small-2507Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
31,20 tok/s
Prefill 0 · TTFT 3.155 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
54Devstral-Small-2507Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
30,97 tok/s
Prefill 1.167 · TTFT 1.890 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
55Mistral-Small-3.1-24B-Instruct-2503Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
29,97 tok/s
Prefill 1.536 · TTFT 2.083 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
56Gemma-4-26B-A4B-itGoogleGodBrain MCP Deep 40 - Wissensbasis-Lebenszyklus 1.0Harnessbenchmark
30,0 %
2.271/7.580 Pkt · 18/40 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_017.07.2026Details →
57Devstral-Small-2-24B-Instruct-2512Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
29,35 tok/s
Prefill 905 · TTFT 3.213 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
58Magistral-Small-2509Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
28,90 tok/s
Prefill 1.318 · TTFT 2.246 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
59Mistral-Small-3.1-24B-Instruct-2503Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
28,76 tok/s
Prefill 1.407 · TTFT 1.945 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
60Magistral-Small-2509Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
28,74 tok/s
Prefill 1.614 · TTFT 1.813 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
61Devstral-Small-2507Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
28,45 tok/s
Prefill 890 · TTFT 3.580 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
62Devstral-Small-2-24B-Instruct-2512Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
28,27 tok/s
Prefill 1.667 · TTFT 1.617 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
63Devstral-Small-2-24B-Instruct-2512Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
28,24 tok/s
Prefill 1.569 · TTFT 1.948 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
64Codestral-22B-v0.1Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
27,68 tok/s
Prefill 1.289 · TTFT 3.106 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
65Magistral-Small-2509Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
26,59 tok/s
Prefill 1.273 · TTFT 1.574 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
66Codestral-22B-v0.1Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
25,47 tok/s
Prefill 1.170 · TTFT 2.158 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
67Gemma-4-26B-A4B-itGoogleTool-Parcours Dossier 40 V1.0Harnessbenchmark
25,0 %
1.489/5.960 Pkt · 22/40 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_019.07.2026Details →
68Devstral-Small-2-24B-Instruct-2512Mistral AITool-Parcours Dossier 40 V1.0Harnessbenchmark
24,1 %
1.436/5.960 Pkt · 21/40 Aufg.
AMD GPU32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_019.07.2026Details →
69OpenReasoning-Nemotron-32BNVIDIATimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
23,25 tok/s
Prefill 1.196 · TTFT 2.115 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
70Qwen2.5-Coder-32B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
21,20 tok/s
Prefill 1.548 · TTFT 1.551 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
71Qwen2.5-Coder-32B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
20,90 tok/s
Prefill 572 · TTFT 7.773 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
72Magistral-Small-2509Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
19,42 tok/s
Prefill 2.175 · TTFT 1.030 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
73Qwen2.5-Coder-32B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
19,14 tok/s
Prefill 1.274 · TTFT 1.705 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
74Qwen2.5-Coder-32B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
19,08 tok/s
Prefill 1.508 · TTFT 1.441 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
75Seed-OSS-36B-InstructByteDance SeedTimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
19,07 tok/s
Prefill 1.108 · TTFT 2.083 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
76Qwen2.5-Coder-32B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
18,95 tok/s
Prefill 835 · TTFT 2.604 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
77Qwen2.5-Coder-32B-InstructQwen (Alibaba)Timebench 3 - Kombi (Prefill + Generation)Performancebenchmark
18,60 tok/s
Prefill 615 · TTFT 4.520 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw21.07.2026Details →
78Magistral-Small-2509Mistral AITimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
18,46 tok/s
Prefill 1.263 · TTFT 1.741 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M21.07.2026Details →
79Llama-3_3-Nemotron-Super-49B-v1_5NVIDIATimebench 3 - Kombi (Prefill + Generation)Performancebenchmark
12,27 tok/s
Prefill 823 · TTFT 2.292 ms
3x AMD Radeon AI PRO R970032x AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclaw Q4_K_M22.07.2026Details →
80Devstral-Small-2-24B-Instruct-2512Mistral AIPerformancetest Small 1.0Performancebenchmark
8,88 tok/s
Prefill 22 · TTFT 906 ms
AMD Radeon Graphics32x AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMllama.cppgodclaw Q8_020.07.2026Details →