ll
Modellserver / Inferenz-Engine
llama.cpp
Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.
Best generation
2.491,2 tok/s
gemma-4-E2B-it
∅ Generation
245,2 tok/s
Schnitt aus 3774 Läufen
Best prefill
41.362 tok/s
Prompt processing
Best TTFT
20 ms
Time to First Token
Top-Modelle auf llama.cpp
Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.
Auf welcher Hardware laeuft llama.cpp am schnellsten?
Bester Durchsatz je Grafikkarte mit diesem Modellserver.
Performance profile
Durchsatz mit llama.cpp
Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.
CPUnach Prozessor
GPUnach Grafikkarte
MODnach Modell
gemma-4-E2B-it 2.491,2 tok/sNVIDIA-Nemotron-3-Nano-4B 2.182,7 tok/sgpt-oss-20b 1.995,4 tok/sOrnith-1.0-35B 1.799,4 tok/sNemotron-3-Nano-Omni-30B-A3B-Reasoning 1.791,5 tok/sNemotron-Cascade-2-30B-A3B 1.784,7 tok/sNVIDIA-Nemotron-3-Nano-30B-A3B 1.775,6 tok/sLaguna-XS-2.1 1.765,0 tok/sgemma-4-E4B-it 1.758,6 tok/sQwen3-Coder-30B-A3B-Instruct 1.716,6 tok/sQwen3-30B-A3B-Thinking-2507 1.683,8 tok/sQwen3-Omni-30B-A3B-Thinking 1.673,6 tok/sQwen3-30B-A3B-Instruct-2507 1.671,0 tok/sQwen3-VL-30B-A3B-Instruct 1.655,9 tok/sNorth-Mini-Code-1.0 1.621,0 tok/sGemma-4-26B-A4B-it 1.604,6 tok/sMeta-Llama-3.1-8B-Instruct 1.593,1 tok/sOrnith-1.0-9B 1.560,5 tok/s
All measurements
← Alle Engines80 Benchmark-Läufe
Jeder veroeffentlichte Lauf mit llama.cpp.
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.491,20 tok/s TG Prefill 15.814 · TTFT 4.382 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 2 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.414,37 tok/s TG Prefill 20.277 · TTFT 4.107 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 3 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.412,27 tok/s TG Prefill 19.677 · TTFT 4.107 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 4 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.182,67 tok/s TG Prefill 10.599 · TTFT 5.598 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 5 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.144,30 tok/s TG Prefill 10.533 · TTFT 5.598 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 6 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.143,55 tok/s TG Prefill 13.632 · TTFT 4.819 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 7 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.995,37 tok/s TG Prefill 16.158 · TTFT 5.309 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 8 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.993,51 tok/s TG Prefill 16.318 · TTFT 5.323 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 9 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.981,08 tok/s TG Prefill 15.983 · TTFT 5.309 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 10 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.937,67 tok/s TG Prefill 9.452 · TTFT 6.288 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 11 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.911,59 tok/s TG Prefill 10.116 · TTFT 6.245 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 12 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.896,98 tok/s TG Prefill 10.984 · TTFT 6.227 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 13 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.890,69 tok/s TG Prefill 14.771 · TTFT 5.489 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 14 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.878,73 tok/s TG Prefill 7.717 · TTFT 6.690 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 15 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.835,73 tok/s TG Prefill 15.972 · TTFT 5.760 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 16 | NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.830,31 tok/s TG Prefill 9.399 · TTFT 6.598 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 17 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.819,53 tok/s TG Prefill 19.732 · TTFT 5.578 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 18 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.801,97 tok/s TG Prefill 18.226 · TTFT 5.605 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 19 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.799,41 tok/s TG Prefill 9.192 · TTFT 8.876 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 20 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.791,48 tok/s TG Prefill 7.972 · TTFT 6.996 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 21 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.784,65 tok/s TG Prefill 8.304 · TTFT 6.952 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 22 | NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.775,62 tok/s TG Prefill 8.642 · TTFT 6.861 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 23 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.774,36 tok/s TG Prefill 7.850 · TTFT 6.996 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 24 | NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.773,53 tok/s TG Prefill 8.720 · TTFT 6.861 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 25 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.768,14 tok/s TG Prefill 8.360 · TTFT 6.952 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 26 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.764,99 tok/s TG Prefill 8.542 · TTFT 7.776 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 27 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.758,55 tok/s TG Prefill 13.889 · TTFT 5.964 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 28 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.757,37 tok/s TG Prefill 9.025 · TTFT 7.343 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 29 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.755,52 tok/s TG Prefill 13.910 · TTFT 5.964 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 30 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.753,82 tok/s TG Prefill 9.267 · TTFT 7.329 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawINT4 | Details → | |
| 31 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.743,96 tok/s TG Prefill 13.641 · TTFT 5.790 ms | 10× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 32 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.743,14 tok/s TG Prefill 12.148 · TTFT 5.881 ms | 10× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 33 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.742,13 tok/s TG Prefill 12.852 · TTFT 6.156 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 34 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.740,43 tok/s TG Prefill 9.653 · TTFT 7.258 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 35 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.740,12 tok/s TG Prefill 8.571 · TTFT 7.786 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliINT4 | Details → | |
| 36 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.738,38 tok/s TG Prefill 8.872 · TTFT 7.722 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 37 | NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.722,05 tok/s TG Prefill 7.387 · TTFT 7.500 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 38 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.719,17 tok/s TG Prefill 8.044 · TTFT 7.376 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 39 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.716,59 tok/s TG Prefill 12.604 · TTFT 6.861 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 40 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.706,69 tok/s TG Prefill 7.511 · TTFT 7.506 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 41 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.706,21 tok/s TG Prefill 8.777 · TTFT 7.343 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 42 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.689,95 tok/s TG Prefill 14.458 · TTFT 6.609 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 43 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.683,83 tok/s TG Prefill 10.169 · TTFT 7.361 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 44 | Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.673,59 tok/s TG Prefill 9.186 · TTFT 7.567 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 45 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.671,02 tok/s TG Prefill 11.920 · TTFT 7.068 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 46 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.670,49 tok/s TG Prefill 10.926 · TTFT 7.140 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 47 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.658,09 tok/s TG Prefill 13.451 · TTFT 6.808 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 48 | Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.655,92 tok/s TG Prefill 10.520 · TTFT 7.269 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 49 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.655,21 tok/s TG Prefill 10.591 · TTFT 7.004 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 50 | NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.654,57 tok/s TG Prefill 10.472 · TTFT 7.000 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 51 | Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.653,28 tok/s TG Prefill 10.581 · TTFT 7.259 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 52 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.653,11 tok/s TG Prefill 13.572 · TTFT 6.808 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 53 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.650,98 tok/s TG Prefill 11.364 · TTFT 6.872 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 54 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.650,72 tok/s TG Prefill 11.398 · TTFT 6.887 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 55 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.649,85 tok/s TG Prefill 10.323 · TTFT 7.066 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 56 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.645,78 tok/s TG Prefill 14.123 · TTFT 6.609 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 57 | Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.639,00 tok/s TG Prefill 13.790 · TTFT 6.893 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 58 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.629,91 tok/s TG Prefill 11.384 · TTFT 7.322 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 59 | Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.623,26 tok/s TG Prefill 10.509 · TTFT 7.259 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 60 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.623,15 tok/s TG Prefill 10.707 · TTFT 7.140 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 61 | North-Mini-Code-1.030.5BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.621,00 tok/s TG Prefill 8.046 · TTFT 7.551 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawUD-Q4_K_M | Details → | |
| 62 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.620,45 tok/s TG Prefill 10.729 · TTFT 7.428 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 63 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.620,41 tok/s TG Prefill 12.519 · TTFT 7.197 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawINT4 | Details → | |
| 64 | Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.617,88 tok/s TG Prefill 13.894 · TTFT 6.893 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 65 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.615,26 tok/s TG Prefill 13.041 · TTFT 7.166 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 66 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.611,02 tok/s TG Prefill 12.725 · TTFT 7.223 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 67 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.609,12 tok/s TG Prefill 10.883 · TTFT 7.452 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 68 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.608,39 tok/s TG Prefill 11.312 · TTFT 7.410 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 69 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.607,66 tok/s TG Prefill 10.675 · TTFT 7.505 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawINT4 | Details → | |
| 70 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.607,48 tok/s TG Prefill 7.448 · TTFT 7.492 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 71 | Laguna-XS-2.133.4BPoolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.606,11 tok/s TG Prefill 12.690 · TTFT 7.203 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawINT4 | Details → | |
| 72 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.606,09 tok/s TG Prefill 12.150 · TTFT 6.323 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 73 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.604,55 tok/s TG Prefill 6.804 · TTFT 8.073 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawUD-Q4_K_XL | Details → | |
| 74 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.593,10 tok/s TG Prefill 16.376 · TTFT 6.849 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 75 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.589,11 tok/s TG Prefill 6.805 · TTFT 8.073 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawUD-Q4_K_XL | Details → | |
| 76 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.586,19 tok/s TG Prefill 7.391 · TTFT 7.492 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 77 | North-Mini-Code-1.030.5BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.573,04 tok/s TG Prefill 8.004 · TTFT 7.551 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | llama.cppgodclawUD-Q4_K_M | Details → | |
| 78 | Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.560,53 tok/s TG Prefill 10.558 · TTFT 6.806 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 79 | North-Mini-Code-1.030.5BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.557,27 tok/s TG Prefill 6.839 · TTFT 8.511 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliUD-Q4_K_M | Details → | |
| 80 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.556,71 tok/s TG Prefill 6.275 · TTFT 8.228 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → |

