⚡
Performance-Benchmark
Wie schnell ist das Modell?
Roher Durchsatz auf echter Hardware – wie viele Token ein Modell pro Sekunde erzeugt, wie schnell es den Prompt verarbeitet und wie kurz die Zeit bis zum ersten Token ausfällt. Je höher, desto besser.
⚡ Generation (tok/s)🔄 Prefill⏱️ TTFT👥 Concurrency
| # | Modell / Hersteller | Messwerte | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 133,73 tok/s TG Prefill 1.652 · TTFT 12.206 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 2 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 130,88 tok/s TG Prefill 4.122 · TTFT 7.946 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 3 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 125,68 tok/s TG Prefill 2.208 · TTFT 11.151 ms | 10× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 4 | gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 125,06 tok/s TG Prefill 1.687 · TTFT 12.063 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 5 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 116,24 tok/s TG Prefill 1.891 · TTFT 5.975 ms | 5× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 6 | Ornith-1.0-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 108,03 tok/s TG Prefill 2.184 · TTFT 10.407 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 7 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 106,92 tok/s TG Prefill 1.319 · TTFT 18.921 ms | 10× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 8 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 106,11 tok/s TG Prefill 1.440 · TTFT 17.007 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 9 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 100,31 tok/s TG Prefill 1.092 · TTFT 10.399 ms | 5× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 10 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 100,04 tok/s TG Prefill 1.231 · TTFT 9.195 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 11 | Ornith-1.0-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 97,31 tok/s TG Prefill 2.232 · TTFT 10.125 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 12 | gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 95,22 tok/s TG Prefill 1.480 · TTFT 6.689 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 13 | gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 93,25 tok/s TG Prefill 1.437 · TTFT 6.874 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 14 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 90,37 tok/s TG Prefill 3.202 · TTFT 4.555 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 15 | gemma-4-12B-it12BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 89,80 tok/s TG Prefill 877 · TTFT 23.528 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 16 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 89,46 tok/s TG Prefill 3.210 · TTFT 4.511 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 17 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 80,94 tok/s TG Prefill 1.564 · TTFT 1.376 ms | 1× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 18 | Ornith-1.0-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 80,74 tok/s TG Prefill 1.579 · TTFT 6.372 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 19 | Ornith-1.0-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 73,89 tok/s TG Prefill 1.619 · TTFT 6.216 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 20 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 72,37 tok/s TG Prefill 865 · TTFT 2.489 ms | 1× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 21 | Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 71,70 tok/s TG Prefill 945 · TTFT 2.277 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 22 | gemma-4-12B-it12BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 60,61 tok/s TG Prefill 776 · TTFT 12.750 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 23 | Ministral-3-14B-Reasoning-251214BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 60,39 tok/s TG Prefill 1.497 · TTFT 8.155 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 24 | gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 53,35 tok/s TG Prefill 1.366 · TTFT 1.447 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 25 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 48,95 tok/s TG Prefill 1.278 · TTFT 1.932 ms | 1× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 26 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 47,41 tok/s TG Prefill 1.800 · TTFT 1.368 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 27 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 45,73 tok/s TG Prefill 1.206 · TTFT 2.050 ms | 1× | 2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 28 | Ornith-1.0-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 42,36 tok/s TG Prefill 986 · TTFT 1.973 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 29 | gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 38,32 tok/s TG Prefill 1.088 · TTFT 1.818 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 30 | Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 31,83 tok/s TG Prefill 1.653 · TTFT 1.506 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 31 | gemma-4-12B-it12BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 30,34 tok/s TG Prefill 684 · TTFT 2.896 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 32 | Ornith-1.0-9B9BDeepReinforce PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 29,62 tok/s TG Prefill 1.037 · TTFT 1.873 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 33 | Ministral-3-14B-Reasoning-251214BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 29,08 tok/s TG Prefill 1.083 · TTFT 2.088 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 34 | gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 15,30 tok/s TG Prefill 1.816 · TTFT 1.088 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 35 | gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 14,49 tok/s TG Prefill 1.763 · TTFT 1.123 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 36 | gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 12,45 tok/s TG Prefill 1.249 · TTFT 8.044 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 37 | gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 12,43 tok/s TG Prefill 1.220 · TTFT 8.133 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 38 | gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 9,73 tok/s TG Prefill 1.306 · TTFT 16.890 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 39 | gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation) | 6,84 tok/s TG Prefill 1.286 · TTFT 17.461 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → |