Created bymario-alka.dePowered bynoob2claw.detricoma.de
🏆
Rankings

LLM Leaderboard

Real measurements of every tested language model on real, named hardware – rated by raw speed (performance in tokens per second, prefill and time to first token) and by practical task quality in complete agent and chat runs (harness). Pick a benchmark type below or filter by model, maker and hardware to see exactly what is tested and how the results are produced.

⚡ Performance (tok/s)🤖 Harness quality👥 Concurrency🖥️ real hardware
Reset
Metric:
Size
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,15 tok/s TG
Prefill 1.978 · TTFT 1.328 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
2NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
56,16 tok/s TG
Prefill 1.002 · TTFT 2.226 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
3NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
35,84 tok/s TG
Prefill 769 · TTFT 2.861 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
4NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
42,29 tok/s TG
Prefill 1.495 · TTFT 1.438 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4
5Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
86,25 tok/s TG
Prefill 8.339 · TTFT 298 ms
1×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw
6NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
847,47 tok/s TG
Prefill 20.262 · TTFT 1.586 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMgodclaw
7NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
689,92 tok/s TG
Prefill 15.701 · TTFT 893 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMgodclaw
8NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
244,31 tok/s TG
Prefill 6.632 · TTFT 325 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMgodclaw
9NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
859,79 tok/s TG
Prefill 25.664 · TTFT 1.257 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMgodclaw
10NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
692,70 tok/s TG
Prefill 19.510 · TTFT 712 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMgodclaw
11NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
239,27 tok/s TG
Prefill 8.237 · TTFT 261 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMgodclaw
12NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
883,69 tok/s TG
Prefill 26.055 · TTFT 1.240 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
13NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
708,32 tok/s TG
Prefill 19.583 · TTFT 709 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
14NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
239,60 tok/s TG
Prefill 8.173 · TTFT 263 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
15NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
837,10 tok/s TG
Prefill 20.352 · TTFT 1.585 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
16NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
695,91 tok/s TG
Prefill 15.678 · TTFT 892 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
17NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
226,44 tok/s TG
Prefill 6.517 · TTFT 330 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
18NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
725,25 tok/s TG
Prefill 16.303 · TTFT 1.813 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
19NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
466,72 tok/s TG
Prefill 12.079 · TTFT 1.048 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
20NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
210,66 tok/s TG
Prefill 6.076 · TTFT 355 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
21NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.867,15 tok/s TG
Prefill 8.107 · TTFT 17.054 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
22NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.375,08 tok/s TG
Prefill 7.576 · TTFT 7.731 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
23NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
175,99 tok/s TG
Prefill 6.612 · TTFT 326 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
24NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
709,02 tok/s TG
Prefill 16.443 · TTFT 1.805 ms
10×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
25NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
462,42 tok/s TG
Prefill 11.388 · TTFT 1.082 ms
5×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
26NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
211,38 tok/s TG
Prefill 5.434 · TTFT 411 ms
1×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
27NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
3.484,32 tok/s TG
Prefill 8.256 · TTFT 14.388 ms
10×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
28NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.694,65 tok/s TG
Prefill 7.768 · TTFT 6.482 ms
5×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
29NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
235,73 tok/s TG
Prefill 6.703 · TTFT 322 ms
1×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
30NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkKurzaufgaben TTFT - Thinking OFF - Outputlimit 1/5/10 V1
218,77 tok/s TG
Prefill - · TTFT 54 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMgodclaw 0.27.1NVFP4 W4A16
31NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkKurzaufgaben TTFT - Thinking OFF - Outputlimit 1/5/10 V1
471,33 tok/s TG
Prefill - · TTFT 31 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
32NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
146,86 tok/s TG
Prefill 2.806 · TTFT 12.191 ms
10×NVIDIA RTX SPARK (N1X)NVIDIA RTX Spark N1X (6144-core GPU, 20-core CPU) · 128 GB RAMllama.cppopenclaw_cliQ4_0
33NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
121,80 tok/s TG
Prefill 1.943 · TTFT 7.408 ms
5×NVIDIA RTX SPARK (N1X)NVIDIA RTX Spark N1X (6144-core GPU, 20-core CPU) · 128 GB RAMllama.cppopenclaw_cliQ4_0
34NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
66,20 tok/s TG
Prefill 1.300 · TTFT 1.660 ms
1×NVIDIA RTX SPARK (N1X)NVIDIA RTX Spark N1X (6144-core GPU, 20-core CPU) · 128 GB RAMllama.cppopenclaw_cliQ4_0
35NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
136,58 tok/s TG
Prefill 2.934 · TTFT 8.734 ms
10×NVIDIA DGX SPARK (GB10) iNVIDIA Grace · 120 GB RAMllama.cppgodclawQ4_K_M
36NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
116,18 tok/s TG
Prefill 2.075 · TTFT 5.320 ms
5×NVIDIA DGX SPARK (GB10) iNVIDIA Grace · 120 GB RAMllama.cppgodclawQ4_K_M
37NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
81,10 tok/s TG
Prefill 1.729 · TTFT 1.247 ms
1×NVIDIA DGX SPARK (GB10) iNVIDIA Grace · 120 GB RAMllama.cppgodclawQ4_K_M
38NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
72,08 tok/s TG
Prefill 1.293 · TTFT 1.669 ms
1×NVIDIA RTX SPARK (N1X)NVIDIA RTX Spark N1X (6144-core GPU, 20-core CPU) · 128 GB RAMllama.cppopenclaw_cliQ4_0
39NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
139,97 tok/s TG
Prefill 1.744 · TTFT 14.686 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
40NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
119,24 tok/s TG
Prefill 1.181 · TTFT 9.551 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
41NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
101,04 tok/s TG
Prefill 1.039 · TTFT 2.074 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
42Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
985,75 tok/s TG
Prefill 7.733 · TTFT 2.448 ms
5×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
43Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
997,65 tok/s TG
Prefill 7.725 · TTFT 2.444 ms
5×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
44NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.005,67 tok/s TG
Prefill 7.555 · TTFT 2.444 ms
5×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
45NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.154,42 tok/s TG
Prefill 8.806 · TTFT 2.065 ms
5×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
46Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.784,65 tok/s TG
Prefill 8.304 · TTFT 6.952 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
47NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.775,62 tok/s TG
Prefill 8.642 · TTFT 6.861 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
48Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.774,36 tok/s TG
Prefill 7.850 · TTFT 6.996 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
49NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.144,30 tok/s TG
Prefill 10.533 · TTFT 5.598 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMllama.cppgodclawQ4_K_M
50NVIDIA-Nemotron-3.5-Super-VL-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,84 tok/s TG
Prefill 3.760 · TTFT 5.878 ms
10×NVIDIA DGX SPARK (GB10) iNVIDIA Grace · 120 GB RAMvLLMgodclawNVFP4
51NVIDIA-Nemotron-3.5-Super-VL-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
54,14 tok/s TG
Prefill 3.027 · TTFT 3.561 ms
5×NVIDIA DGX SPARK (GB10) iNVIDIA Grace · 120 GB RAMvLLMgodclawNVFP4
52NVIDIA-Nemotron-3.5-Super-VL-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,44 tok/s TG
Prefill 1.717 · TTFT 1.294 ms
1×NVIDIA DGX SPARK (GB10) iNVIDIA Grace · 120 GB RAMvLLMgodclawNVFP4
53NVIDIA-Nemotron-3.5-Super-VL-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.562,27 tok/s TG
Prefill 2.593 · TTFT 34.854 ms
10×NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition iAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAMvLLMgodclawNVFP4
54NVIDIA-Nemotron-3.5-Super-VL-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
795,03 tok/s TG
Prefill 2.378 · TTFT 15.951 ms
5×NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition iAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAMvLLMgodclawNVFP4
55NVIDIA-Nemotron-3.5-Super-VL-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
158,32 tok/s TG
Prefill 1.225 · TTFT 2.121 ms
1×NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition iAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAMvLLMgodclawNVFP4
56NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
4,01 tok/s TG
Prefill 93 · TTFT 268.279 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
57NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,03 tok/s TG
Prefill 58 · TTFT 186.063 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
58NVIDIA-Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
6,11 tok/s TG
Prefill 50 · TTFT 45.015 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
59Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,28 tok/s TG
Prefill 1.036 · TTFT 25.801 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
60Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
66,29 tok/s TG
Prefill 806 · TTFT 15.256 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
61Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
26,15 tok/s TG
Prefill 752 · TTFT 3.251 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
62Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
65,50 tok/s TG
Prefill 90 · TTFT 241.475 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
63Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
46,97 tok/s TG
Prefill 120 · TTFT 105.288 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
64Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
24,61 tok/s TG
Prefill 266 · TTFT 8.049 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
65NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
244,91 tok/s TG
Prefill 3.719 · TTFT 6.905 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
66NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
185,17 tok/s TG
Prefill 2.486 · TTFT 4.503 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
67NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
82,38 tok/s TG
Prefill 2.297 · TTFT 936 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
68OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
124,63 tok/s TG
Prefill 2.910 · TTFT 13.342 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
69OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
77,55 tok/s TG
Prefill 2.327 · TTFT 6.323 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
70OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,72 tok/s TG
Prefill 1.260 · TTFT 1.939 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
71Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
76,68 tok/s TG
Prefill 2.234 · TTFT 966 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
72Nemotron-Cascade-2-30B-A3B30BNVIDIA Harness benchmarkDecision-Benchmark V 1.0
159 Pkt
79,5% · 159/200 Aufg. · 35,0 s
1×NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition iAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAMvLLMvllmAWQ
73Nemotron-Cascade-2-30B-A3B30BNVIDIA Harness benchmarkDecision-Benchmark V 1.0
159 Pkt
79,5% · 159/200 Aufg. · 21,0 s
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMvLLMvllmAWQ
74Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
418,31 tok/s TG
Prefill 11.855 · TTFT 2.651 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0
75Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
294,88 tok/s TG
Prefill 9.401 · TTFT 1.516 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0
76Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,40 tok/s TG
Prefill 5.361 · TTFT 457 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0
77Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
419,38 tok/s TG
Prefill 11.768 · TTFT 2.673 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
78Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,18 tok/s TG
Prefill 9.348 · TTFT 1.524 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
79Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
139,33 tok/s TG
Prefill 4.729 · TTFT 517 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
80NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
283,50 tok/s TG
Prefill 4.190 · TTFT 6.068 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
81NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
213,81 tok/s TG
Prefill 3.032 · TTFT 3.678 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
82NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
112,45 tok/s TG
Prefill 2.607 · TTFT 827 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
83NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
294,37 tok/s TG
Prefill 4.193 · TTFT 6.057 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
84NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
215,49 tok/s TG
Prefill 2.912 · TTFT 3.858 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
85NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,73 tok/s TG
Prefill 2.564 · TTFT 841 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
86Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
295,16 tok/s TG
Prefill 4.265 · TTFT 6.027 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
87Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
216,61 tok/s TG
Prefill 2.963 · TTFT 3.827 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
88Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
112,56 tok/s TG
Prefill 2.551 · TTFT 851 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
89OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
116,53 tok/s TG
Prefill 1.848 · TTFT 16.560 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
90OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
54,60 tok/s TG
Prefill 1.544 · TTFT 9.340 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
91OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
23,60 tok/s TG
Prefill 916 · TTFT 2.675 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_M
92NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
344,85 tok/s TG
Prefill 4.759 · TTFT 5.325 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
93NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
247,85 tok/s TG
Prefill 2.967 · TTFT 3.756 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
94NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
100,85 tok/s TG
Prefill 2.620 · TTFT 821 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
95NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
3,22 tok/s TG
Prefill 54 · TTFT 382.793 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
96Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,14 tok/s TG
Prefill 21 · TTFT 117.641 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
97Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,12 tok/s TG
Prefill 16 · TTFT 141.049 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
98NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
15,11 tok/s TG
Prefill 29 · TTFT 379.775 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
99NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
13,88 tok/s TG
Prefill 24 · TTFT 89.272 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
100OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,44 tok/s TG
Prefill 39 · TTFT 324.717 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
101OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,57 tok/s TG
Prefill 51 · TTFT 47.947 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
102Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
153,61 tok/s TG
Prefill 1.063 · TTFT 74.931 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
103Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,80 tok/s TG
Prefill 936 · TTFT 25.459 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
104Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,83 tok/s TG
Prefill 830 · TTFT 2.957 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
105Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
88,14 tok/s TG
Prefill 506 · TTFT 123.878 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
106Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
51,74 tok/s TG
Prefill 518 · TTFT 40.780 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
107Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,58 tok/s TG
Prefill 531 · TTFT 4.053 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
108NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
755,10 tok/s TG
Prefill 6.964 · TTFT 14.586 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
109NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
448,68 tok/s TG
Prefill 6.418 · TTFT 4.484 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
110NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
174,14 tok/s TG
Prefill 4.417 · TTFT 488 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
111Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
723,68 tok/s TG
Prefill 7.027 · TTFT 14.700 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
112Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
435,36 tok/s TG
Prefill 6.438 · TTFT 4.605 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
113Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
171,10 tok/s TG
Prefill 4.389 · TTFT 495 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
114OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
138,38 tok/s TG
Prefill 2.698 · TTFT 27.790 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
115OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
77,23 tok/s TG
Prefill 1.954 · TTFT 6.520 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
116OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,24 tok/s TG
Prefill 1.232 · TTFT 1.990 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
117Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
197,25 tok/s TG
Prefill 5.040 · TTFT 7.497 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
118Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
111,56 tok/s TG
Prefill 3.912 · TTFT 4.256 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
119Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
25,25 tok/s TG
Prefill 1.890 · TTFT 1.299 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
120Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
38,08 tok/s TG
Prefill 288 · TTFT 117.541 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
121Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
23,08 tok/s TG
Prefill 299 · TTFT 39.857 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
122Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
13,25 tok/s TG
Prefill 547 · TTFT 3.937 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
123Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
464,35 tok/s TG
Prefill 6.208 · TTFT 9.108 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
124Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
246,65 tok/s TG
Prefill 5.984 · TTFT 1.807 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
125Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
159,92 tok/s TG
Prefill 4.351 · TTFT 495 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
126OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
143,98 tok/s TG
Prefill 2.767 · TTFT 24.248 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
127OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
45,16 tok/s TG
Prefill 2.426 · TTFT 5.640 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
128OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,26 tok/s TG
Prefill 1.308 · TTFT 1.876 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
129NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
403,66 tok/s TG
Prefill 6.688 · TTFT 9.919 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
130NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
216,68 tok/s TG
Prefill 6.212 · TTFT 1.740 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
131NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,08 tok/s TG
Prefill 4.478 · TTFT 481 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
132Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
391,02 tok/s TG
Prefill 6.784 · TTFT 9.603 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
133Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
216,78 tok/s TG
Prefill 6.257 · TTFT 1.743 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
134Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,15 tok/s TG
Prefill 4.443 · TTFT 489 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
135Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
406,79 tok/s TG
Prefill 6.308 · TTFT 9.948 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
136Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
217,44 tok/s TG
Prefill 6.099 · TTFT 1.773 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
137Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,19 tok/s TG
Prefill 4.389 · TTFT 491 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
138Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
254,90 tok/s TG
Prefill 18.787 · TTFT 1.624 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
139Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
160,14 tok/s TG
Prefill 13.947 · TTFT 961 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
140Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,92 tok/s TG
Prefill 3.629 · TTFT 955 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
141NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.485,14 tok/s TG
Prefill 44.171 · TTFT 717 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
142NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
899,41 tok/s TG
Prefill 30.214 · TTFT 473 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
143NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
217,76 tok/s TG
Prefill 13.576 · TTFT 159 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
144Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
87,25 tok/s TG
Prefill 547 · TTFT 119.494 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
145Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
50,46 tok/s TG
Prefill 539 · TTFT 37.888 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
146Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,60 tok/s TG
Prefill 535 · TTFT 4.022 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
147Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
723,16 tok/s TG
Prefill 7.058 · TTFT 14.771 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
148Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
436,89 tok/s TG
Prefill 6.484 · TTFT 4.583 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
149Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
170,84 tok/s TG
Prefill 4.390 · TTFT 495 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
150OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
140,00 tok/s TG
Prefill 2.638 · TTFT 27.354 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
151OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
77,33 tok/s TG
Prefill 2.366 · TTFT 6.171 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
152OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,27 tok/s TG
Prefill 1.242 · TTFT 1.972 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
153Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
38,05 tok/s TG
Prefill 280 · TTFT 122.770 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
154Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
23,92 tok/s TG
Prefill 304 · TTFT 38.888 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
155Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
13,25 tok/s TG
Prefill 548 · TTFT 3.936 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
156Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
459,46 tok/s TG
Prefill 6.270 · TTFT 9.164 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
157Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
245,18 tok/s TG
Prefill 6.028 · TTFT 1.795 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
158Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
160,16 tok/s TG
Prefill 4.303 · TTFT 501 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
159OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
137,80 tok/s TG
Prefill 3.136 · TTFT 23.970 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
160OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
56,97 tok/s TG
Prefill 2.087 · TTFT 6.098 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
161OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,26 tok/s TG
Prefill 1.316 · TTFT 1.864 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
162Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
393,14 tok/s TG
Prefill 6.810 · TTFT 9.694 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
163Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
217,05 tok/s TG
Prefill 6.295 · TTFT 1.732 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
164Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,40 tok/s TG
Prefill 4.443 · TTFT 489 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
165Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
405,35 tok/s TG
Prefill 6.324 · TTFT 9.987 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
166Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
218,30 tok/s TG
Prefill 6.103 · TTFT 1.772 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
167Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,51 tok/s TG
Prefill 4.445 · TTFT 485 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
168Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
154,03 tok/s TG
Prefill 1.071 · TTFT 74.122 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
169Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
79,37 tok/s TG
Prefill 1.020 · TTFT 24.303 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
170Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,84 tok/s TG
Prefill 838 · TTFT 2.930 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
171NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
756,69 tok/s TG
Prefill 7.030 · TTFT 14.551 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
172NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
450,85 tok/s TG
Prefill 6.453 · TTFT 4.464 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
173NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
174,04 tok/s TG
Prefill 4.376 · TTFT 492 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
174Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
183,48 tok/s TG
Prefill 4.576 · TTFT 8.283 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
175Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
106,74 tok/s TG
Prefill 3.548 · TTFT 4.684 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
176Llama-3_3-Nemotron-Super-49B-v1_549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
24,54 tok/s TG
Prefill 1.688 · TTFT 1.452 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
177NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
403,64 tok/s TG
Prefill 6.778 · TTFT 9.874 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
178NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
218,20 tok/s TG
Prefill 6.296 · TTFT 1.720 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
179NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,62 tok/s TG
Prefill 4.471 · TTFT 482 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
180Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
244,60 tok/s TG
Prefill 17.751 · TTFT 1.716 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
181Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
150,67 tok/s TG
Prefill 13.212 · TTFT 1.015 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
182Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,77 tok/s TG
Prefill 4.036 · TTFT 628 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
183Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
377,91 tok/s TG
Prefill 19.202 · TTFT 1.574 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
184Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
264,74 tok/s TG
Prefill 14.044 · TTFT 926 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
185Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
129,35 tok/s TG
Prefill 4.876 · TTFT 465 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
186NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
361,45 tok/s TG
Prefill 21.222 · TTFT 1.462 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
187NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
254,44 tok/s TG
Prefill 15.556 · TTFT 831 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
188NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
118,53 tok/s TG
Prefill 5.670 · TTFT 398 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
189Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
365,67 tok/s TG
Prefill 21.449 · TTFT 1.461 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
190Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
256,29 tok/s TG
Prefill 15.783 · TTFT 824 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
191Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
118,36 tok/s TG
Prefill 5.676 · TTFT 402 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
192Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
526,99 tok/s TG
Prefill 19.777 · TTFT 1.490 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
193Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
348,18 tok/s TG
Prefill 14.665 · TTFT 859 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
194Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
120,86 tok/s TG
Prefill 5.701 · TTFT 397 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
195NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
588,46 tok/s TG
Prefill 20.028 · TTFT 1.512 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
196NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
419,41 tok/s TG
Prefill 14.676 · TTFT 885 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
197NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
180,63 tok/s TG
Prefill 5.766 · TTFT 396 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
198Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
592,35 tok/s TG
Prefill 20.010 · TTFT 1.519 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
199Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
415,29 tok/s TG
Prefill 14.768 · TTFT 886 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
200Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
180,59 tok/s TG
Prefill 5.816 · TTFT 394 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
201NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
791,54 tok/s TG
Prefill 26.710 · TTFT 1.226 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
202NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
478,23 tok/s TG
Prefill 20.306 · TTFT 679 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
203NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
115,67 tok/s TG
Prefill 8.650 · TTFT 249 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
204Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
32,44 tok/s TG
Prefill 172 · TTFT 124.920 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
205Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,50 tok/s TG
Prefill 313 · TTFT 43.987 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
206Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
18,49 tok/s TG
Prefill 521 · TTFT 4.134 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
207OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
115,88 tok/s TG
Prefill 1.609 · TTFT 16.977 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
208OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
65,74 tok/s TG
Prefill 1.560 · TTFT 8.148 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
209OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
27,10 tok/s TG
Prefill 1.215 · TTFT 2.015 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
210NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
203,05 tok/s TG
Prefill 5.938 · TTFT 3.711 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
211NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
176,89 tok/s TG
Prefill 5.267 · TTFT 2.069 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
212NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
131,91 tok/s TG
Prefill 4.351 · TTFT 495 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
213NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
259,38 tok/s TG
Prefill 5.806 · TTFT 3.790 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
214NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
228,62 tok/s TG
Prefill 4.731 · TTFT 2.273 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
215NVIDIA-Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
159,67 tok/s TG
Prefill 4.269 · TTFT 504 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
216Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
200,64 tok/s TG
Prefill 5.982 · TTFT 3.713 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
217Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
176,37 tok/s TG
Prefill 4.856 · TTFT 2.232 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
218Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
131,82 tok/s TG
Prefill 4.293 · TTFT 505 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
219Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
206,47 tok/s TG
Prefill 5.182 · TTFT 4.151 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
220Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
177,79 tok/s TG
Prefill 4.968 · TTFT 2.169 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
221Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
132,37 tok/s TG
Prefill 4.260 · TTFT 505 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
222Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
228,98 tok/s TG
Prefill 5.092 · TTFT 4.227 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
223Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
200,96 tok/s TG
Prefill 4.634 · TTFT 2.321 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
224Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
146,23 tok/s TG
Prefill 4.216 · TTFT 510 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
225NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
208,19 tok/s TG
Prefill 6.153 · TTFT 3.583 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
226NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
187,29 tok/s TG
Prefill 4.534 · TTFT 2.375 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
227NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
124,60 tok/s TG
Prefill 4.092 · TTFT 528 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ8_0
228NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
277,39 tok/s TG
Prefill 5.993 · TTFT 3.672 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
229NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
248,17 tok/s TG
Prefill 4.462 · TTFT 2.412 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
230NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
166,48 tok/s TG
Prefill 4.080 · TTFT 529 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
231Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
250,32 tok/s TG
Prefill 5.865 · TTFT 3.782 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
232Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
219,79 tok/s TG
Prefill 4.791 · TTFT 2.262 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
233Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
157,01 tok/s TG
Prefill 4.273 · TTFT 508 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cliQ4_K_M
234NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
106,92 tok/s TG
Prefill 1.319 · TTFT 18.921 ms
10×2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
235NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
100,31 tok/s TG
Prefill 1.092 · TTFT 10.399 ms
5×2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
236NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
72,37 tok/s TG
Prefill 865 · TTFT 2.489 ms
1×2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
237NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
125,68 tok/s TG
Prefill 2.208 · TTFT 11.151 ms
10×2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
238NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
116,24 tok/s TG
Prefill 1.891 · TTFT 5.975 ms
5×2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
239NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
80,94 tok/s TG
Prefill 1.564 · TTFT 1.376 ms
1×2x NVIDIA GeForce RTX 2060 iIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
240Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
326,85 tok/s TG
Prefill 125 · TTFT 210.255 ms
5×AMD Radeon 8060S GraphicsAMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMvLLMgodclawBF16
241Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkPerformancetest Small 1.0
27,50 tok/s TG
Prefill 299 · TTFT 224 ms
1×AMD Radeon 8060S GraphicsAMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMvLLMgodclawBF16
242Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkPerformancetest Small 1.0
27,50 tok/s TG
Prefill 109 · TTFT 604 ms
1×AMD Radeon 8060S GraphicsAMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 31 GB RAMvLLMgodclawBF16
243NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
106,11 tok/s TG
Prefill 1.440 · TTFT 17.007 ms
10×2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
244NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
100,04 tok/s TG
Prefill 1.231 · TTFT 9.195 ms
5×2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
245NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
71,70 tok/s TG
Prefill 945 · TTFT 2.277 ms
1×2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAMllama.cppopenclaw_cliQ4_K_M
246Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
101,36 tok/s TG
Prefill 3.237 · TTFT 6.850 ms
10×NVIDIA DGX SPARK (GB10)NVIDIA Grace · 120 GB RAMllama.cppopenclaw_cliQ8_0
247Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
84,61 tok/s TG
Prefill 2.849 · TTFT 3.806 ms
5×NVIDIA DGX SPARK (GB10)NVIDIA Grace · 120 GB RAMllama.cppopenclaw_cliQ8_0
248Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
62,04 tok/s TG
Prefill 2.341 · TTFT 935 ms
1×NVIDIA DGX SPARK (GB10)NVIDIA Grace · 120 GB RAMllama.cppopenclaw_cliQ8_0
249Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
148,79 tok/s TG
Prefill 3.362 · TTFT 6.588 ms
10×NVIDIA DGX SPARK (GB10)NVIDIA Grace · 120 GB RAMllama.cppopenclaw_cliQ4_K_M
250Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
123,54 tok/s TG
Prefill 2.897 · TTFT 3.743 ms
5×NVIDIA DGX SPARK (GB10)NVIDIA Grace · 120 GB RAMllama.cppopenclaw_cliQ4_K_M