te
Modellserver / Inferenz-Engine
tensorfold
Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.
Best generation
3.484,3 tok/s
NVIDIA-Nemotron-3.5-Lightning-30B-A3B
∅ Generation
781,4 tok/s
Schnitt aus 22 Läufen
Best prefill
26.055 tok/s
Prompt processing
Best TTFT
31 ms
Time to First Token
Top-Modelle auf tensorfold
Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.
Auf welcher Hardware laeuft tensorfold am schnellsten?
Bester Durchsatz je Grafikkarte mit diesem Modellserver.
Performance profile
Durchsatz mit tensorfold
Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.
CPUnach Prozessor
GPUnach Grafikkarte
All measurements
← Alle Engines22 Benchmark-Läufe
Jeder veroeffentlichte Lauf mit tensorfold.
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 3.484,32 tok/s TG Prefill 8.256 · TTFT 14.388 ms | 10× | NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 2 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2.867,15 tok/s TG Prefill 8.107 · TTFT 17.054 ms | 10× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 3 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.694,65 tok/s TG Prefill 7.768 · TTFT 6.482 ms | 5× | NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 4 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.375,08 tok/s TG Prefill 7.576 · TTFT 7.731 ms | 5× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 5 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 883,69 tok/s TG Prefill 26.055 · TTFT 1.240 ms | 10× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-4bit | Details → | |
| 6 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 837,10 tok/s TG Prefill 20.352 · TTFT 1.585 ms | 10× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-4bit | Details → | |
| 7 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 833,31 tok/s TG Prefill 17.472 · TTFT 5.292 ms | 10× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-q4g64 | Details → | |
| 8 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 708,32 tok/s TG Prefill 19.583 · TTFT 709 ms | 5× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-4bit | Details → | |
| 9 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 695,91 tok/s TG Prefill 15.678 · TTFT 892 ms | 5× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-4bit | Details → | |
| 10 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 623,62 tok/s TG Prefill 19.024 · TTFT 1.534 ms | 10× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX-4bit | Details → | |
| 11 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkKurzaufgaben TTFT - Thinking OFF - Outputlimit 1/5/10 V1 | 471,33 tok/s TG Prefill - · TTFT 31 ms | 1× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 12 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 448,85 tok/s TG Prefill 14.780 · TTFT 853 ms | 5× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-q4g64 | Details → | |
| 13 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 446,60 tok/s TG Prefill 14.822 · TTFT 850 ms | 5× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX-4bit | Details → | |
| 14 | Qwen3.8-Flash-Next180BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 344,40 tok/s TG Prefill 16.771 · TTFT 2.037 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 123 GB RAM | tensorfoldgodclaw 1.1.5NVFP4/MXFP8 | Details → | |
| 15 | Qwen3.8-Flash-Next180BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 244,85 tok/s TG Prefill 14.238 · TTFT 934 ms | 5× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 123 GB RAM | tensorfoldgodclaw 1.1.5NVFP4/MXFP8 | Details → | |
| 16 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 239,60 tok/s TG Prefill 8.173 · TTFT 263 ms | 1× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-4bit | Details → | |
| 17 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 235,73 tok/s TG Prefill 6.703 · TTFT 322 ms | 1× | NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 18 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 226,44 tok/s TG Prefill 6.517 · TTFT 330 ms | 1× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-4bit | Details → | |
| 19 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 175,99 tok/s TG Prefill 6.612 · TTFT 326 ms | 1× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX 4-bit | Details → | |
| 20 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 145,72 tok/s TG Prefill 6.273 · TTFT 308 ms | 1× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclawMLX-q4g64 | Details → | |
| 21 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 132,71 tok/s TG Prefill 6.165 · TTFT 313 ms | 1× | NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAM | tensorfoldgodclaw 1.1.6MLX-4bit | Details → | |
| 22 | Qwen3.8-Flash-Next180BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 74,79 tok/s TG Prefill 5.764 · TTFT 344 ms | 1× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 123 GB RAM | tensorfoldgodclaw 1.1.5NVFP4/MXFP8 | Details → |

