Created bymario-alka.dePowered bynoob2claw.detricoma.de
te
Modellserver / Inferenz-Engine

tensorfold

Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.

22Benchmarks
3Models
Best generation
3.484,3 tok/s
NVIDIA-Nemotron-3.5-Lightning-30B-A3B
∅ Generation
781,4 tok/s
Schnitt aus 22 Läufen
Best prefill
26.055 tok/s
Prompt processing
Best TTFT
31 ms
Time to First Token

Top-Modelle auf tensorfold

Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.

NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6B
3.484,3 tok/s
Qwen3.6-35B-A3B35B
833,3 tok/s
Qwen3.8-Flash-Next180B
344,4 tok/s

Auf welcher Hardware laeuft tensorfold am schnellsten?

Bester Durchsatz je Grafikkarte mit diesem Modellserver.

NVIDIA GeForce RTX 3090
3.484,3 tok/s
NVIDIA RTX PRO 6000 Blackwell Workstation Edition
344,4 tok/s
Performance profile

Durchsatz mit tensorfold

Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

CPUnach Prozessor

4.4613.3462.2301.1150,011.47511.98312.49213.001Prefill (tok/s)Generation (tok/s)AMD Ryzen 5 5600X 6-Core Processor - 3.484,3 tok/s Generation, 12.218 tok/s Prefill, TTFT 3.184 ms (19 Laufe)AMD Ryzen 5 5600X 6-C...AMD Ryzen 9 9950X 16-Core Processor - 344,4 tok/s Generation, 12.258 tok/s Prefill, TTFT 1.105 ms (3 Laufe)AMD Ryzen 9 9950X 16-...
AMD Ryzen 5 5600X 6-Core Processor 3.484,3 tok/sAMD Ryzen 9 9950X 16-Core Processor 344,4 tok/s

GPUnach Grafikkarte

4.4613.3462.2301.1150,011.47511.98312.49213.001Prefill (tok/s)Generation (tok/s)NVIDIA GeForce RTX 3090 - 3.484,3 tok/s Generation, 12.218 tok/s Prefill, TTFT 3.184 ms (19 Laufe)NVIDIA GeForce RTX 30...NVIDIA RTX PRO 6000 Blackwell Workstation Edition - 344,4 tok/s Generation, 12.258 tok/s Prefill, TTFT 1.105 ms (3 Laufe)NVIDIA RTX PRO 6000 B...
NVIDIA GeForce RTX 3090 3.484,3 tok/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 344,4 tok/s

MODnach Modell

4.4613.3462.2301.1150,010.76111.87712.99414.110Prefill (tok/s)Generation (tok/s)NVIDIA-Nemotron-3.5-Lightning-30B-A3B - 3.484,3 tok/s Generation, 11.782 tok/s Prefill, TTFT 3.950 ms (13 Laufe)NVIDIA-Nemotron-3.5-L...Qwen3.6-35B-A3B - 833,3 tok/s Generation, 13.089 tok/s Prefill, TTFT 1.525 ms (6 Laufe)Qwen3.6-35B-A3BQwen3.8-Flash-Next - 344,4 tok/s Generation, 12.258 tok/s Prefill, TTFT 1.105 ms (3 Laufe)Qwen3.8-Flash-Next
NVIDIA-Nemotron-3.5-Lightning-30B-A3B 3.484,3 tok/sQwen3.6-35B-A3B 833,3 tok/sQwen3.8-Flash-Next 344,4 tok/s
All measurements

22 Benchmark-Läufe

Jeder veroeffentlichte Lauf mit tensorfold.

← Alle Engines
Metric:
Size
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
3.484,32 tok/s TG
Prefill 8.256 · TTFT 14.388 ms
10×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
2NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.867,15 tok/s TG
Prefill 8.107 · TTFT 17.054 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
3NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.694,65 tok/s TG
Prefill 7.768 · TTFT 6.482 ms
5×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
4NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.375,08 tok/s TG
Prefill 7.576 · TTFT 7.731 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
5NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
883,69 tok/s TG
Prefill 26.055 · TTFT 1.240 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
6NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
837,10 tok/s TG
Prefill 20.352 · TTFT 1.585 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
7Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
833,31 tok/s TG
Prefill 17.472 · TTFT 5.292 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-q4g64
8NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
708,32 tok/s TG
Prefill 19.583 · TTFT 709 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
9NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
695,91 tok/s TG
Prefill 15.678 · TTFT 892 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
10Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
623,62 tok/s TG
Prefill 19.024 · TTFT 1.534 ms
10×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX-4bit
11NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkKurzaufgaben TTFT - Thinking OFF - Outputlimit 1/5/10 V1
471,33 tok/s TG
Prefill - · TTFT 31 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
12Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
448,85 tok/s TG
Prefill 14.780 · TTFT 853 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-q4g64
13Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
446,60 tok/s TG
Prefill 14.822 · TTFT 850 ms
5×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX-4bit
14Qwen3.8-Flash-Next180BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
344,40 tok/s TG
Prefill 16.771 · TTFT 2.037 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 123 GB RAMtensorfoldgodclaw 1.1.5NVFP4/MXFP8
15Qwen3.8-Flash-Next180BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
244,85 tok/s TG
Prefill 14.238 · TTFT 934 ms
5×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 123 GB RAMtensorfoldgodclaw 1.1.5NVFP4/MXFP8
16NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
239,60 tok/s TG
Prefill 8.173 · TTFT 263 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
17NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
235,73 tok/s TG
Prefill 6.703 · TTFT 322 ms
1×NVIDIA GeForce RTX 3090AMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
18NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
226,44 tok/s TG
Prefill 6.517 · TTFT 330 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-4bit
19NVIDIA-Nemotron-3.5-Lightning-30B-A3B31.6BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
175,99 tok/s TG
Prefill 6.612 · TTFT 326 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX 4-bit
20Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,72 tok/s TG
Prefill 6.273 · TTFT 308 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclawMLX-q4g64
21Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
132,71 tok/s TG
Prefill 6.165 · TTFT 313 ms
1×NVIDIA GeForce RTX 3090 iAMD Ryzen 5 5600X 6-Core Processor · 31 GB RAMtensorfoldgodclaw 1.1.6MLX-4bit
22Qwen3.8-Flash-Next180BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
74,79 tok/s TG
Prefill 5.764 · TTFT 344 ms
1×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 123 GB RAMtensorfoldgodclaw 1.1.5NVFP4/MXFP8