Created bymario-alka.dePowered bynoob2claw.detricoma.de
Meta

Llama-3.2-3B-Instruct3B

Benchmarkprofil und veröffentlichte Ergebnisse.

Dense3B
Einordnung im Feld

Bestwerte im Vergleich

Beste Messwerte dieses Modells gegen Minimum, Durchschnitt und Maximum aller veröffentlichten Systeme.

Generation465,7 tok/s
Min 0,0Ø 245,7Max 3.484,3
Ueber dem Durchschnitt · 4971 Systeme im Feld
Prefill23.398 tok/s
Min 4Ø 4.892Max 112.346
Ueber dem Durchschnitt · 4953 Systeme im Feld
Time to First Token302 ms
Min 20Ø 31.420Max 535.235
Ueber dem Durchschnitt · 4968 Systeme im Feld
Performanceprofil

Durchsatz nach Hardware & Engine

Jede Blase steht für eine GPU, CPU bzw. Engine – die Position zeigt Prompt-Verarbeitung (X) und Ausgabe-Geschwindigkeit (Y), die Blasengröße die Anzahl der Messläufe.

GPUnach Grafikkarte

5934452961480,005.81511.63117.446Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 465,7 tok/s Generation, 14.196 tok/s Prefill, TTFT 1.049 ms (9 Laufe)AMD Radeon RX 7900 XTXAMD Radeon AI PRO R9700 - 397,2 tok/s Generation, 11.893 tok/s Prefill, TTFT 1.135 ms (3 Laufe)AMD Radeon AI PRO R97...NVIDIA GeForce RTX 2060 - 226,6 tok/s Generation, 6.927 tok/s Prefill, TTFT 2.004 ms (6 Laufe)NVIDIA GeForce RTX 20...CPU-only - 62,9 tok/s Generation, 870 tok/s Prefill, TTFT 163.072 ms (5 Laufe)CPU-only
AMD Radeon RX 7900 XTX 465,7 tok/sAMD Radeon AI PRO R9700 397,2 tok/sNVIDIA GeForce RTX 2060 226,6 tok/sCPU-only 62,9 tok/s

CPUnach Prozessor

6054543021510,006.36012.71919.079Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 3955WX 16-Cores - 465,7 tok/s Generation, 15.391 tok/s Prefill, TTFT 992 ms (6 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 3 3100 4-Core Processor - 397,2 tok/s Generation, 11.893 tok/s Prefill, TTFT 1.135 ms (3 Laufe)AMD Ryzen 3 3100 4-Co...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 354,2 tok/s Generation, 8.553 tok/s Prefill, TTFT 1.723 ms (9 Laufe)Intel(R) Xeon(R) CPU ...Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) - 62,9 tok/s Generation, 2.562 tok/s Prefill, TTFT 1.026 ms (1 Lauf)Apple M6 (unified CPU...Apple M6 (unified CPU/GPU SoC) - 52,9 tok/s Generation, 1.698 tok/s Prefill, TTFT 1.508 ms (1 Lauf)Apple M6 (unified CPU...4 x Cortex-A78C + 4 x Cortex-A55 - 2,5 tok/s Generation, 29 tok/s Prefill, TTFT 270.941 ms (3 Laufe)4 x Cortex-A78C + 4 x...
AMD Ryzen Threadripper PRO 3955WX 16-Cores 465,7 tok/sAMD Ryzen 3 3100 4-Core Processor 397,2 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 354,2 tok/sApple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) 62,9 tok/sApple M6 (unified CPU/GPU SoC) 52,9 tok/s4 x Cortex-A78C + 4 x Cortex-A55 2,5 tok/s

ENGnach Engine

5954462971490,006.87513.75020.624Prefill (tok/s)Generation (tok/s)vLLM - 465,7 tok/s Generation, 16.879 tok/s Prefill, TTFT 1.016 ms (3 Laufe)vLLMllama.cpp - 397,2 tok/s Generation, 8.581 tok/s Prefill, TTFT 46.369 ms (18 Laufe)llama.cpplmstudio-mlx - 62,9 tok/s Generation, 2.562 tok/s Prefill, TTFT 1.026 ms (1 Lauf)lmstudio-mlxOllama - 52,9 tok/s Generation, 1.698 tok/s Prefill, TTFT 1.508 ms (1 Lauf)Ollama
vLLM 465,7 tok/sllama.cpp 397,2 tok/slmstudio-mlx 62,9 tok/sOllama 52,9 tok/s
Durchsatz & Latenz

Performancebenchmark

Messwert:
GPUs
#Modell / HerstellerMesswerteParallelGPU / CPU / RAMRuntime
1Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
465,73 tok/s TG
Prefill 23.398 · TTFT 1.915 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclaw
2Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
397,16 tok/s TG
Prefill 14.911 · TTFT 2.001 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
3Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
360,12 tok/s TG
Prefill 19.755 · TTFT 1.579 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
4Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
354,21 tok/s TG
Prefill 17.373 · TTFT 1.825 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
5Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
318,54 tok/s TG
Prefill 19.091 · TTFT 830 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclaw
6Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
261,48 tok/s TG
Prefill 13.512 · TTFT 1.061 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
7Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
241,23 tok/s TG
Prefill 15.447 · TTFT 931 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
8Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
226,59 tok/s TG
Prefill 9.526 · TTFT 3.488 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
9Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
215,97 tok/s TG
Prefill 9.775 · TTFT 3.283 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0
10Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
196,57 tok/s TG
Prefill 12.975 · TTFT 1.133 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
11Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
173,08 tok/s TG
Prefill 7.158 · TTFT 2.024 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
12Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
158,90 tok/s TG
Prefill 7.616 · TTFT 1.908 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0
13Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
109,96 tok/s TG
Prefill 5.072 · TTFT 529 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
14Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
107,15 tok/s TG
Prefill 6.505 · TTFT 397 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
15Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
106,38 tok/s TG
Prefill 7.256 · TTFT 344 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
16Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
100,62 tok/s TG
Prefill 3.646 · TTFT 677 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
17Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
70,69 tok/s TG
Prefill 3.840 · TTFT 645 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0
18Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
69,35 tok/s TG
Prefill 8.149 · TTFT 302 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclaw
19Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
62,86 tok/s TG
Prefill 2.562 · TTFT 1.026 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
20Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
52,92 tok/s TG
Prefill 1.698 · TTFT 1.508 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
21Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
2,46 tok/s TG
Prefill 35 · TTFT 360.878 ms
5×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_M
22Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
1,99 tok/s TG
Prefill 22 · TTFT 104.643 ms
1×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_M
23Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
0,09 tok/s TG
Prefill 31 · TTFT 347.304 ms
10×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_M
Agent- & Chat-Bewertung

Harnessbenchmark

Noch keine Harnessbenchmarks fuer dieses Modell.