Created bymario-alka.dePowered bynoob2claw.detricoma.de
Qwen (Alibaba)

Qwen2.5-32B-Instruct-AWQ32B

Benchmark profile and published results.

Dense32B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

Generation222,0 tok/s
Min 0,0Ø 245,7Max 3.484,3
Unter dem Durchschnitt · 4971 Systeme im Feld
Prefill2.436 tok/s
Min 4Ø 4.892Max 112.346
Unter dem Durchschnitt · 4953 Systeme im Feld
Time to First Token1.705 ms
Min 20Ø 31.420Max 535.235
Ueber dem Durchschnitt · 4968 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

2442332222112001.9262.0082.0902.172Prefill (tok/s)Generation (tok/s)NVIDIA GeForce RTX 3090 Ti - 222,0 tok/s Generation, 2.049 tok/s Prefill, TTFT 20.300 ms (3 Laufe)NVIDIA GeForce RTX 30...
NVIDIA GeForce RTX 3090 Ti 222,0 tok/s

CPUby processor

2442332222112001.9262.0082.0902.172Prefill (tok/s)Generation (tok/s)AMD Ryzen 9 8945HX with Radeon Graphics - 222,0 tok/s Generation, 2.049 tok/s Prefill, TTFT 20.300 ms (3 Laufe)AMD Ryzen 9 8945HX wi...
AMD Ryzen 9 8945HX with Radeon Graphics 222,0 tok/s

ENGby engine

2442332222112001.9262.0082.0902.172Prefill (tok/s)Generation (tok/s)llama.cpp - 222,0 tok/s Generation, 2.049 tok/s Prefill, TTFT 20.300 ms (3 Laufe)llama.cpp
llama.cpp 222,0 tok/s
Throughput & latency

Performance benchmark

Metric:
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
222,01 tok/s TG
Prefill 2.436 · TTFT 45.755 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketAWQ
2Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
124,33 tok/s TG
Prefill 2.267 · TTFT 13.440 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketAWQ
3Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
40,18 tok/s TG
Prefill 1.444 · TTFT 1.705 ms
1×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketAWQ
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.