Created bymario-alka.dePowered bynoob2claw.detricoma.de
Qwen (Alibaba)

Qwen2.5-7B-Instruct7B

Benchmark profile and published results.

Dense7B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

Generation442,0 tok/s
Min 0,0Ø 245,7Max 3.484,3
Ueber dem Durchschnitt · 4971 Systeme im Feld
Prefill16.333 tok/s
Min 4Ø 4.892Max 112.346
Ueber dem Durchschnitt · 4953 Systeme im Feld
Time to First Token411 ms
Min 20Ø 31.420Max 535.235
Ueber dem Durchschnitt · 4968 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

5684262841420,003.9057.81011.714Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 442,0 tok/s Generation, 9.575 tok/s Prefill, TTFT 1.732 ms (6 Laufe)AMD Radeon RX 7900 XTXAMD Radeon AI PRO R9700 - 250,6 tok/s Generation, 6.580 tok/s Prefill, TTFT 2.039 ms (3 Laufe)AMD Radeon AI PRO R97...NVIDIA GeForce RTX 2060 - 160,3 tok/s Generation, 3.649 tok/s Prefill, TTFT 3.816 ms (3 Laufe)NVIDIA GeForce RTX 20...CPU-only - 31,5 tok/s Generation, 880 tok/s Prefill, TTFT 3.112 ms (2 Laufe)CPU-only
AMD Radeon RX 7900 XTX 442,0 tok/sAMD Radeon AI PRO R9700 250,6 tok/sNVIDIA GeForce RTX 2060 160,3 tok/sCPU-only 31,5 tok/s

CPUby processor

5694272841420,003.9197.83911.758Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 3955WX 16-Cores - 442,0 tok/s Generation, 9.575 tok/s Prefill, TTFT 1.732 ms (6 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 3 3100 4-Core Processor - 250,6 tok/s Generation, 6.580 tok/s Prefill, TTFT 2.039 ms (3 Laufe)AMD Ryzen 3 3100 4-Co...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 160,3 tok/s Generation, 3.649 tok/s Prefill, TTFT 3.816 ms (3 Laufe)Intel(R) Xeon(R) CPU ...Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) - 31,5 tok/s Generation, 1.123 tok/s Prefill, TTFT 2.322 ms (1 Lauf)Apple M6 (unified CPU...Apple M6 (unified CPU/GPU SoC) - 28,3 tok/s Generation, 637 tok/s Prefill, TTFT 3.902 ms (1 Lauf)Apple M6 (unified CPU...
AMD Ryzen Threadripper PRO 3955WX 16-Cores 442,0 tok/sAMD Ryzen 3 3100 4-Core Processor 250,6 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 160,3 tok/sApple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) 31,5 tok/sApple M6 (unified CPU/GPU SoC) 28,3 tok/s

ENGby engine

5694272841420,004.8859.77014.656Prefill (tok/s)Generation (tok/s)vLLM - 442,0 tok/s Generation, 11.912 tok/s Prefill, TTFT 1.633 ms (3 Laufe)vLLMllama.cpp - 261,4 tok/s Generation, 5.822 tok/s Prefill, TTFT 2.562 ms (9 Laufe)llama.cpplmstudio-mlx - 31,5 tok/s Generation, 1.123 tok/s Prefill, TTFT 2.322 ms (1 Lauf)lmstudio-mlxOllama - 28,3 tok/s Generation, 637 tok/s Prefill, TTFT 3.902 ms (1 Lauf)Ollama
vLLM 442,0 tok/sllama.cpp 261,4 tok/slmstudio-mlx 31,5 tok/sOllama 28,3 tok/s
Throughput & latency

Performance benchmark

Metric:
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
442,02 tok/s TG
Prefill 16.333 · TTFT 3.286 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclaw
2Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
326,73 tok/s TG
Prefill 13.426 · TTFT 1.202 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclaw
3Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
261,39 tok/s TG
Prefill 10.101 · TTFT 2.985 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
4Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
250,57 tok/s TG
Prefill 9.085 · TTFT 3.442 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
5Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
178,27 tok/s TG
Prefill 8.322 · TTFT 1.724 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
6Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
168,09 tok/s TG
Prefill 6.490 · TTFT 2.083 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
7Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
160,26 tok/s TG
Prefill 4.979 · TTFT 6.510 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
8Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,79 tok/s TG
Prefill 3.816 · TTFT 3.796 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
9Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
76,83 tok/s TG
Prefill 3.292 · TTFT 785 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_M
10Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
68,23 tok/s TG
Prefill 5.975 · TTFT 411 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclaw
11Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
58,79 tok/s TG
Prefill 4.166 · TTFT 593 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0
12Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
56,92 tok/s TG
Prefill 2.151 · TTFT 1.142 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
13Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,46 tok/s TG
Prefill 1.123 · TTFT 2.322 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
14Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,30 tok/s TG
Prefill 637 · TTFT 3.902 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.