Created bymario-alka.dePowered bynoob2claw.detricoma.de
Google

gemma-2-9b-it9B

Benchmark profile and published results.

Dense9B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

Generation171,6 tok/s
Min 0,0Ø 245,7Max 3.484,3
Unter dem Durchschnitt · 4971 Systeme im Feld
Prefill1.337 tok/s
Min 4Ø 4.892Max 112.346
Unter dem Durchschnitt · 4953 Systeme im Feld
Time to First Token1.405 ms
Min 20Ø 31.420Max 535.235
Ueber dem Durchschnitt · 4968 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

21916410954,70,04746798831.088Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 171,6 tok/s Generation, 964 tok/s Prefill, TTFT 22.191 ms (3 Laufe)AMD Radeon RX 7900 XTXCPU-only - 22,0 tok/s Generation, 598 tok/s Prefill, TTFT 3.286 ms (2 Laufe)CPU-only
AMD Radeon RX 7900 XTX 171,6 tok/sCPU-only 22,0 tok/s

CPUby processor

21916410954,70,04176448701.097Prefill (tok/s)Generation (tok/s)Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 171,6 tok/s Generation, 964 tok/s Prefill, TTFT 22.191 ms (3 Laufe)Intel(R) Xeon(R) CPU ...Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) - 22,0 tok/s Generation, 646 tok/s Prefill, TTFT 3.068 ms (1 Lauf)Apple M6 (unified CPU...Apple M6 (unified CPU/GPU SoC) - 20,3 tok/s Generation, 550 tok/s Prefill, TTFT 3.503 ms (1 Lauf)Apple M6 (unified CPU...
Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 171,6 tok/sApple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) 22,0 tok/sApple M6 (unified CPU/GPU SoC) 20,3 tok/s

ENGby engine

21916410954,70,04176448701.097Prefill (tok/s)Generation (tok/s)llama.cpp - 171,6 tok/s Generation, 964 tok/s Prefill, TTFT 22.191 ms (3 Laufe)llama.cpplmstudio-mlx - 22,0 tok/s Generation, 646 tok/s Prefill, TTFT 3.068 ms (1 Lauf)lmstudio-mlxOllama - 20,3 tok/s Generation, 550 tok/s Prefill, TTFT 3.503 ms (1 Lauf)Ollama
llama.cpp 171,6 tok/slmstudio-mlx 22,0 tok/sOllama 20,3 tok/s
Throughput & latency

Performance benchmark

Metric:
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1gemma-2-9b-it9BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
171,56 tok/s TG
Prefill 378 · TTFT 49.531 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
2gemma-2-9b-it9BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
111,81 tok/s TG
Prefill 1.178 · TTFT 15.636 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
3gemma-2-9b-it9BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
56,49 tok/s TG
Prefill 1.337 · TTFT 1.405 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
4gemma-2-9b-it9BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,98 tok/s TG
Prefill 646 · TTFT 3.068 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
5gemma-2-9b-it9BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
20,31 tok/s TG
Prefill 550 · TTFT 3.503 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.