Created bymario-alka.dePowered bynoob2claw.detricoma.de
Google

gemma-4-31B-it31B

Benchmark profile and published results.

Dense31B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

Generation251,1 tok/s
Min 5,0Ø 249,2Max 1.568,0
Ueber dem Durchschnitt · 1031 Systeme im Feld
Prefill7.025 tok/s
Min 94Ø 8.420Max 55.260
Unter dem Durchschnitt · 1031 Systeme im Feld
Time to First Token751 ms
Min 117Ø 5.294Max 124.920
Ueber dem Durchschnitt · 1031 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

32224216180,60,001.0252.0503.075Prefill (tok/s)Generation (tok/s)NVIDIA RTX A6000 - 251,1 tok/s Generation, 2.529 tok/s Prefill, TTFT 12.576 ms (30 Laufe)NVIDIA RTX A6000AMD Radeon RX 7900 XTX - 108,0 tok/s Generation, 399 tok/s Prefill, TTFT 62.261 ms (3 Laufe)AMD Radeon RX 7900 XTXAMD Radeon PRO W7900 Dual Slot - 101,9 tok/s Generation, 351 tok/s Prefill, TTFT 52.767 ms (6 Laufe)AMD Radeon PRO W7900 ...NVIDIA GeForce RTX 3090 Ti - 19,5 tok/s Generation, 339 tok/s Prefill, TTFT 80.766 ms (1 Lauf)NVIDIA GeForce RTX 30...
NVIDIA RTX A6000 251,1 tok/sAMD Radeon RX 7900 XTX 108,0 tok/sAMD Radeon PRO W7900 Dual Slot 101,9 tok/sNVIDIA GeForce RTX 3090 Ti 19,5 tok/s

CPUby processor

32224216180,60,001.0252.0503.075Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 7955WX 16-Cores - 251,1 tok/s Generation, 2.529 tok/s Prefill, TTFT 12.576 ms (30 Laufe)AMD Ryzen Threadrippe...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 108,0 tok/s Generation, 399 tok/s Prefill, TTFT 62.261 ms (3 Laufe)Intel(R) Xeon(R) CPU ...AMD Ryzen Threadripper PRO 5975WX 32-Cores - 101,9 tok/s Generation, 351 tok/s Prefill, TTFT 52.767 ms (6 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 9 8945HX with Radeon Graphics - 19,5 tok/s Generation, 339 tok/s Prefill, TTFT 80.766 ms (1 Lauf)AMD Ryzen 9 8945HX wi...
AMD Ryzen Threadripper PRO 7955WX 16-Cores 251,1 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 108,0 tok/sAMD Ryzen Threadripper PRO 5975WX 32-Cores 101,9 tok/sAMD Ryzen 9 8945HX with Radeon Graphics 19,5 tok/s

ENGby engine

32224216180,60,001.4842.9694.453Prefill (tok/s)Generation (tok/s)vLLM - 251,1 tok/s Generation, 3.640 tok/s Prefill, TTFT 6.597 ms (18 Laufe)vLLMllama.cpp - 129,2 tok/s Generation, 650 tok/s Prefill, TTFT 36.282 ms (21 Laufe)llama.cppllm - 19,5 tok/s Generation, 339 tok/s Prefill, TTFT 80.766 ms (1 Lauf)llm
vLLM 251,1 tok/sllama.cpp 129,2 tok/sllm 19,5 tok/s
Throughput & latency

Performance benchmark

Metric:
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
251,07 tok/s TG
Prefill 7.025 · TTFT 4.495 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
2gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
236,35 tok/s TG
Prefill 6.513 · TTFT 4.844 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
3gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
236,03 tok/s TG
Prefill 5.472 · TTFT 5.845 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
4gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
222,83 tok/s TG
Prefill 5.037 · TTFT 6.267 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
5gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
152,19 tok/s TG
Prefill 5.535 · TTFT 2.477 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
6gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
145,41 tok/s TG
Prefill 5.094 · TTFT 2.683 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
7gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
143,75 tok/s TG
Prefill 4.220 · TTFT 3.249 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
8gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
140,87 tok/s TG
Prefill 3.732 · TTFT 25.360 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
9gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
138,37 tok/s TG
Prefill 2.284 · TTFT 42.505 ms
10×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
10gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
137,92 tok/s TG
Prefill 3.964 · TTFT 3.462 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
11gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
129,24 tok/s TG
Prefill 622 · TTFT 48.720 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
12gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
128,36 tok/s TG
Prefill 621 · TTFT 48.995 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
13gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
127,17 tok/s TG
Prefill 646 · TTFT 44.975 ms
10×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
14gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
127,05 tok/s TG
Prefill 641 · TTFT 45.163 ms
10×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
15gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,03 tok/s TG
Prefill 121 · TTFT 163.657 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
16gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
101,94 tok/s TG
Prefill 176 · TTFT 112.412 ms
10×2x AMD Radeon PRO W7900 Dual SlotAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
17gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
99,95 tok/s TG
Prefill 166 · TTFT 119.519 ms
10×2x AMD Radeon PRO W7900 Dual SlotAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
18gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
78,73 tok/s TG
Prefill 574 · TTFT 19.191 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
19gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
74,56 tok/s TG
Prefill 3.148 · TTFT 4.375 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
20gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
72,98 tok/s TG
Prefill 351 · TTFT 36.215 ms
5×2x AMD Radeon PRO W7900 Dual SlotAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
21gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
70,54 tok/s TG
Prefill 2.088 · TTFT 6.294 ms
5×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
22gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
69,09 tok/s TG
Prefill 846 · TTFT 16.044 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
23gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
68,74 tok/s TG
Prefill 843 · TTFT 15.945 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
24gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
54,15 tok/s TG
Prefill 435 · TTFT 38.812 ms
5×2x AMD Radeon PRO W7900 Dual SlotAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
25gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
52,97 tok/s TG
Prefill 874 · TTFT 15.734 ms
5×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
26gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
52,93 tok/s TG
Prefill 870 · TTFT 15.730 ms
5×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
27gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
36,87 tok/s TG
Prefill 2.641 · TTFT 751 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
28gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
35,35 tok/s TG
Prefill 2.414 · TTFT 821 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawFP8
29gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
34,84 tok/s TG
Prefill 1.988 · TTFT 997 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
30gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
33,60 tok/s TG
Prefill 1.857 · TTFT 1.065 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawAWQ
31gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,45 tok/s TG
Prefill 1.090 · TTFT 1.818 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
32gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,44 tok/s TG
Prefill 1.083 · TTFT 1.829 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ4_K_M
33gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,24 tok/s TG
Prefill 502 · TTFT 3.937 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
34gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
24,81 tok/s TG
Prefill 701 · TTFT 2.818 ms
1×2x AMD Radeon PRO W7900 Dual SlotAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_M
35gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,50 tok/s TG
Prefill 339 · TTFT 80.766 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllmcodex_cli
36gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
19,44 tok/s TG
Prefill 276 · TTFT 6.828 ms
1×2x AMD Radeon PRO W7900 Dual SlotAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ8_0
37gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
18,78 tok/s TG
Prefill 1.111 · TTFT 1.784 ms
1×2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
38gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
18,77 tok/s TG
Prefill 1.107 · TTFT 1.793 ms
1×2x NVIDIA RTX A6000 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppgodclawQ8_0
39gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
17,73 tok/s TG
Prefill 1.490 · TTFT 1.332 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliFP8
40gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
16,81 tok/s TG
Prefill 1.027 · TTFT 1.929 ms
1×NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMopenclaw_cliAWQ
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.