Created bymario-alka.dePowered bynoob2claw.detricoma.de
Mistral AI

Mistral-Small-24B-Instruct-250123.6B

Benchmark profile and published results.

Dense23.6B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

Generation153,1 tok/s
Min 0,0Ø 245,7Max 3.484,3
Unter dem Durchschnitt · 4971 Systeme im Feld
Prefill4.348 tok/s
Min 4Ø 4.892Max 112.346
Unter dem Durchschnitt · 4953 Systeme im Feld
Time to First Token1.390 ms
Min 20Ø 31.420Max 535.235
Ueber dem Durchschnitt · 4968 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

19714898,549,30,001.2702.5403.810Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 153,1 tok/s Generation, 3.108 tok/s Prefill, TTFT 4.107 ms (3 Laufe)AMD Radeon RX 7900 XTXCPU-only - 9,9 tok/s Generation, 246 tok/s Prefill, TTFT 9.302 ms (2 Laufe)CPU-only
AMD Radeon RX 7900 XTX 153,1 tok/sCPU-only 9,9 tok/s

CPUby processor

19714898,749,30,001.2712.5423.814Prefill (tok/s)Generation (tok/s)Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 153,1 tok/s Generation, 3.108 tok/s Prefill, TTFT 4.107 ms (3 Laufe)Intel(R) Xeon(R) CPU ...Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) - 9,9 tok/s Generation, 266 tok/s Prefill, TTFT 8.275 ms (1 Lauf)Apple M6 (unified CPU...Apple M6 (unified CPU/GPU SoC) - 8,5 tok/s Generation, 226 tok/s Prefill, TTFT 10.329 ms (1 Lauf)Apple M6 (unified CPU...
Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 153,1 tok/sApple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) 9,9 tok/sApple M6 (unified CPU/GPU SoC) 8,5 tok/s

ENGby engine

19714898,749,30,001.2712.5423.814Prefill (tok/s)Generation (tok/s)llama.cpp - 153,1 tok/s Generation, 3.108 tok/s Prefill, TTFT 4.107 ms (3 Laufe)llama.cpplmstudio-mlx - 9,9 tok/s Generation, 266 tok/s Prefill, TTFT 8.275 ms (1 Lauf)lmstudio-mlxOllama - 8,5 tok/s Generation, 226 tok/s Prefill, TTFT 10.329 ms (1 Lauf)Ollama
llama.cpp 153,1 tok/slmstudio-mlx 9,9 tok/sOllama 8,5 tok/s
Throughput & latency

Performance benchmark

Metric:
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
153,12 tok/s TG
Prefill 4.348 · TTFT 7.017 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
2Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
86,87 tok/s TG
Prefill 3.428 · TTFT 3.916 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
3Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
38,51 tok/s TG
Prefill 1.549 · TTFT 1.390 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_M
4Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,90 tok/s TG
Prefill 266 · TTFT 8.275 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit
5Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
8,47 tok/s TG
Prefill 226 · TTFT 10.329 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_M
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.