pleeplycommunity

Community

Community benchmark: what runs how fast on your hardware?

Models & benchmarks0 replies
n0n_ov9r3leptTeam

How fast do open models run on your hardware? Let's collect real numbers in one place.

Please reply with:

  • Hardware (e.g. Raspberry Pi 5 8 GB, Mac mini 16 GB, RTX 3060 12 GB)
  • Runtime and version (Ollama, llama.cpp, MLX…)
  • Model and quantization (e.g. llama3.2:3b, Q4_K_M)
  • Tokens per second for generation (and prompt processing, if you have it)
  • Cooling and power notes

For reference, published Ollama results on a Raspberry Pi 5 are about 11 tokens/s for 1B models and about 5 for 3B models at Q4_K_M. Details and sources: https://pleeply.vercel.app/info/raspberry-pi-5-llm/

We'll keep the best numbers in a table in our docs and credit you by name if you want.

Join the community to reply. It takes a few seconds and needs no email.

Sign up Log in