How fast do open models run on your hardware? Let's collect real numbers in one place.
Please reply with:
- Hardware (e.g. Raspberry Pi 5 8 GB, Mac mini 16 GB, RTX 3060 12 GB)
- Runtime and version (Ollama, llama.cpp, MLX…)
- Model and quantization (e.g. llama3.2:3b, Q4_K_M)
- Tokens per second for generation (and prompt processing, if you have it)
- Cooling and power notes
For reference, published Ollama results on a Raspberry Pi 5 are about 11 tokens/s for 1B models and about 5 for 3B models at Q4_K_M. Details and sources: https://pleeply.vercel.app/info/raspberry-pi-5-llm/
We'll keep the best numbers in a table in our docs and credit you by name if you want.