pleeplydocs

Run it yourself

Run AI at home: choose your hardware

Updated 4 min read View .md

You can run a private AI assistant on anything from a single-board computer to a desktop with a graphics card. Memory is what decides: it sets the largest model you can run, and the model's size sets how capable and how fast your assistant is.

Choose your setup#

Jetson Orin Nano Super

Best for: faster small models on a low-power board.

Models: 3B-class models at roughly 22–43 tokens per second in published tests.

Requirements: 8 GB of shared memory, so small models only. Launched at $249 in December 2024.

Mac mini with Apple Silicon

Best for: quiet, capable 7–8B models.

Models: 16 GB runs 7–8B at 4-bit; 24 GB or more opens 14–32B.

Requirements: 16 GB of memory for a useful assistant.

PC or mini PC with a GPU

Best for: larger models and the fastest answers.

Models: set by the card's memory; a 12 GB card such as an RTX 3060 runs 7–8B models.

Requirements: more power and noise, so it usually lives away from the speaker.

Compare at a glance#

SetupMemoryPractical model sizeWatch out for
Raspberry Pi 58 or 16 GB1–3B; 7B is slowThrottles without active cooling
Jetson Orin Nano Super8 GB sharedUp to about 3–4BPublished speeds vary by model and runtime
Mac mini, 16 GB16 GB unified7–8B at 4-bitMemory can't be upgraded later
Mac mini, 24 GB or more24–64 GB unified14–32B at 4-bitCost rises quickly with memory
PC with a 12 GB GPU12 GB of VRAM plus system RAM7–8B; more with bigger cardsPower draw and fan noise
Home Assistant Voice PEUses your serverDepends on the serverNeeds a Home Assistant server

What size of model do you need?#

  • 1–3B parameters: commands, timers, short answers, simple lookups in your notes. Fits 8 GB devices.
  • 7–8B: natural conversation, summaries, better at following instructions. Needs about 16 GB.
  • 14–32B: noticeably better reasoning and writing. Needs 24–48 GB.

Details in How much RAM a local LLM needs.

FAQ#

What is the cheapest way to run a local LLM at home?#

A Raspberry Pi 5 with 8 GB is the cheapest common option. Third-party tests show it running 1–3B models at roughly 5–11 tokens per second — fine for commands and short answers, slow for conversation.

Is a Raspberry Pi 5 or a Jetson Orin Nano better for local AI?#

For language models, the Jetson Orin Nano Super is several times faster on small models because it has a CUDA GPU: published third-party 3B results are roughly 22–43 tokens per second against about 5 on a Pi 5. The Pi 5 is cheaper, simpler and has more community guides.

Can a Mac mini run a local LLM?#

Yes. Apple Silicon Macs share memory between the CPU and GPU, so a 16 GB Mac mini runs 7–8B models at 4-bit quantization, and 24 GB or more fits 14–32B models.

Do I need a GPU to run AI at home?#

No. Small models run on CPUs, including the Raspberry Pi 5's. A GPU or Apple Silicon makes larger models fast enough for conversation.

Sources#

Want this without building a server?

pleeply is an early project: a speaker with a built-in computer that runs open models at home. There's no prototype yet, and your answers decide what we build first.

Answer 5 quick questions →30 seconds · only email is required · no payment

pleeply Community

Keep going with the community

Running AI at home is easier with people who've already done it. On the pleeply forum you can:

  • ask about local models, hardware and Home Assistant, and get answers from people running them;
  • vote on what pleeply should do first;
  • show your own local-AI setup and get feedback;
  • hear first about test builds and reservations.

Free · no email needed

This page is also available as Markdown. For the full docs index, see llms.txt.