GPU memory for a local LLM
Which GPU, and how many, a model needs before you buy hardware.
- From the articleI want to run a local LLM: which server should I buy?
- Help with thisLocal and private AI for sensitive data
Free, runs in your browser. Nothing you enter is sent anywhere unless you copy the link or send it yourself.
All toolsTry it
Dominik Heinrich · dominikheinrich.ch
GPU memory for a local LLM
Created on Nov 9, 2026
Interactive
Which GPU does your model need?
Pick a model size and precision. Weights take parameters × bytes per parameter; about 20% on top covers the KV cache for normal context lengths.
Model size (billion parameters)
Precision
≈ 19 GB
GPU memory needed
1×
fits on 24 GB (RTX 4090)
- 1× 24 GB (RTX 4090)
- 1× 32 GB (RTX 5090)
- 1× 48 GB (L40S)
- 1× 80 GB (H100)
- 1× 96 GB (RTX PRO 6000 Blackwell)
- 1× 141 GB (H200)
Rule of thumb for planning, not a benchmark. Long contexts and many parallel users need more KV cache; 4-bit costs a little quality.
Get in touch
Tell me what you're building. I'll reply within 1–2 business days.
- Company
- MADO Consulting
- Follow me on LinkedIn
Available worldwide, remote. On-site on request, depending on the country.
Opens Calendly's booking calendar here. Loading it sends data to Calendly (see Impressum).