Mixture-of-experts hosting
Why these models need the memory of a big model but compute like a small one.
- From the articleMiniMax M3: an open frontier model you can run yourself?
- Help with thisLocal and private AI for sensitive data
Free, runs in your browser. Nothing you enter is sent anywhere unless you copy the link or send it yourself.
All toolsTry it
Dominik Heinrich · dominikheinrich.ch
Mixture-of-experts hosting
Created on Nov 9, 2026
Interactive
What does a mixture-of-experts model cost to host?
All experts must sit in GPU memory, but each token only runs through the active ones. Memory scales with total parameters, speed with active parameters.
Precision
≈ 428 GB
weights in GPU memory
7
80 GB GPUs, minimum
23B
compute per token, like a dense model this size
Planning estimate: weights plus about 15% for KV cache and runtime. Real throughput depends on batch size, interconnect and the serving engine.
Get in touch
Tell me what you're building. I'll reply within 1–2 business days.
- Company
- MADO Consulting
- Follow me on LinkedIn
Available worldwide, remote. On-site on request, depending on the country.
Opens Calendly's booking calendar here. Loading it sends data to Calendly (see Impressum).