Journal · · 4 min read
How much VRAM you actually need for a 27B or 70B model
Four numbers decide whether a local model runs well: weights, context cache, the driver, and what is left over. Here is the arithmetic, and which card clears it.
Most buying advice starts with the card. It is better to start with the arithmetic, because the card is only the answer.
Weights are roughly one to two bytes per parameter
Quantised weights scale with the number of parameters and the precision they are stored in. At 4 bits a parameter takes about half a byte, at 8 bits about one byte, and with the overhead of quantisation blocks the practical figure lands near one byte per parameter at 4-bit and two at 8-bit.
- 27–32B at 4-bit: about 18–20 GB
- 27–32B at 8-bit: about 32–34 GB
- 70B at 4-bit: about 40 GB
- 70B at 8-bit: about 72 GB
The context cache is the part people forget
The cache that keeps the conversation in the model grows with the context window and with how many requests run at once. At 32K tokens it adds roughly 2–4 GB on a 30B model and 5–8 GB on a 70B one; at 128K those numbers multiply. A card that fits the weights with nothing to spare will thrash the moment a long document arrives.
| Model | Precision | Weights | Cache at 32K | Comfortable card |
|---|---|---|---|---|
| 8B | 4-bit | ~5 GB | ~1–2 GB | 12–16 GB |
| 14B | 4-bit | ~8 GB | ~2–3 GB | 16–24 GB |
| 27–32B | 4-bit | ~18–20 GB | ~3–4 GB | 24 GB tight, 32–48 GB fine |
| 27–32B | 8-bit | ~32–34 GB | ~3–4 GB | 48 GB |
| 70B | 4-bit | ~40 GB | ~5–8 GB | 48 GB tight, 64 GB good |
| 70B | 8-bit | ~72 GB | ~5–8 GB | two 48 or 64 GB cards |
Leave about a gigabyte for the driver and the desktop. A 24 GB card running a 20 GB model is not a 24 GB card — it is a 20 GB card with 4 GB of headroom, which is exactly where 27B at 4-bit lives.
What that means for the cards we sell
- 64 GB on one card — a 70B at 4-bit fits with a long context and several parallel requests. The Instinct MI210 is the cheapest way to that number per gigabyte, and it is passive, so it wants airflow (see the AI hardware page).
- 48 GB on one card — a 70B at 4-bit fits at moderate context, and a 32B at 8-bit fits comfortably. The Radeon PRO W7900 does this in a normal tower case with its own blower and display outputs.
- 32 GB — 32B at 8-bit, or 70B at 4-bit only with a short context. The Radeon Pro V620 sits here and is the value pick if the model is 30B-class.
- Two smaller cards — 2 × 24 GB gives 48 GB of total memory but each layer has to fit on one card, so a 70B at 4-bit works only with tensor parallelism, and that needs a fast link between the cards. One card of the same total memory is simpler and quieter.
Used or new, and why it matters for the maths
A used data-centre card is usually the cheapest way to buy memory: the compute is older, the memory is exactly what the model needs. What changes is the support list — ROCm officially supports the Instinct MI200 series and RDNA 3, while older RDNA 2 and CDNA 1 parts run llama.cpp but fall off the vendor's list. Check the software row on each listing before buying, not after.
One more option: share the work with the processor
Not every model has to fit in VRAM. Mixture-of-experts models keep most of their weights idle for any given word, so an engine can hold the experts in system RAM and compute them on the CPU while the card does the rest. Strata does exactly this with Qwen3.8-Flash-Next and DeepSeek-V4.1-Flash: a 12 GB card with 64 GB of RAM answers at a few tokens per second, and a 24 GB card is noticeably faster because more experts stay on the GPU. That is the case for 64 GB of system memory in a workstation — what each card needs is here.
The short version
For a 27–32B model at 4-bit, 24 GB is enough and 32 GB is comfortable. For 70B at 4-bit, buy 48 GB if you keep context short and 64 GB if you do not. For 70B at 8-bit, plan two cards or a 128 GB module budget in system RAM. Everything else is a detail about how the memory is cooled and which software supports it.
Where this comes from
Every figure and claim above is taken from the pieces below. The links open the originals — worth reading if the details matter to you.