VRAM Calculator: Will This Model Fit Your GPU?

A working calculator for LLM VRAM: enter parameters, quant bits, layers, KV heads, head dim, and context to see weight + KV cache + headroom totals.

VRAM Calculator: Will This Model Fit Your GPU?

This page runs a local LLM memory estimator directly in your browser. Enter the model’s shape and your card’s memory, and it returns the weight size, the KV cache, an engine-overhead estimate, and the grand total compared against what you have. Nothing leaves the page; there is no network call and no external library.

The formulas

The calculator applies three equations. They are printed here so you can check the math yourself.

1. Weight memory (GB) = parameters_in_billions × weight_bits ÷ 8
2. KV cache (GB) = 2 × layers × kv_heads × head_dim × context × (kv_bits ÷ 8) ÷ 1,000,000,000
3. Total (GB) = weight_memory + kv_cache + engine_headroom

The factor of 2 in the KV cache exists because the model stores both the K (key) and V (value) tensors. Each ÷ 8 converts bits to bytes.

Weight precision and cache precision are separate inputs on purpose, and this is where most online estimates go wrong. Quantizing the weights to 4 bits does not quantize the cache: most engines keep the KV cache at 16-bit regardless of the weight format, so a Q4_K_M model still carries a full-precision cache that can be several times larger than the weights at long context. Leave the cache field at 16 unless you have deliberately enabled a KV cache quantization option and confirmed its bit depth in your engine’s settings.

The calculator

Worked example 1: a 7B model at Q4_K_M on a 24 GB card

Enter params=7, bits=4.5, kvbits=16, layers=32, kvheads=8, headdim=128, ctx=4096, vram=24, over=1.5.

• Weight memory = 7 × 4.5 ÷ 8 = 3.94 GB
• Cache per token = 2 × 32 × 8 × 128 × 2 bytes = 131,072 bytes = 128 KB per token
• KV cache = 131,072 × 4,096 = 536,870,912 bytes = 0.537 GB
• Total = 3.94 + 0.537 + 1.5 = 5.97 GB → FITS in 24 GB with room to spare.

(Each line above is rounded for display; the tool adds the unrounded values, so a total can differ from the sum of the printed lines by a hundredth. That is display rounding, not an error.)

That 128 KB per token figure is worth memorising, because it turns any context length into an instant estimate: 32K context is 128 KB × 32,768, or about 4 GB, for a model whose weights are under 4 GB.

Worked example 2: a 70B model at Q4_K_M

Enter params=70, bits=4.5, kvbits=16, layers=80, kvheads=8, headdim=128, ctx=4096, vram=24, over=1.5.

• Weight memory = 70 × 4.5 ÷ 8 = 39.38 GB
• Cache per token = 2 × 80 × 8 × 128 × 2 bytes = 327,680 bytes = 320 KB per token
• KV cache = 327,680 × 4,096 = 1,342,177,280 bytes = 1.342 GB
• Total = 39.38 + 1.342 + 1.5 = 42.22 GB → does not fit in a single 24 GB card. You need a multi-GPU setup or far more unified memory; read the 70B local rundown for the realistic combinations.

Worked example 3: long context turns the cache into the biggest line item

Keep params=7, bits=4.5, kvbits=16, layers=32, kvheads=8, headdim=128 but raise ctx=131072 on a 12 GB card with over=1.5.

• Weight memory stays 3.94 GB
• KV cache = 131,072 × 131,072 = 17,179,869,184 bytes = 17.18 GB
• Total = 3.94 + 17.18 + 1.5 = 22.62 GB → does not fit in 12 GB. It does not fit in 24 GB either.

This is the case that catches people out. At 128K context the cache is more than four times the size of the weights, so shrinking the quantization from 4-bit to 3-bit changes almost nothing while cutting context in half changes everything. This is exactly why context length costs VRAM as a separate budget line, and why the calculator prints the per-token figure alongside the total.

What this tool does NOT do

• It does not predict tokens per second. Speed depends on the compiler, drivers, batch size, and how many layers you offload, none of which this page models.
• It assumes the cache grows linearly with every token and that no attention optimisation is in play. Architectures that use sliding-window attention, a compressed or latent cache, or sparse attention hold less than the formula says. If your model uses one of those, the cache figure here is an upper bound.
• It does not model batching. Running several conversations at once multiplies KV cache requirements as a separate budget line.
• It uses nominal available VRAM. The number printed on the box is not the number the model gets; macOS, Windows, and the driver keep part of it, so subtract a reserve before trusting the figure.

Common input mistakes

• Entering parameters in millions instead of billions (7000 instead of 7) inflates weight memory a thousandfold.
• Leaving GQA KV heads equal to the full attention-head count overstates the KV cache on modern models, which use far fewer KV heads.
• Forgetting that a 32 GB card does not give the model 32 GB. Subtract the system and engine reserve first.
• Using the wrong quant bits: Q4_K_M is about 4.5 bits per parameter, not 4.0. The quantization comparison lists the common approximations.

Use the verdict as a planning signal. If the total fits with a few gigabytes to spare, you are in good shape. If it is within a gigabyte of your card, expect swapping or out-of-memory errors under load.