LLM VRAM Calculator
A rough estimate of the GPU memory needed for inference, based on model size, quantization, and context length. Real usage varies by architecture and runtime — treat this as a ballpark, not a guarantee.
Bytes stored per parameter.
Activations, framework, and fragmentation headroom.
Estimated VRAM neededWeights + KV cache + overhead.
—Model weights alone
—FAQ
- Why does quantization reduce VRAM?
Lower-bit quantization stores each weight in fewer bytes — 4-bit uses roughly a quarter of the memory of 16-bit, at some cost to quality.
- Is this exact?
No. KV cache size in particular depends on the model's layer count and hidden size. This is a first-order estimate for planning.