Fit a model to your GPU before you download it.
How to use VRAM Calculator
Enter the model size. Type the parameter count in billions into Model size (billions of parameters), for example 7 for a 7B model.
Pick a weight quantization. Choose Weight quantization: FP16 / BF16, INT8, AWQ INT4, GGUF Q8_0, GGUF Q5_K_M or GGUF Q4_K_M.
Fill in the architecture. Enter Layers, Hidden size and Attention heads from the model's config. Choose MHA, GQA or MQA under Attention type, and for GQA enter the KV heads.
Set context, batch and KV dtype. Enter Context length (tokens) and Batch size, then choose a KV cache dtype of FP16 or INT8.
Adjust the overhead margin and read the result. Set Overhead margin (%) and read the weights, KV cache, overhead and total figures, plus which common GPU sizes the total fits under.
Limits and privacy
- Every number is an estimate. The total is weights plus KV cache plus an overhead margin you set (default 10%) applied to those two.
- Weights use fixed bytes per parameter: 2 for FP16, 1 for INT8, 0.55 for AWQ INT4, and approximate averages for GGUF Q8_0 (1.0625), Q5_K_M (0.6875) and Q4_K_M (0.5625).
- KV cache is 2 x batch x context x layers x KV heads x head dimension x bytes per element, where bytes is 2 for FP16 or 1 for INT8. Head dimension is hidden size divided by attention heads.
- MQA is treated as one KV head. GQA uses your KV head count, capped at the number of attention heads.
- The fit check compares the total against five sizes only: 8, 12, 24, 48 and 80 GB.
- Activations, CUDA context, fragmentation and framework behavior are not modeled beyond the overhead percentage, so real usage can differ.
- Everything is calculated in your browser from the numbers you type. Nothing is uploaded.
Frequently asked questions
How much VRAM do I need to run a 7B model?
Enter 7 as the model size, choose your quantization and context length, and the tool shows the weights, KV cache and total. As a reference, 7 billion parameters at FP16 is about 13 GB of weights before KV cache and overhead.
What does the VRAM estimate include?
It adds model weights, the KV cache for your batch size and context length, and an overhead margin you control. It does not model activations or framework internals separately.
Why does context length change the VRAM so much?
The KV cache grows linearly with context length and batch size. A longer context or larger batch stores more key and value tensors for every layer.
Where do I find layers, hidden size and heads?
They are in the model's config.json on its repository page, under fields such as num_hidden_layers, hidden_size, num_attention_heads and num_key_value_heads.