GPU/VRAM Calculator for AI
< ToolsFind a GPU for Llama 4, Qwen 3.7, DeepSeek V4, GLM-5.2 and image/video models — or check whether your card has enough VRAM. Accounts for quantization, context length, batch, and light fine-tuning. Linked with the AI catalog and selector.
Pick a model first.
Estimate ±15–20%. Real usage depends on runtime (llama.cpp, vLLM, Ollama) and drivers. Data verified: 2026-07-23.
Popular model × quant × GPU combos
Indexed starter examples. Numbers come from the calculator formula (inference, batch 1).
GPU/model dataset verified: 2026-07-23
| Model | Quant | Context | VRAM | Min GPU | |
|---|---|---|---|---|---|
| Llama 3.1 8B | GGUF Q4 | 8.192k | 7.4 GB | RTX 3080 10GB (10 GB) | Open |
| Mistral 7B | GGUF Q4 | 8.192k | 6.8 GB | RTX 4060 (8 GB) | Open |
| Llama 3.1 70B | INT4 / GPTQ / AWQ | 4.096k | 41.4 GB | A100 80GB (80 GB) | Open |
| Llama 4 Maverick | GGUF Q4 | 8.192k | 64.5 GB | A100 80GB (80 GB) | Open |
| Qwen 3.7 | GGUF Q4 | 8.192k | 23 GB | RTX 5090 (32 GB) | Open |
| GLM-5.2 | INT4 / GPTQ / AWQ | 8.192k | 59.1 GB | A100 80GB (80 GB) | Open |
| FLUX.1 Dev | FP16 (full) | 2.048k | 27.2 GB | RTX 5090 (32 GB) | Open |
VRAM & local LLM FAQ
- How much VRAM do I need for a 70B model?
- In FP16 a 70B model needs roughly 140 GB just for weights. With INT4 / AWQ / GPTQ you can often land near 35–50 GB at a moderate context (4k–8k), so cards like A100 40GB (tight) or 48–80 GB class GPUs are realistic. Use the calculator with context length enabled — KV-cache grows with tokens.
- Q4 vs INT4 — what is the difference?
- Both target ~4-bit weights. INT4 usually means GPTQ/AWQ in CUDA stacks (vLLM, transformers). GGUF Q4 is the llama.cpp / Ollama family format, great for consumer GPUs and Apple Silicon. Quality and exact VRAM differ by runtime; Q4_K variants often need slightly more than a raw 0.5 byte/param estimate.
- Is 8 GB of VRAM enough for local LLMs?
- Yes for smaller models: 7–8B in GGUF Q4 at 2k–8k context typically fits on 8 GB with limited headroom. 13–14B Q4 may be tight. 70B will not fit on 8 GB without heavy offloading. Image models like FLUX.1 Dev usually want 12–24 GB in FP16.
- Why does longer context need more VRAM?
- KV-cache scales with sequence length × layers × KV heads. Doubling context roughly doubles the KV portion even if weights stay the same. That is why 8B Q4 can jump from ~5–6 GB at 2k to noticeably more at 32k–128k.
- How accurate is this calculator?
- Treat results as ±15–20%. Real usage depends on runtime (llama.cpp, vLLM, Ollama), CUDA graphs, KV quantization, and driver overhead. Prefer a card with comfortable headroom rather than a perfect tight fit.