GPUforLLM Logo

Local LLM VRAM Calculator

Text / LLM Image / Video

Inspect model weights and cache memory from Hugging Face or GGUF

Popular models first. Scroll or type a model name.

Select LLM Model

Checking for current models…

Tokens per sequence, including prompt and generated tokens. Cache allocation depends on the architecture.

Includes quantization block scales. Execution support depends on your llama.cpp build and device.

Multi-GPU allocation and compatibility are pending validation.

Context above is per sequence. This profile uses a unified cache; recurrent state is counted per sequence.

Optional. Tokens already in the conversation, not allocated cache capacity. Default: context window minus 128 tokens reserved for generation. Does not change the VRAM calculation.

An editable planning allowance for compute buffers and other runtime allocations. The initial 1 GiB is not a measured prediction or guaranteed margin.

Affects cache and compute buffers. Effective size is capped by the prompt batch limit and context. Flash Attention is assumed enabled.

Tokens per logical prompt batch. This is separate from concurrent requests.

For recurrent/hybrid architectures. Match the runtime setting; speculative decoding can require additional state.

Optional local storage only. Shared links take priority. Uncheck to forget the saved setup.

ℹ
Choose a model to get started

0.00 GiB
Memory planning estimate
Listed capacity: —

Hardware Guide & FAQ

Llama 4 uses mixed chunked and full attention. This calculator reads that structure and estimates its cache. Actual peak memory also depends on the artifact and runtime; 16 GiB capacity alone does not establish compatibility.
GGUF is inspected through its metadata and tensor tables. EXL2 and AWQ require different execution support and are outside this llama.cpp profile. A numeric BPW scenario does not validate those artifacts.
Generation speed depends on the runtime, quantization kernels and conversation length. A provisional range is available for the checked RTX 4090 dense Q4_K_M/F16 KV profile; its evidence and limits are in Advanced. Other setups show why speed cannot yet be estimated. Prompt processing, speculative decoding and partial offload are excluded.
This calculator does not certify hardware purchases. Memory capacity alone cannot establish whether a model will run under a particular runtime configuration.
Longer context increases full-attention cache. Sliding layers and recurrent state use different allocation rules. Compute buffers and actual peak GPU memory remain uncalibrated; contexts beyond the declared model limit are rejected.

Browse model memory guides: precision and context comparisons

Popular VRAM Calculations