What the memory estimate can tell you

The calculator reads architecture metadata, then estimates weights, cache and a runtime budget. Reviewed native layouts follow a particular llama.cpp profile; a separately labelled analytical scenario is available for OPT. It runs entirely in your browser.

Choose the right kind of weight estimate

A model repository with a CONFIG gives a conversion scenario: parameter count × effective bits per weight. Quantized tensors do not all use the same storage type, so the usual Q4_K_M label does not imply one universal effective BPW.

For greater specificity, paste a GGUF repository or file link and choose the actual artifact. The reader pins its revision, checks every shard and sums the tensor payload using GGML block sizes. File headers and padding are excluded. CPU-held tensors, tied-weight duplication and backend repacking can still change GPU residency.

If Hugging Face omits a parameter total, the reader can count floating tensors from a pinned safetensors file or complete shard index. Only bounded headers are requested. Native packed quantization cannot use its stored element count as an unpacked parameter count; choose an actual GGUF artifact in that case.

The calculation profile

When only a weight scenario minimum is available: for reviewed dense/MoE structures, we can count known gate/up/down matrices from the config even when packed storage prevents a complete parameter count. All stored experts count. The main number applies your selected effective BPW to those known matrices; it excludes other weights and runtime memory. Red means that this assumed full-GPU weight scenario already exceeds capacity. A smaller number never establishes a fit. It is not the native mixed-quantized artifact size or a measured VRAM lower bound. Cache-only values appear in the breakdown, not as the main requirement.

OPT uses logical full-attention cache from its config: layers × context × sequences × hidden size × 4 bytes for 16-bit keys and values. When a floating tensor inventory is unavailable, its logical parameter count can be derived from the reviewed embedding, projection, MLP and normalization shapes, including biases and tied weights. The Transformers implementation was reviewed on September 22, 2026. This formula scenario adds the selected runtime budget and can be compared with capacity, but does not establish runtime or quantization compatibility. Native padding is not applied to this analytical cache.

Text generation, one device, full-residency planning and Flash Attention. Context is per sequence. The cache equations follow llama.cpp revision 6a1a922d269908a29cbd4b49c27e6a8e7fd10fae. This is not a certification that each listed device can execute that build or artifact.

• Full attention: allocated cells × stored key/value row bytes, summed over cache-owning layers. Unified context is padded to 256 cells.
• Sliding/chunked attention: the native window, sequence count and prompt micro-batch determine allocated cells. The native runtime may allocate full cache even when the HF config declares a sliding window.
• Gated delta hybrid models: F32 convolution state plus recurrent matrix state per sequence and rollback snapshot, plus the full-attention layers.
• MLA: modern compressed cache and legacy expanded GGUF layouts are distinguished. Config-only scenarios explicitly assume modern conversion.
• MoE: all expert weights count in a full-residency scenario, including experts that are inactive for a particular token.

Relevant implementations: cache dimensions, window allocation, recurrent allocation, tensor block storage.

When the runtime allowance is demonstrably too small

The effective micro-batch is the minimum of requested micro-batch, logical prompt batch and raw total context, following native initialization. Both batch settings are available in Advanced settings and preserved in shared links.

For ordinary full-KV attention on discrete CUDA with one sequence, one F16 attention-mask buffer alone needs 2 × padded context × effective micro-batch bytes. At 131,072 context and 8,192 effective micro-batch, this is 2 GiB. An allowance below that minimum now produces a partial result, without a total, fit or shopping recommendation. Weights and cache remain visible.

This is a minimum for one proven buffer, not sufficient total workspace. It follows the graph mask, scheduler copies and CUDA buffer support. The guard does not cover other backends, multiple sequences or specialized cache layouts. Their runtime budgets remain uncalibrated.

Cross-checks against published allocations

These are individual cache allocations recorded by users in upstream issue reports. They are useful independent arithmetic checks, with less provenance than a controlled benchmark. They do not measure whole-device peak VRAM.

The Llama report includes additional models. The Qwen 27B report includes CPU cache, disabled Flash Attention and a later crash. Both Qwen shared pools are normalized to the same total cells, not the same independent context allowance per slot. The Qwen3 MoE report automatically moves expert weights to CPU despite reporting every layer offloaded; its total is excluded. The hybrid MoE report later crashes during cache restoration. These seven component matches do not establish whole-device peak accuracy.

Where a close total can be misleading

Reviewed September 12, 2026. These comparisons use the log's rounded text parameter count and our standard conversion BPW, with a 1 GiB budget. The observed column sums GPU model, cache and compute buffers; it excludes host buffers and is not a measured device peak. Neither row is a matched artifact/runtime benchmark.

Gemma's historical build allocates 1,536 MiB of KV; our current windowed profile calculates 736 MiB (−52.08%). The larger runtime budget partly cancels that difference. The hybrid MoE artifact uses dynamic mixed quantization, so its GPU weight allocation differs from our generic Q3 scenario. Closeness of a combined total is not proof that its components are correct.

A total-memory comparison that does not calibrate this runtime

Qwen's official benchmark reports Qwen2.5-1.5B-Instruct BF16 with Transformers 4.46.0 on A100, one sequence and 2,048 generated tokens. Our different llama.cpp profile with a 1 GiB planning budget produces the following comparison:

*The source labels memory “GB” without specifying the divisor. The range shows both binary and decimal interpretations. Runtime, historical artifact identity and allocation measurement differ, so these rows are excluded from calibration. They demonstrate why a fixed allowance cannot guarantee total-memory accuracy.

Automatic models, explicit limits

The capacity indicator is green when selected capacity covers the planning estimate plus 10% headroom. Yellow means the estimate fits with less headroom, or exceeds capacity by at most 20%. Larger shortfalls are red. Unsupported, partial or unresolved inputs are neutral. These are explicit planning thresholds, not measured error bounds or execution guarantees. Recommendation confidence is at most MEDIUM while runtime memory remains uncalibrated; incomplete information lowers it to LOW.

Optimization preserves the selected weights/artifact, weight and KV precision, and runtime budget. It first reduces concurrent requests while keeping context, then finds the largest fitting context with one request. Smart Result reuses that result and checks at most three nearby shorter contexts, offering one recommendation and at most one alternative with more headroom. A shorter context holds fewer tokens and can affect long-document tasks. Settings change only when you apply the recommendation. Evidence, assumptions and existing memory options remain in Advanced details; CPU offload and speed are not verified.

For red results, hardware and cloud suggestions use the current settings plus 10% shopping headroom. The smallest qualifying option in a short capacity-ranked list is shown; it is not a price/performance ranking or tested compatibility. Vendor GB labels are conservatively converted as decimal bytes before comparison with GiB. Confirm the exact memory variant, free device memory, runtime support and current availability before buying or renting. The margin is not a measured accuracy bound.

Desktop capacities come from NVIDIA's RTX 3060, 4060 Ti, 4090, 5090 and RTX 6000 Ada specifications. Named cloud examples follow the RunPod GPU catalog; the Vultr link opens its own inventory, which can differ. This shortlist was checked on September 11, 2026. Prices, stock and affiliate listing variants are not verified live.

Public Hugging Face discovery refreshes in the browser on visits and when returning to a tab, using a four-hour cache. Search and direct links cover models outside the initial list. Metadata is pinned and validated on selection; a newly released checkpoint using known semantics needs no manual catalogue edit.

DeepSeek V3.2 and GLM DSA now expose compressed MLA and indexer cache components for the single-sequence F16 CUDA profile. GPT-OSS exposes its alternating sliding/full cache. Packed native weights remain unresolved until an actual artifact is inspected; known cache can still be displayed. Original Qwen, GPT2 and GPT-NeoX use reviewed native full-cache layouts. These additions follow runtime source; they are not new measured peak benchmarks.

Unknown architectures and unreviewed cache features remain pending. GLM5-Next, other unknown indexed-cache variants, Bamba, Zamba2 and recursive execution currently need additional mappings. DeepSeek-V4 has partial component accounting as described above. An unknown architecture can still show a weight-storage scenario if reported floating tensor counts are internally consistent, or inspected GGUF payload; neither produces a total or fit. A metadata-loaded message distinguishes calculation limits from repository-access failures. Multi-GPU placement, partial offload, image/audio execution, speculative decoding, adapters and other runtime implementations are not certified.

A refresh failure preserves saved choices. Private or gated files may be inaccessible to anonymous browser requests. Metadata reads have timeouts and size limits; GGUF parsing runs in a worker and reads bounded prefixes that may include a small amount of tensor data at the header boundary.

Models can update automatically; new mathematical mechanisms cannot safely invent their own implementation. Those still require an engine update and a new static upload.

Before any stronger accuracy claim

Measure the exact GGUF revision on a pinned llama.cpp CUDA build, single GPU and full offload. Record model, cache and compute buffers separately, plus device usage before load, after load, during prefill and decode. Vary context, micro-batch, sequence count and cache precision; test unseen checkpoints and report median, p95 and worst underestimation. Match the runtime configuration before reusing any result.

Current compatible, independently measured full-peak calibration samples: 0. Download the evidence data.