Lab · On-device AI · LLM VRAM
ComputedWill it fit?
Weight bytes, KV-cache growth and the bandwidth ceiling on decode speed, worked out for 62 open models against 22 real accelerators. Every architecture figure is read from the model's own published config; nothing is estimated and nothing is written by a language model.
Pick a pairing
62 models × 22 accelerators. Every combination below is a page that already exists and is already solved — the panel names the exact pairing it resolved to, never a nearby one.
Resolved pairing
DeepSeek-R1-Distill-Llama 8B on GeForce RTX 4090
Comfortable
The unquantized weights fit and leave room for a long context.
- Quantization
- FP16 / BF16
- largest that fits with usable context
- Weights
- 15.0 GiB
- of 22.08 GiB usable
- Max context
- 58,347 tokens
- Decode ceiling
- 59 tok/s
The fit map
All 62 models against the selected accelerator. Horizontal is the weight bytes of the quantization each model is drawn at; vertical is the context those weights leave room for. Both axes are logarithmic — this set spans 0.09 GiB to 557 GiB of weights. The vertical rule is the memory a runtime can actually allocate on that device; the horizontal rule is the 4,096-token floor this lane counts as usable context.
ComfortableFitsQuantized onlyToo tightNo fit
A dot's colour is the verdict the pair page prints, never which side of a rule it landed on. The two disagree on purpose: the usable-context floor is capped at each model's own maximum position count, so a model whose entire window is 2,048 tokens is a complete fit while sitting below the 4,096-token line. Models with no usable fit are drawn at their smallest quantization — the leftmost point they can reach — which is why they cluster against the vertical rule.
The short answer, by size
Largest quantization that fits with at least 4k of context. A model that needs more than a card holds is not unusable — it splits across devices or spills into host RAM — but the speed cost of doing that is severe, and each pair page states it.
| Model | Q4_K_M | RTX 3060 12GB 12 GB | RTX 4060 Ti 16GB 16 GB | RTX 4090 24 GB | RTX 5090 32 GB | M4 Max (128GB) 128 GB | H100 80GB (SXM5) 80 GB |
|---|---|---|---|---|---|---|---|
| Gemma 3 1B Instruct | 576 MiB | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 |
| Qwen2.5 3B Instruct | 1.8 GiB | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 |
| DeepSeek-R1-Distill-Llama 8B | 4.6 GiB | Q8_0 | Q8_0 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 |
| OLMo 2 13B Instruct | 7.8 GiB | Q4_K_M | Q6_K | Q8_0 | FP16 / BF16 | FP16 / BF16 | FP16 / BF16 |
| Qwen3 32B | 18.4 GiB | no fit | no fit | Q4_K_M | Q6_K | FP16 / BF16 | FP16 / BF16 |
| DeepSeek-R1-Distill-Llama 70B | 39.6 GiB | no fit | no fit | no fit | no fit | Q8_0 | Q8_0 |
Every accelerator
Every model
62 models across 20 families, each solved against all 22 accelerators — 1,448 computed pages in total. That includes one model view and one GPU view for each axis, plus every solved pairing. Open a family to browse it directly.
Current-model discovery comes from a ranked Hugging Face text-generation snapshot captured August 7, 2026; previously published model URLs remain stable when the ranking moves.
Qwen17 models
- Qwen2.5 72B Instruct44.2 GiB
- Qwen2.5 32B Instruct18.5 GiB
- Qwen2.5-Coder 32B Instruct18.5 GiB
- QwQ 32B18.4 GiB
- Qwen3 32B18.4 GiB
- Qwen3 30B-A3B17.3 GiB
- Qwen2.5 14B Instruct8.4 GiB
- Qwen3 14B8.4 GiB
- Qwen3 8B4.7 GiB
- Qwen2.5 7B Instruct4.4 GiB
- Qwen2.5-Coder 7B Instruct4.4 GiB
- Qwen3 4B2.3 GiB
- Qwen2.5 3B Instruct1.8 GiB
- Qwen3 1.7B1.1 GiB
- Qwen2.5 1.5B Instruct940 MiB
- Qwen3 0.6B433 MiB
- Qwen2.5 0.5B Instruct379 MiB
DeepSeek6 models
Gemma6 models
Llama6 models
Mistral5 models
gpt-oss2 models
Laguna2 models
Qwen3 5 MOE Text2 models
Falcon1 model
Granite1 model
HY V31 model
Nanbeige1 model
Nemotron1 model
OLMo1 model
Qwen3 5 Text1 model
Seed OSS1 model
TinyLlama1 model
Yi1 model
How these are computed
Weights
The byte size of the real published GGUF file, wherever one exists — measured, not multiplied out. Where none does, the exact parameter count from the Hub's index over the checkpoint's tensor shapes times the published llama.cpp bits-per-weight. Not "7B" — 7,615,616,512. Every row says which basis it used, because the nominal figure is sound in the middle and wrong at both ends.
KV cache
2 · layers · kv_heads · head_dim bytes per token per element, summed over layers with each sliding-window layer capped at its window. It scales with key/value heads, which is why parameter count predicts it badly: 59 of these 62 models use grouped-query attention and the rest do not.
Speed ceiling
Decode is memory-bound, so tok/s ≤ bandwidth ÷ bytes read per token. It is a bound, not a benchmark — real runtimes reach roughly 60–80% of it and none exceed it. For mixture-of-expert models only the routed experts are read, and that share is derived from the config rather than assumed.
Model data fetched from Hugging Face on 2026-08-07 (scripts/llm/fetch-model-configs.mjs); accelerator capacity and bandwidth come from vendor specifications, and every bandwidth figure that derives from a memory bus is checked against the bus width and data rate that produce it. The one assumption applied throughout is how much of a device's memory a runtime can actually allocate; it is printed on every page that uses it. The picker and the fit map read solved answers computed at build time — the browser positions and looks up, and does no arithmetic of its own.