gpt-oss 20B memory requirements

21.5B parameters — 12.8 GiB of weights at FP16, 10.8 GiB at Q4_K_M, and 48.0 KiB of KV cache per token on top. Solved against 22 real accelerators; 21 of them can hold it.

Open the calculator →All models

Parameters

21.5B

21,511,953,984 exactly

Weights, Q4_K_M

10.8 GiB

12.8 GiB unquantized

KV cache

48.0 KiB

per token · 24L × 8 KV × 64

Smallest card (Q4)

12 GB

GeForce RTX 3060 12GB

Weights at each precision

Precision	Bits/weight	Weights	Source	Notes
As released	5.12	12.8 GiB	measured file	Unquantized. Two bytes per parameter, exactly.
Q8_0	4.50	11.3 GiB	measured file	Effectively lossless; the usual reference point for quantized quality.
Q6_K	4.48	11.2 GiB	measured file	Quality loss is hard to measure on most benchmarks.
Q5_K_M	4.36	10.9 GiB	measured file	A middle point when Q4 fits with too little room for context.
Q4_K_M	4.32	10.8 GiB	measured file	The default choice for local inference — the best size/quality knee.
Q3_K_M	4.28	10.7 GiB	measured file	Measurable quality loss. Worth it only to make a model fit at all.

This release is already quantized. The published checkpoint measures 12.8 GiB against the 40.1 GiB that two bytes per parameter would imply, so the FP16 row above describes a file its authors never shipped. The measurement is the sum of the shard sizes in openai/gpt-oss-20b, not a figure derived from the dtype metadata — which reports this model's 4-bit tensors in a way that cannot be told apart from 8-bit ones.

Every accelerator

Largest quant that fits with at least 4k of context, the context it leaves, and the bandwidth ceiling on decode speed. Each row links to the worked page for that pairing.

Accelerator	Memory	Bandwidth	Best quant	Max context	Ceiling
Apple M3 Ultra (512GB)	512 GB	819 GB/s	FP16 / BF16	131,072	277 tok/s
H200 141GB (SXM)	141 GB	4800 GB/s	FP16 / BF16	131,072	1622 tok/s
Apple M4 Max (128GB)	128 GB	546 GB/s	FP16 / BF16	131,072	184 tok/s
H100 80GB (SXM5)	80 GB	3350 GB/s	FP16 / BF16	131,072	1132 tok/s
A100 80GB (SXM)	80 GB	2039 GB/s	FP16 / BF16	131,072	689 tok/s
Jetson AGX Orin (64GB)	64 GB	204.8 GB/s	FP16 / BF16	131,072	69 tok/s
RTX 6000 Ada Generation	48 GB	960 GB/s	FP16 / BF16	131,072	324 tok/s
L40S	48 GB	864 GB/s	FP16 / BF16	131,072	292 tok/s
Apple M4 Pro (48GB)	48 GB	273 GB/s	FP16 / BF16	131,072	92 tok/s
A100 40GB (SXM)	40 GB	1555 GB/s	FP16 / BF16	131,072	525 tok/s
GeForce RTX 5090	32 GB	1792 GB/s	FP16 / BF16	131,072	605 tok/s
GeForce RTX 4090	24 GB	1008 GB/s	FP16 / BF16	131,072	341 tok/s
Radeon RX 7900 XTX	24 GB	960 GB/s	FP16 / BF16	131,072	324 tok/s
GeForce RTX 3090	24 GB	936 GB/s	FP16 / BF16	131,072	316 tok/s
Apple M4 (24GB)	24 GB	120 GB/s	FP16 / BF16	131,072	41 tok/s
GeForce RTX 5080	16 GB	960 GB/s	FP16 / BF16	83,049	324 tok/s
GeForce RTX 5070 Ti	16 GB	896 GB/s	FP16 / BF16	83,049	303 tok/s
GeForce RTX 4080 SUPER	16 GB	736 GB/s	FP16 / BF16	83,049	249 tok/s
GeForce RTX 4070 Ti SUPER	16 GB	672 GB/s	FP16 / BF16	83,049	227 tok/s
GeForce RTX 4060 Ti 16GB	16 GB	288 GB/s	FP16 / BF16	83,049	97 tok/s
GeForce RTX 3060 12GB	12 GB	360 GB/s	Q5_K_M	5,436	145 tok/s
Jetson Orin Nano Super (8GB)	8 GB	102 GB/s	no fit	—	—

Questions about this model

How much VRAM does gpt-oss 20B need?

12.8 GiB for the weights at FP16 and 10.8 GiB at Q4_K_M, from an exact count of 21,511,953,984 parameters. On top of that, the KV cache costs 48.0 KiB per token of context — 195 MiB at 8k and 771 MiB at 32k, less than linear because 12 of 24 layers stop growing at a 128-token window.

What is the smallest GPU that runs gpt-oss 20B?

Of the 22 accelerators here, the smallest that holds Q4_K_M weights is the GeForce RTX 3060 12GB at 12 GB — 10.8 GiB of weights against 11.0 GiB usable, leaving room for 9k tokens. For unquantized weights you need at least a GeForce RTX 5080.

Why is the KV cache for gpt-oss 20B the size it is?

Because it stores one key and one value vector per token, per layer: 24 layers × 8 KV heads × 64 dimensions × 2 (K and V) × 2 bytes = 48.0 KiB per token. Grouped-query attention shares those 8 KV heads across 64 query heads, cutting the cache by 8× against multi-head attention. None of this can be read off the parameter count.

Compare with

Method and limits. The parameter count and every architecture figure on this page are read from the model's own published config and the Hub's index over its tensor shapes, fetched 2026-08-07 — nothing here is recalled from a model's name. Weight bytes are the size of a real published file wherever one exists and the parameter count times the published llama.cpp bits-per-weight where it does not; every row says which. Speed figures are roofline bounds — memory bandwidth divided by bytes read per token — not benchmarks. Nothing on this page is written by a language model.