Lab · LLM VRAM · NVIDIA
ComputedWhat LLMs can the GeForce RTX 4060 Ti 16GB run?
43 of 62 rostered models have a usable on-device fit. The table keeps all 62 visible, including 1 below-context fits and 18 that exceed this device, so “not listed” never masquerades as an answer.
Published memory
16 GB
GDDR6
Assumed usable
14.7 GiB
92% of capacity
Peak bandwidth
288 GB/s
theoretical device spec
Usable fits
43 / 62
largest: 30.5B at Q3_K_M
Every model, solved on this accelerator
Usable rows fit weights plus at least 4k tokens, capped at the model’s own trained window. Tight rows hold weights but fall below that floor. The speed column is a memory-bandwidth ceiling, not a benchmark.
43 usable · 1 tight · 18 no fit
| Model | Parameters | On-device result | Selected weights | Max context | Decode ceiling · read per token |
|---|---|---|---|---|---|
| Qwen3 30B-A3BQwen | 30.5B | Quantized fit | Q3_K_M · 13.7 GiBpublished file | 11k | 119 tok/sreads 2.3 GiB/token |
| Gemma 3 27B InstructGemma | 27.4B | Quantized fit | Q3_K_M · 12.5 GiBarchitecture calculation | 23k | 20 tok/sreads 13.5 GiB/token |
| Gemma 2 27B InstructGemma | 27.2B | Quantized fit | Q3_K_M · 12.5 GiBpublished file | 6k | 18 tok/sreads 14.7 GiB/token |
| Dolphin Mistral 24B Venice EditionMistral | 24B | Quantized fit | Q4_K_M · 13.5 GiBarchitecture calculation | 8k | 18 tok/sreads 14.7 GiB/token |
| Mistral Small 24B InstructMistral | 23.6B | Quantized fit | Q4_K_M · 13.3 GiBpublished file | 9k | 18 tok/sreads 14.6 GiB/token |
| gpt-oss 20Bgpt-oss | 21.5B | Full precision · comfortable | As released · 12.8 GiBpublished file | 81k | 97 tok/sreads 2.8 GiB/token |
| DeepSeek-R1-Distill-Qwen 14BDeepSeek | 14.8B | Quantized fit | Q6_K · 11.3 GiBpublished file | 18k | 21 tok/sreads 12.8 GiB/token |
| Qwen2.5 14B InstructQwen | 14.8B | Quantized fit | Q6_K · 11.3 GiBpublished file | 18k | 21 tok/sreads 12.8 GiB/token |
| Qwen3 14BQwen | 14.8B | Quantized fit | Q6_K · 11.3 GiBpublished file | 22k | 21 tok/sreads 12.5 GiB/token |
| Phi-4 14BPhi | 14.7B | Quantized fit | Q6_K · 11.2 GiBpublished file | 16k | 21 tok/sreads 12.8 GiB/token |
| OLMo 2 13B InstructOLMo | 13.7B | Quantized fit | Q6_K · 10.5 GiBpublished file | 4k | 20 tok/sreads 13.6 GiB/token |
| Mistral Nemo 12B InstructMistral | 12.2B | Quantized fit | Q8_0 · 12.1 GiBpublished file | 17k | 20 tok/sreads 13.4 GiB/token |
| Gemma 3 12B InstructGemma | 12.2B | Quantized fit | Q8_0 · 12.1 GiBarchitecture calculation | 38k | 21 tok/sreads 12.9 GiB/token |
| Qwythos 9B Claude Mythos 5 1MQwen3 5 Text | 9.4B | Quantized fit | Q8_0 · 9.3 GiBarchitecture calculation | 43k | 26 tok/sreads 10.3 GiB/token |
| Gemma 2 9B InstructGemma | 9.2B | Quantized fit | Q8_0 · 9.2 GiBpublished file | 8k | 23 tok/sreads 11.8 GiB/token |
| Qwen3 8BQwen | 8.2B | Quantized fit | Q8_0 · 8.1 GiBpublished file | 40k | 29 tok/sreads 9.2 GiB/token |
| Granite 3.3 8B InstructGranite | 8.2B | Quantized fit | Q8_0 · 8.1 GiBarchitecture calculation | 42k | 29 tok/sreads 9.3 GiB/token |
| DeepSeek-R1-Distill-Llama 8BDeepSeek | 8B | Quantized fit | Q8_0 · 8.0 GiBpublished file | 54k | 30 tok/sreads 9.0 GiB/token |
| Llama 3.1 8B InstructLlama | 8B | Quantized fit | Q8_0 · 8.0 GiBpublished file | 54k | 30 tok/sreads 9.0 GiB/token |
| DeepSeek-R1-Distill-Qwen 7BDeepSeek | 7.6B | Full precision · limited context | FP16 / BF16 · 14.2 GiBpublished file | 10k | 18 tok/sreads 14.6 GiB/token |
| Qwen2.5 7B InstructQwen | 7.6B | Full precision · limited context | FP16 / BF16 · 14.2 GiBpublished file | 10k | 18 tok/sreads 14.6 GiB/token |
| Qwen2.5-Coder 7B InstructQwen | 7.6B | Full precision · limited context | FP16 / BF16 · 14.2 GiBpublished file | 10k | 18 tok/sreads 14.6 GiB/token |
| Falcon3 7B InstructFalcon | 7.5B | Full precision · limited context | FP16 / BF16 · 13.9 GiBpublished file | 8k | 18 tok/sreads 14.7 GiB/token |
| Mistral 7B Instruct v0.3Mistral | 7.2B | Full precision · limited context | FP16 / BF16 · 13.5 GiBpublished file | 10k | 18 tok/sreads 14.5 GiB/token |
| Gemma 3 4B InstructGemma | 4.3B | Full precision · comfortable | FP16 / BF16 · 8.0 GiBpublished file | 128k | 32 tok/sreads 8.3 GiB/token |
| Nanbeige4.2 3BNanbeige | 4.2B | Full precision · comfortable | FP16 / BF16 · 7.8 GiBpublished file | 81k | 32 tok/sreads 8.5 GiB/token |
| Qwen3 4BQwen | 4B | Full precision · comfortable | FP16 / BF16 · 7.5 GiBpublished file | 40k | 31 tok/sreads 8.6 GiB/token |
| Phi-4-mini 3.8B InstructPhi | 3.8B | Full precision · comfortable | FP16 / BF16 · 7.1 GiBpublished file | 61k | 33 tok/sreads 8.1 GiB/token |
| Phi-3.5-mini 3.8B InstructPhi | 3.8B | Full precision · limited context | FP16 / BF16 · 7.1 GiBpublished file | 20k | 27 tok/sreads 10.1 GiB/token |
| Llama 3.2 3B InstructLlama | 3.2B | Full precision · comfortable | FP16 / BF16 · 6.0 GiBpublished file | 80k | 39 tok/sreads 6.9 GiB/token |
| Qwen2.5 3B InstructQwen | 3.1B | Full precision · comfortable | FP16 / BF16 · 5.7 GiBpublished file | 32k | 44 tok/sreads 6.0 GiB/token |
| Qwen3 1.7BQwen | 2B | Full precision · comfortable | FP16 / BF16 · 3.8 GiBpublished file | 40k | 58 tok/sreads 4.7 GiB/token |
| DeepSeek-R1-Distill-Qwen 1.5BDeepSeek | 1.8B | Full precision · comfortable | FP16 / BF16 · 3.3 GiBpublished file | 128k | 76 tok/sreads 3.5 GiB/token |
| SmolLM2 1.7B InstructSmolLM | 1.7B | Full precision · comfortable | FP16 / BF16 · 3.2 GiBpublished file | 8k | 57 tok/sreads 4.7 GiB/token |
| Qwen2.5 1.5B InstructQwen | 1.5B | Full precision · comfortable | FP16 / BF16 · 2.9 GiBpublished file | 32k | 87 tok/sreads 3.1 GiB/token |
| Llama 3.2 1B InstructLlama | 1.2B | Full precision · comfortable | FP16 / BF16 · 2.3 GiBpublished file | 128k | 105 tok/sreads 2.6 GiB/token |
| TinyLlama 1.1B ChatTinyLlama | 1.1B | Full precision · comfortable | FP16 / BF16 · 2.0 GiBpublished file | 2k | 128 tok/sreads 2.1 GiB/token |
| MiniCPM5 1BLlama | 1.1B | Full precision · comfortable | FP16 / BF16 · 2.0 GiBpublished file | 128k | 122 tok/sreads 2.2 GiB/token |
| Gemma 3 1B InstructGemma | 1000M | Full precision · comfortable | FP16 / BF16 · 1.9 GiBpublished file | 32k | 141 tok/sreads 1.9 GiB/token |
| Qwen3 0.6BQwen | 752M | Full precision · comfortable | FP16 / BF16 · 1.4 GiBpublished file | 40k | 118 tok/sreads 2.3 GiB/token |
| Qwen2.5 0.5B InstructQwen | 494M | Full precision · comfortable | FP16 / BF16 · 942 MiBpublished file | 32k | 265 tok/sreads 1.0 GiB/token |
| SmolLM2 360M InstructSmolLM | 362M | Full precision · comfortable | FP16 / BF16 · 690 MiBpublished file | 8k | 272 tok/sreads 1010 MiB/token |
| SmolLM2 135M InstructSmolLM | 135M | Full precision · comfortable | FP16 / BF16 · 257 MiBpublished file | 8k | 629 tok/sreads 437 MiB/token |
| Laguna XS 2.1Laguna | 33.4B | Below usable context | Q3_K_M · 14.5 GiBpublished file | 4k | 201 tok/sreads 1.3 GiB/token |
| Hy3HY V3 | 298.8B | No on-device fit | Q3_K_M · 128.1 GiBpublished file | — | Not resident— |
| Laguna S 2.1Laguna | 117.6B | No on-device fit | Q3_K_M · 50.3 GiBpublished file | — | Not resident— |
| gpt-oss 120Bgpt-oss | 116.8B | No on-device fit | Q3_K_M · 58.3 GiBpublished file | — | Not resident— |
| Qwen2.5 72B InstructQwen | 72.7B | No on-device fit | Q3_K_M · 35.1 GiBpublished file | — | Not resident— |
| DeepSeek-R1-Distill-Llama 70BDeepSeek | 70.6B | No on-device fit | Q3_K_M · 31.9 GiBpublished file | — | Not resident— |
| Llama 3.1 70B InstructLlama | 70.6B | No on-device fit | Q3_K_M · 31.9 GiBpublished file | — | Not resident— |
| Llama 3.1 Nemotron 70B InstructNemotron | 70.6B | No on-device fit | Q3_K_M · 31.9 GiBpublished file | — | Not resident— |
| Llama 3.3 70B InstructLlama | 70.6B | No on-device fit | Q3_K_M · 31.9 GiBpublished file | — | Not resident— |
| Mixtral 8x7B InstructMistral | 46.7B | No on-device fit | Q3_K_M · 21.3 GiBarchitecture calculation | — | Not resident— |
| Hermes 4.3 36BSeed OSS | 36.2B | No on-device fit | Q3_K_M · 16.5 GiBarchitecture calculation | — | Not resident— |
| KAT Coder V2.5 DevQwen3 5 MOE Text | 34.7B | No on-device fit | Q3_K_M · 15.8 GiBarchitecture calculation | — | Not resident— |
| Qwen AgentWorld 35B A3BQwen3 5 MOE Text | 34.7B | No on-device fit | Q3_K_M · 15.5 GiBpublished file | — | Not resident— |
| Yi 1.5 34B ChatYi | 34.4B | No on-device fit | Q3_K_M · 15.5 GiBpublished file | — | Not resident— |
| DeepSeek-R1-Distill-Qwen 32BDeepSeek | 32.8B | No on-device fit | Q3_K_M · 14.8 GiBpublished file | — | Not resident— |
| Qwen2.5 32B InstructQwen | 32.8B | No on-device fit | Q3_K_M · 14.8 GiBpublished file | — | Not resident— |
| Qwen2.5-Coder 32B InstructQwen | 32.8B | No on-device fit | Q3_K_M · 14.8 GiBpublished file | — | Not resident— |
| QwQ 32BQwen | 32.8B | No on-device fit | Q3_K_M · 14.9 GiBarchitecture calculation | — | Not resident— |
| Qwen3 32BQwen | 32.8B | No on-device fit | Q3_K_M · 14.9 GiBarchitecture calculation | — | Not resident— |
Device specification and assumption
- Memory
- 16 GB GDDR6
- Bandwidth
- 288 GB/s
- Bus
- 128-bit × 18 Gbps
- Usable budget
- 92% → 14.7 GiB
Capacity and bandwidth come from the vendor specification. The usable fraction is an explicit planning assumption, not a device specification.
What the table does — and does not — claim
Weight sizes use published checkpoint or GGUF files where the roster has one, otherwise the documented bits-per-weight calculation. Context comes from each model’s layer and KV-head geometry. The decode figure is the card’s peak bandwidth divided by the bytes one token reads — the weights it routes through plus one pass over the KV cache, printed under each ceiling — so it is a roofline bound, not measured application throughput. Multi-GPU splitting and host-memory offload are outside this on-device table.
Questions about this card
What LLMs can the GeForce RTX 4060 Ti 16GB run?
43 of the 62 open models in this roster have an on-device configuration with usable context. The largest by exact parameter count is Qwen3 30B-A3B at Q3_K_M, with room for 11k tokens. “Largest” describes parameter count, not model quality or task performance.
How much memory is usable on the GeForce RTX 4060 Ti 16GB?
The device publishes 16 GB of GDDR6. This calculator budgets 92%, or 14.7 GiB, for model weights and KV cache; the remainder is an explicit allowance for the runtime, driver or operating system, workspace, and display. It is an assumption rather than a vendor specification.
Are the speed figures benchmarks for the GeForce RTX 4060 Ti 16GB?
No. They are bandwidth-bound roofline ceilings: 288 GB/s divided by the bytes one decoded token actually reads, which the table prints beside every ceiling. That is not the size of the file on disk — a token reads the weights it routes through, all of them for a dense model but only the selected experts for a mixture-of-experts, plus one pass over the KV cache. Laguna XS 2.1 holds 14.5 GiB resident at Q3_K_M yet reads 1.3 GiB/token, which is what produces its 201 tok/s. Real throughput is lower because kernels, cache traffic, prompt processing, scheduling, and runtime overhead also consume time.