GPU compatibility

What LLMs can run on 6 GB VRAM entry GPU?

Small local chat models only. Examples: GTX 1660, RTX 2060 6GB.

Open calculator with this GPU preset

Clean local fits2
Offload / slower12
Too large23
Planning capacity4.5 GB usable VRAM

Recommended route: local with limits

This preset is useful locally, but model choice matters. Stay close to the green list for the best experience.

Use locally forsmall chat models, coding helpers, Telegram bots, routing layers, and selected 7B-14B Q4 models.
Watch out foryellow models may run slowly because they need RAM/offload or reduced context settings.
Fallback triggerUse hosted API or cloud GPU when the chosen model lands in yellow for production work.
Green

2 models fit inside the clean planning capacity.

Yellow

12 models can run with RAM/offload tradeoffs.

Red

Examples to avoid locally: gpt-oss 20B, gpt-oss-safeguard 20B, Devstral Small 2 24B.

Best clean fits

Start here for responsive local inference.
2 models

Qwen3 4B Thinking 2507

4.02B Runs locally Q4 about 3.2 GB

Small local reasoning, routing, tool decisions, and lightweight coding on 6GB-8GB GPUs

Best local fit Open model page ->

Gemma 3 4B

4B Runs locally Q4 about 3.5 GB

Small multimodal local assistant and low-resource setups

Best local fit Open model page ->

Possible with offload

Use only when slower generation and tighter settings are acceptable.
12 models

Qwen2.5 Coder 7B

7B RAM offload Q4 about 5.5 GB

Small local coding assistant and agent tool generation

Possible, slower Open model page ->

Mistral 7B

7B RAM offload Q4 about 5.5 GB

Fast local chat and simple agent tasks

Possible, slower Open model page ->

Llama 3.1 8B Instruct

8B RAM offload Q4 about 6 GB

Fast local chat, lightweight agents, low-cost local testing

Possible, slower Open model page ->

Qwen3 8B

8B RAM offload Q4 about 6 GB

Fast general local assistant with reasoning/coding balance

Possible, slower Open model page ->

Full Q4 model fit table

ModelSizeQ4 needStatusCalculator
Qwen3 4B Thinking 25074.02B3.2 GBRuns locallyOpen calculator
Gemma 3 4B4B3.5 GBRuns locallyOpen calculator
Qwen2.5 Coder 7B7B5.5 GBRAM offloadOpen calculator
Mistral 7B7B5.5 GBRAM offloadOpen calculator
Llama 3.1 8B Instruct8B6 GBRAM offloadOpen calculator
Qwen3 8B8B6 GBRAM offloadOpen calculator
DeepSeek-R1-0528-Qwen3-8B8B6 GBRAM offloadOpen calculator
Qwen3-VL 8B Instruct8B6.5 GBRAM offloadOpen calculator
Qwen3.5 9B9B6.6 GBRAM offloadOpen calculator
Gemma 3 12B12B9 GBRAM offloadOpen calculator
Gemma 4 E4B4B9.6 GBRAM offloadOpen calculator
Qwen2.5 Coder 14B14B10.5 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 14B14B10.5 GBRAM offloadOpen calculator
Phi-4 14B14B10.5 GBRAM offloadOpen calculator
gpt-oss 20B20B14 GBToo largeOpen calculator
gpt-oss-safeguard 20B21B14 GBToo largeOpen calculator
Devstral Small 2 24B24B15 GBToo largeOpen calculator
Qwen3.5 27B27B17 GBToo largeOpen calculator
Qwen3.6 27B27B17 GBToo largeOpen calculator
Gemma 3 27B27B18 GBToo largeOpen calculator
Qwen3 30B-A3B Instruct 250730.5B19 GBToo largeOpen calculator
Qwen3-Coder 30B-A3B30B19 GBToo largeOpen calculator
GLM-4.7-Flash30B19 GBToo largeOpen calculator
Gemma 4 31B31B20 GBToo largeOpen calculator
Qwen3-VL 30B-A3B Instruct30B20 GBToo largeOpen calculator
Qwen2.5 Coder 32B32B21 GBToo largeOpen calculator
DeepSeek R1 Distill Qwen 32B32B21 GBToo largeOpen calculator
Qwen3.8 27B27B22 GBToo largeOpen calculator
Qwen3.6 35B-A3B35B24 GBToo largeOpen calculator
Mixtral 8x7B46.7B28 GBToo largeOpen calculator
Llama 3.1 70B Instruct70B44 GBToo largeOpen calculator
Qwen3-Next 80B-A3B Instruct80B50 GBToo largeOpen calculator
Qwen3-Coder-Next80B52 GBToo largeOpen calculator
gpt-oss 120B120B65 GBToo largeOpen calculator
gpt-oss-safeguard 120B117B65 GBToo largeOpen calculator
Devstral 2 123B123B75 GBToo largeOpen calculator
GLM-5.2744B466 GBToo largeOpen calculator