GPU compatibility

What LLMs can run on 8 GB VRAM mainstream GPU?

Good for 7B/8B Q4 models. Examples: RTX 3060 Ti, RTX 4060, RTX 3070.

Open calculator with this GPU preset

Clean local fits4
Offload / slower25
Too large8
Planning capacity6.5 GB usable VRAM

Recommended route: local with limits

This preset is useful locally, but model choice matters. Stay close to the green list for the best experience.

Use locally forsmall chat models, coding helpers, Telegram bots, routing layers, and selected 7B-14B Q4 models.
Watch out foryellow models may run slowly because they need RAM/offload or reduced context settings.
Fallback triggerUse hosted API or cloud GPU when the chosen model lands in yellow for production work.
Green

4 models fit inside the clean planning capacity.

Yellow

25 models can run with RAM/offload tradeoffs.

Red

Examples to avoid locally: Mixtral 8x7B, Llama 3.1 70B Instruct, Qwen3-Next 80B-A3B Instruct.

Best clean fits

Start here for responsive local inference.
4 models

Qwen3 4B Thinking 2507

4.02B Runs locally Q4 about 3.2 GB

Small local reasoning, routing, tool decisions, and lightweight coding on 6GB-8GB GPUs

Best local fit Open model page ->

Gemma 3 4B

4B Runs locally Q4 about 3.5 GB

Small multimodal local assistant and low-resource setups

Best local fit Open model page ->

Qwen2.5 Coder 7B

7B Runs locally Q4 about 5.5 GB

Small local coding assistant and agent tool generation

Best local fit Open model page ->

Mistral 7B

7B Runs locally Q4 about 5.5 GB

Fast local chat and simple agent tasks

Best local fit Open model page ->

Possible with offload

Use only when slower generation and tighter settings are acceptable.
25 models

Llama 3.1 8B Instruct

8B RAM offload Q4 about 6 GB

Fast local chat, lightweight agents, low-cost local testing

Possible, slower Open model page ->

Qwen3 8B

8B RAM offload Q4 about 6 GB

Fast general local assistant with reasoning/coding balance

Possible, slower Open model page ->

DeepSeek-R1-0528-Qwen3-8B

8B RAM offload Q4 about 6 GB

Updated local reasoning experiments, coding logic checks, and step-by-step technical analysis

Possible, slower Open model page ->

Qwen3-VL 8B Instruct

8B RAM offload Q4 about 6.5 GB

Local image understanding, OCR-style document triage, screenshots, and lightweight multimodal agent routing

Possible, slower Open model page ->

Full Q4 model fit table

ModelSizeQ4 needStatusCalculator
Qwen3 4B Thinking 25074.02B3.2 GBRuns locallyOpen calculator
Gemma 3 4B4B3.5 GBRuns locallyOpen calculator
Qwen2.5 Coder 7B7B5.5 GBRuns locallyOpen calculator
Mistral 7B7B5.5 GBRuns locallyOpen calculator
Llama 3.1 8B Instruct8B6 GBRAM offloadOpen calculator
Qwen3 8B8B6 GBRAM offloadOpen calculator
DeepSeek-R1-0528-Qwen3-8B8B6 GBRAM offloadOpen calculator
Qwen3-VL 8B Instruct8B6.5 GBRAM offloadOpen calculator
Qwen3.5 9B9B6.6 GBRAM offloadOpen calculator
Gemma 3 12B12B9 GBRAM offloadOpen calculator
Gemma 4 E4B4B9.6 GBRAM offloadOpen calculator
Qwen2.5 Coder 14B14B10.5 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 14B14B10.5 GBRAM offloadOpen calculator
Phi-4 14B14B10.5 GBRAM offloadOpen calculator
gpt-oss 20B20B14 GBRAM offloadOpen calculator
gpt-oss-safeguard 20B21B14 GBRAM offloadOpen calculator
Devstral Small 2 24B24B15 GBRAM offloadOpen calculator
Qwen3.5 27B27B17 GBRAM offloadOpen calculator
Qwen3.6 27B27B17 GBRAM offloadOpen calculator
Gemma 3 27B27B18 GBRAM offloadOpen calculator
Qwen3 30B-A3B Instruct 250730.5B19 GBRAM offloadOpen calculator
Qwen3-Coder 30B-A3B30B19 GBRAM offloadOpen calculator
GLM-4.7-Flash30B19 GBRAM offloadOpen calculator
Gemma 4 31B31B20 GBRAM offloadOpen calculator
Qwen3-VL 30B-A3B Instruct30B20 GBRAM offloadOpen calculator
Qwen2.5 Coder 32B32B21 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 32B32B21 GBRAM offloadOpen calculator
Qwen3.8 27B27B22 GBRAM offloadOpen calculator
Qwen3.6 35B-A3B35B24 GBRAM offloadOpen calculator
Mixtral 8x7B46.7B28 GBToo largeOpen calculator
Llama 3.1 70B Instruct70B44 GBToo largeOpen calculator
Qwen3-Next 80B-A3B Instruct80B50 GBToo largeOpen calculator
Qwen3-Coder-Next80B52 GBToo largeOpen calculator
gpt-oss 120B120B65 GBToo largeOpen calculator
gpt-oss-safeguard 120B117B65 GBToo largeOpen calculator
Devstral 2 123B123B75 GBToo largeOpen calculator
GLM-5.2744B466 GBToo largeOpen calculator