GPU compatibility

What LLMs can run on 32 GB VRAM Blackwell workstation?

Ultra high-end consumer workstation for 30B+ models with extra headroom. Examples: RTX 5090.

Open calculator with this GPU preset

Clean local fits22

Offload / slower2

Too large0

Planning capacity30.5 GB clean VRAM

Recommended route: local-first

This preset has enough clean Q4 headroom for most curated local models in this dataset.

Use locally forcoding assistants, chat, routing, agents, and larger 14B-32B class models depending on context length.

Watch out forvery large models, long context windows, and parallel workloads can still exceed practical memory.

Fallback triggerUse cloud/API when you need 70B+ models, long context, or reliable multi-user throughput.

Green

22 models fit inside the clean planning capacity.

Yellow

2 models can run with RAM/offload tradeoffs.

Red

Examples to avoid locally: None in this dataset.

Best clean fits

Start here for responsive local inference.

22 models

Gemma 3 4B

4B Runs locally Q4 about 3.5 GB

Small multimodal local assistant and low-resource setups

Best local fit Open model page ->

Qwen2.5 Coder 7B

7B Runs locally Q4 about 5.5 GB

Small local coding assistant and agent tool generation

Best local fit Open model page ->

Mistral 7B

7B Runs locally Q4 about 5.5 GB

Fast local chat and simple agent tasks

Best local fit Open model page ->

Llama 3.1 8B Instruct

8B Runs locally Q4 about 6 GB

Fast local chat, lightweight agents, low-cost local testing

Best local fit Open model page ->

Possible with offload

Use only when slower generation and tighter settings are acceptable.

2 models

Llama 3.1 70B Instruct

70B RAM offload Q4 about 44 GB

High-quality local chat and reasoning on workstation-class hardware

Possible, slower Open model page ->

gpt-oss 120B

120B RAM offload Q4 about 65 GB

Large local reasoning servers, heavy agent orchestration, and high-end homelab inference

Possible, slower Open model page ->

Full Q4 model fit table

Model	Size	Q4 need	Status	Calculator
Gemma 3 4B	4B	3.5 GB	Runs locally	Open calculator
Qwen2.5 Coder 7B	7B	5.5 GB	Runs locally	Open calculator
Mistral 7B	7B	5.5 GB	Runs locally	Open calculator
Llama 3.1 8B Instruct	8B	6 GB	Runs locally	Open calculator
Qwen3 8B	8B	6 GB	Runs locally	Open calculator
DeepSeek-R1-0528-Qwen3-8B	8B	6 GB	Runs locally	Open calculator
Qwen3.5 9B	9B	6.6 GB	Runs locally	Open calculator
Gemma 3 12B	12B	9 GB	Runs locally	Open calculator
Gemma 4 E4B	4B	9.6 GB	Runs locally	Open calculator
Qwen2.5 Coder 14B	14B	10.5 GB	Runs locally	Open calculator
DeepSeek R1 Distill Qwen 14B	14B	10.5 GB	Runs locally	Open calculator
Phi-4 14B	14B	10.5 GB	Runs locally	Open calculator
gpt-oss 20B	20B	14 GB	Runs locally	Open calculator
Devstral Small 2 24B	24B	15 GB	Runs locally	Open calculator
Qwen3.5 27B	27B	17 GB	Runs locally	Open calculator
Qwen3.6 27B	27B	17 GB	Runs locally	Open calculator
Gemma 3 27B	27B	18 GB	Runs locally	Open calculator
Qwen3-Coder 30B-A3B	30B	19 GB	Runs locally	Open calculator
Gemma 4 31B	31B	20 GB	Runs locally	Open calculator
Qwen2.5 Coder 32B	32B	21 GB	Runs locally	Open calculator
DeepSeek R1 Distill Qwen 32B	32B	21 GB	Runs locally	Open calculator
Mixtral 8x7B	46.7B	28 GB	Runs locally	Open calculator
Llama 3.1 70B Instruct	70B	44 GB	RAM offload	Open calculator
gpt-oss 120B	120B	65 GB	RAM offload	Open calculator