GPU compatibility

What LLMs can run on 48 GB VRAM workstation?

Large local models and long context. Examples: RTX A6000, L40S 48GB.

Open calculator with this GPU preset

Clean local fits29

Offload / slower4

Too large1

Planning capacity46.5 GB clean VRAM

Recommended route: local-first

This preset has enough clean Q4 headroom for most curated local models in this dataset.

Use locally forcoding assistants, chat, routing, agents, and larger 14B-32B class models depending on context length.

Watch out forvery large models, long context windows, and parallel workloads can still exceed practical memory.

Fallback triggerUse cloud/API when you need 70B+ models, long context, or reliable multi-user throughput.

Green

29 models fit inside the clean planning capacity.

Yellow

4 models can run with RAM/offload tradeoffs.

Red

Examples to avoid locally: GLM-5.2.

Best clean fits

Start here for responsive local inference.

29 models

Qwen3 4B Thinking 2507

4.02B Runs locally Q4 about 3.2 GB

Small local reasoning, routing, tool decisions, and lightweight coding on 6GB-8GB GPUs

Best local fit Open model page ->

Gemma 3 4B

4B Runs locally Q4 about 3.5 GB

Small multimodal local assistant and low-resource setups

Best local fit Open model page ->

Qwen2.5 Coder 7B

7B Runs locally Q4 about 5.5 GB

Small local coding assistant and agent tool generation

Best local fit Open model page ->

Mistral 7B

7B Runs locally Q4 about 5.5 GB

Fast local chat and simple agent tasks

Best local fit Open model page ->

Possible with offload

Use only when slower generation and tighter settings are acceptable.

4 models

Qwen3-Next 80B-A3B Instruct

80B RAM offload Q4 about 50 GB

High-end local reasoning, long-context planning, and tool-agent workloads when 48GB+ memory is available

Possible, slower Open model page ->

Qwen3-Coder-Next

80B RAM offload Q4 about 52 GB

High-end local coding agents, repository-scale code edits, and tool-calling development workflows

Possible, slower Open model page ->

gpt-oss 120B

120B RAM offload Q4 about 65 GB

Large local reasoning servers, heavy agent orchestration, and high-end homelab inference

Possible, slower Open model page ->

Devstral 2 123B

123B RAM offload Q4 about 75 GB

High-end local software engineering agents, large-repository navigation, and tool-heavy coding workflows on server-class memory

Possible, slower Open model page ->

Full Q4 model fit table

Model	Size	Q4 need	Status	Calculator
Qwen3 4B Thinking 2507	4.02B	3.2 GB	Runs locally	Open calculator
Gemma 3 4B	4B	3.5 GB	Runs locally	Open calculator
Qwen2.5 Coder 7B	7B	5.5 GB	Runs locally	Open calculator
Mistral 7B	7B	5.5 GB	Runs locally	Open calculator
Llama 3.1 8B Instruct	8B	6 GB	Runs locally	Open calculator
Qwen3 8B	8B	6 GB	Runs locally	Open calculator
DeepSeek-R1-0528-Qwen3-8B	8B	6 GB	Runs locally	Open calculator
Qwen3-VL 8B Instruct	8B	6.5 GB	Runs locally	Open calculator
Qwen3.5 9B	9B	6.6 GB	Runs locally	Open calculator
Gemma 3 12B	12B	9 GB	Runs locally	Open calculator
Gemma 4 E4B	4B	9.6 GB	Runs locally	Open calculator
Qwen2.5 Coder 14B	14B	10.5 GB	Runs locally	Open calculator
DeepSeek R1 Distill Qwen 14B	14B	10.5 GB	Runs locally	Open calculator
Phi-4 14B	14B	10.5 GB	Runs locally	Open calculator
gpt-oss 20B	20B	14 GB	Runs locally	Open calculator
Devstral Small 2 24B	24B	15 GB	Runs locally	Open calculator
Qwen3.5 27B	27B	17 GB	Runs locally	Open calculator
Qwen3.6 27B	27B	17 GB	Runs locally	Open calculator
Gemma 3 27B	27B	18 GB	Runs locally	Open calculator
Qwen3 30B-A3B Instruct 2507	30.5B	19 GB	Runs locally	Open calculator
Qwen3-Coder 30B-A3B	30B	19 GB	Runs locally	Open calculator
GLM-4.7-Flash	30B	19 GB	Runs locally	Open calculator
Gemma 4 31B	31B	20 GB	Runs locally	Open calculator
Qwen3-VL 30B-A3B Instruct	30B	20 GB	Runs locally	Open calculator
Qwen2.5 Coder 32B	32B	21 GB	Runs locally	Open calculator
DeepSeek R1 Distill Qwen 32B	32B	21 GB	Runs locally	Open calculator
Qwen3.6 35B-A3B	35B	24 GB	Runs locally	Open calculator
Mixtral 8x7B	46.7B	28 GB	Runs locally	Open calculator
Llama 3.1 70B Instruct	70B	44 GB	Runs locally	Open calculator
Qwen3-Next 80B-A3B Instruct	80B	50 GB	RAM offload	Open calculator
Qwen3-Coder-Next	80B	52 GB	RAM offload	Open calculator
gpt-oss 120B	120B	65 GB	RAM offload	Open calculator
Devstral 2 123B	123B	75 GB	RAM offload	Open calculator
GLM-5.2	744B	466 GB	Too large	Open calculator