This article was originally published on runaihome.com

Every "best local AI model" article skips the question that actually matters: best for
what VRAM. A model that runs beautifully on a 4090 is irrelevant to someone with a
laptop iGPU; a model that fits on 6 GB is left on the table by people with 24 GB to spare.
This guide goes by tier, listing the practical best-in-class picks for language, image, and
audio at each common GPU memory size as of mid-2026.

For each tier I will name the realistic GPUs that hit it, what you can run comfortably, and
what stretches the limit. All models assumed at sensible quantization (typically Q4_K_M for
LLMs, fp8 for image, default for audio) — see our
quantization explainer for the math.

4 GB — laptop integrated graphics, oldest dGPUs

Cards: Apple M1/M2/M3 base (8 GB unified, ~4 GB usable for AI), older entry-level cards
like GTX 1650.

Realistic for:

• Language: Llama 3.2 1B (Q4_K_M, ~700 MB), Phi-3-mini (3.8B Q4, ~2.3 GB), Qwen 2.5 1.5B / 3B (Q4_K_M).
• Image: SD 1.5 with --lowvram flag, 512×512 only.
• Audio: Whisper.cpp tiny or base for transcription.

Stretch: With CPU offload (GGUF), you can technically run Llama 3.1 8B Q4 by spilling
half the layers to system RAM. Speed will be 1–3 tokens/sec, which is enough for casual chat
but tedious for anything else.

Honest take: This tier is for "the model is on my computer, I can read its output," not
for daily AI workflows. Lean into smaller, faster, well-distilled models — the gap between
3B and 8B at this VRAM tier is much less than the gap in inference speed.

6–8 GB — entry consumer (RTX 3050, 3060, 4060)

Cards: RTX 3050 8GB, RTX 3060 8GB, RTX 4060, GTX 1080, M1/M2 Pro 16 GB unified.

Realistic for:

• Language: Llama 3.1 8B Q4_K_M (~5 GB) — the canonical "8 GB tier" model. Mistral 7B, Qwen 2.5 7B, Phi-3 medium, Gemma 7B.
• Image: SD 1.5 comfortably; SDXL with --medvram flag (slower, but works).
• Audio: Whisper.cpp small or medium.

Stretch: Llama 3.1 8B Q5_K_M (~6 GB) for slightly higher quality. Mixed-precision SDXL
on 8 GB. Llama 3.1 70B with heavy CPU offload (very slow but functional).

This is the "first real local AI" tier. Llama 3.1 8B at Q4_K_M genuinely covers most
chat, coding, summarization, and document Q&A use cases. SD 1.5 plus a good LoRA gives
respectable image generation. You will not be running Flux or 70B models, but everything
real-world useful is on the table.

12 GB — mid-tier (RTX 3060 12GB, RTX 4070, M-series 16–24 GB)

Cards: RTX 3060 12GB, RTX 4070, RTX 5070 12GB, M3/M4 Pro 24 GB unified.

Realistic for:

• Language: Llama 3.1 8B Q8_0 (~9 GB) — full quality 8B with comfortable context. Llama 3.2 11B Vision Q4. Qwen 2.5 14B Q4 (~9 GB). Mistral Small 22B Q3.
• Image: SDXL natively, full 1024×1024, with room for ControlNet stacks. SDXL Turbo blazing fast.
• Audio: Whisper.cpp large-v3 (~3 GB) for high-quality transcription.

Stretch: Llama 4 Scout with very aggressive quant + CPU offload (still slow). Flux dev
fp8 with image-only generation at the limit. Llama 3.1 70B Q2 (compromised quality but
runs).

This is the inflection tier. 12 GB unlocks SDXL natively and lets 8B LLMs run at higher
precision. The quality jump from "8 GB Q4 8B" to "12 GB Q8 8B + SDXL" is large enough that
this is where most enthusiasts settle.

16 GB — the 2026 sweet spot (RTX 4060 Ti 16GB, RTX 5060 Ti 16GB)

Cards: RTX 4060 Ti 16GB, RTX 5060 Ti 16GB, RTX 4070 Ti Super, M3/M4 Pro 32 GB unified.

Realistic for:

• Language: Llama 3.1 8B Q8 with 32K context (~12 GB). Qwen 2.5 14B Q5 (~10 GB) at reasonable context. Llama 3.2 11B Vision Q8 with images.
• Image: SDXL with multiple LoRAs + ControlNet at 1024×1024 plus an upscaler stage. Flux dev fp8 (slow but works) or Flux schnell.
• Audio: Whisper large-v3 plus a TTS model like Coqui or XTTS in parallel.
• Multimodal: Run an LLM and SD/Flux concurrently for full-stack pipelines (chatbot that draws).

Stretch: Flux dev fp16 (just barely fits, no headroom). Llama 3.3 70B Q3_K (compromised
quality, ~30 GB so requires heavy CPU offload).

This is the "smartest dollar in 2026" tier. RTX 5060 Ti 16GB is the card I would
recommend to anyone asking "what should I buy for local AI" who is not comfortable spending
$1000+. It runs everything in this tier comfortably and most things in the next tier with
quantization. For the Stable Diffusion / SDXL / Flux comparison,
this is the threshold where Flux becomes practical.

24 GB — the "no compromises" consumer tier (RTX 3090, 4090, 5090)

Cards: RTX 3090, RTX 3090 Ti, RTX 4090, RTX 5090, RTX 6000 Ada (48 GB).

Realistic for:

• Language: Llama 3.1 8B Q8 with 128K context. Qwen 2.5 32B Q4 (~17 GB). Llama 3.2 90B Vision Q3 (compromised). Llama 3.3 70B Q2 (very compromised).
• Image: Flux dev fp16 comfortably, with LoRAs and ControlNet. SDXL with the entire refiner + upscale pipeline in a single workflow.
• Audio: All Whisper variants plus voice cloning models.
• Combined: A 13B LLM + SDXL running concurrently for an interactive multi-modal app.

Stretch: Llama 3.1 70B Q2 (technically fits, quality is poor — go to two GPUs or 48 GB
single GPU instead). Llama 4 Scout with aggressive offload.

This tier eliminates compromises for everything except 70B+ language models at full
quality. It is overkill for anyone whose primary use is 8B LLM chat, sufficient for
serious image work including Flux, and the entry point for "running a 13B–32B model with
generous context." Used RTX 3090s remain the best price/VRAM ratio in this bracket.

48 GB — prosumer (RTX A6000, RTX 6000 Ada, dual 3090/4090)

Cards: RTX A6000 (48 GB), RTX 6000 Ada (48 GB), 2× RTX 3090 (~48 GB combined), Mac
Studio M2 Ultra 64 GB.

Realistic for:

• Language: Llama 3.1 70B Q4_K_M (~40 GB) at full quality with reasonable context. Qwen 2.5 72B Q4. Mistral Large 123B Q3.
• Image: Multiple Flux instances or batch generation at scale.
• Combined: 70B LLM + Flux concurrently — the "do everything" tier.

Stretch: Llama 3.1 70B Q8 (~75 GB — too much for single 48 GB, requires multi-GPU).
Llama 4 Scout (~62 GB Q4 — fits with very tight budget).

The 70B comfort threshold. Below 48 GB, every 70B inference is a compromise on
quantization or context. At 48 GB, 70B Q4_K_M runs cleanly with 16K+ context, which puts
the model at near-FP16 quality. For research or serious local development, this is the
target.

80 GB — datacenter consumer (A100, H100, H200)

Cards: A100 80GB, H100, H200 (141 GB).

Realistic for:

• Language: Llama 3.1 70B Q8 (~75 GB) at full quality. Qwen 2.5 72B Q8. Llama 4 Scout Q4 with comfortable headroom.
• Image: Multi-Flux pipelines at production scale. Training small LoRAs on Flux.
• Multimodal: Llama 3.2 90B Vision Q4 with full context.

Stretch: Llama 4 Scout fp16 (~218 GB — multi-GPU only).

Mostly relevant if you have access through cloud providers
(RunPod, Vast.ai, Lambda Cloud). Direct ownership of
an H100 is rare outside professional deployments, but