How to read the table

Every GGUF model ships in multiple quantization levels — same model, different precision, different file size. The trade is simple: more bits = better quality = bigger file = less room left for context. This tool does the arithmetic for your exact machine: file size per quant, then whatever memory remains becomes your context budget (the KV cache eats it per token).

The recommendation logic is the community consensus from our quantization guide: take the highest quant that still leaves ≥8k of context. Q6/Q5 are near-lossless, Q4_K_M is the sweet spot, and below Q3 quality falls off fast — if you're forced down there, you usually want a smaller model instead (a bigger model at Q4 beats a smaller one at Q8, but a Q2 of anything beats very little).

Honest limits

File sizes are computed from bits-per-weight, not scraped from Hugging Face — real files vary a little by quantizer version (K-quants vs I-quants, imatrix variants). The KV-cache math assumes a GQA-typical architecture; exotic models differ. And max context here is what fits — models also have their own context limits, and quality at extreme context is its own story. Treat the numbers as a reliable guide, not a contract.

Shopping rather than downloading? Can I run it? finds hardware that fits a model. Wondering if you should buy hardware at all? The cost calculator compares buying vs renting vs the API.