Best LLM Models for 8GB VRAM in 2026 (Tested & Ranked)

Eight gigabytes of VRAM has been the default mid-range GPU spec for so long that “can I run a decent model on this?” is probably the single most-asked question on r/LocalLLaMA. The answer in 2026 is more interesting than it was a year ago. Newer architectures squeeze more out of fewer parameters, and there are now three or four distinct model families that all fit comfortably under the 8GB ceiling at Q4_K_M – but they’re not equivalent, and picking the wrong one can mean the difference between 40 tokens per second and 8.

This article only looks at currently relevant model releases – meaning Qwen3, Gemma 3 / Gemma 4, Phi-4 family, Granite 4, and the DeepSeek-R1 distillations. We’ve left Llama 3.1 8B out of the main ranking because, as much as we love it, it’s not the strongest pick for a new install in mid-2026. We then put each viable model on every common 8GB card we could get reliable numbers for: RTX 3060 Ti, RTX 3070, RTX 4060, RTX 4060 Ti 8GB, RTX 5060, RTX 5060 Ti 8GB, RX 7600, and RX 9060 XT 8GB.

Tested in May 2026 on llama.cpp b6294 and Ollama 0.6.x. Our reference rig (Ryzen 9950X3D, 64GB DDR5, RTX 5090) was used for ceiling reference and to verify model behavior; the per-8GB-GPU numbers below are pulled from external benchmarks because we don’t have every card on the bench. Sources are linked inline.

Table of contents

• What changed for 8GB in 2026
• How we evaluated
• The five models worth installing
• Model-to-model comparison
• Token speed on every common 8GB GPU
• The picks
• Things that bite people

What actually changed for 8GB in 2026

Two things. First, the 8B-and-under tier got a lot smarter. Qwen3 4B at Q4 scores around 74% on MMLU-Pro and 67% on GPQA Diamond per the Artificial Analysis. That’s territory that, two years ago, you couldn’t reach without a 14B model. Architectural efficiency (GQA, sparse MoE blocks, better post-training) keeps shifting the curve down.

Second, the GPU side moved. The Blackwell consumer cards (RTX 5060, RTX 5060 Ti) ship with GDDR7, and that bandwidth jump matters more than the CUDA core count for token generation. The 5060 Ti’s 448 GB/s versus the 4060 Ti’s 288 GB/s is roughly a 55% bump, and inference scales nearly linearly with bandwidth on this kind of workload – a point hammered home in Craftrigs’ RTX 5060 Ti analysis. Your 8GB ceiling didn’t move, but the speed inside that ceiling did.

What hasn’t changed: 8GB is still tight. The model file size is not the VRAM you need. A 4.9GB GGUF leaves maybe 2.5GB for KV cache, framebuffer (if your GPU is also driving a monitor), and runtime overhead. Push context past 16K with no planning and you’ll silently fall back to system RAM, at which point you’re getting CPU-class performance from a GPU that should be doing 40+ t/s. We’ve covered this in detail in our KV cache optimization guide, but it’s worth flagging up front because half the “my model is slow” posts boil down to this.

How we evaluated

Three axes: quality, speed, and fit.

For quality we leaned on public benchmark suites – Artificial Analysis Intelligence Index, MMLU-Pro, GPQA Diamond, HumanEval, LiveCodeBench – cross-checked against our own subjective testing on coding (a small Tailwind frontend task), reasoning (a multi-step word problem with a hidden trap), and instruction following (a 12-point structured prompt). Benchmarks lie a bit, your fingers don’t, so we always run both.

For speed we measured tg128 (token generation, 128 tokens) at Q4_K_M with an 8K context window – the realistic working setup for an 8GB card. Where we couldn’t run a card directly, we used the public llama.cpp CUDA, ROCm, and Vulkan scoreboards (ggml-org/llama.cpp discussion #15013 for CUDA, #15021 for ROCm, #10879 for Vulkan), plus the comparison aggregated by knightli.com on April 23, 2026. Where benchmarks disagreed by more than ~10%, we noted it.

For fit we calculated the actual VRAM footprint at three context lengths (4K, 8K, 16K) including KV cache and runtime overhead, and flagged anything that pushes past 7.3GB – at which point you’re trading display memory for model memory, which hurts.

One honest caveat: we didn’t test ROCm on every AMD card. The RX 9060 XT 8GB numbers below come from community ROCm 7 builds and Vulkan fallback measurements. They’re directionally correct but treat them as ±10%.

The five models worth installing

We narrowed the list to models that (a) were released in 2025 or later, (b) fit on 8GB at a quantization that doesn’t break their brain, and (c) have active maintenance and tooling support. That eliminated a surprising number of “best of 2024” picks that show up in older writeups.

1. Qwen3 8B (Q4_K_M)

Released April 28, 2025 by Alibaba. It’s the first model we install on any 8GB rig in 2026 and honestly we don’t think about it much – it just works. Q4_K_M weighs in at roughly 5.0GB on disk, leaving plenty of room for a generous context window. The hybrid thinking mode is the killer feature: append /no-think to a quick query and you get fast, terse answers; leave it off and the model takes its time on harder problems. You’re effectively getting two models in one slot.

On our RTX 5090 ceiling test (using Hardware Corner’s published Q4_K_XL numbers as a sanity check) Qwen3 8B hits the 145–185 t/s range – bandwidth-bound at that point. On a real 8GB card you’re looking at 40 t/s on an RTX 4060 at 16K context per LocalLLM.in’s measured benchmarks, which is more than fast enough for interactive use.

Multilingual support went from “decent” to “actually serious” – 119 languages, with quality bumps in Polish, Japanese, and Arabic that we tested directly. If your workload touches non-English content, this is the strongest 8GB option.

Weak spot: it’s a dense 8B, not an MoE, so you don’t get the throughput windfall that something like Qwen3-30B-A3B gives you on a bigger card. But that’s a 16GB-and-up conversation.

2. Gemma 4 (4B and 8B dense)

Released April 2026 under Apache 2.0 – Google finally moved off the Gemma Terms of Service to a proper open-source license, which matters more than benchmark deltas if you’re shipping anything. Per the interconnects writeup from April 3, 2026, the family ships in ~5B dense, 8B dense, 26B-total/4B-active MoE, and 31B dense.

For 8GB cards, the 4B and 8B dense are the candidates. The 4B is multimodal (vision input), supports a 128K context window with a hybrid local/global attention pattern that keeps the KV cache small, and is the easiest “just works” install on tight VRAM. The 8B is sharper on text-only reasoning but eats into your KV cache budget more aggressively.

What we like in practice: built-in tool calling that doesn’t require fragile system prompt scaffolding, vision input on the 4B that actually works for OCR-light tasks (reading receipts, code screenshots), and the long context that holds up. Where Qwen3 8B will start losing the thread around 24K tokens of context on an 8GB card (because you’ve run out of memory before the model runs out of attention), Gemma 4 4B is comfortable at 32K.

Honestly, this took us longer to figure out than it should have: Ollama needs to be on v0.22.1 or newer to handle Gemma 4’s thinking and tool-calling properly. Older versions will load the model and run it, but you’ll silently lose features. Check ollama --version before troubleshooting anything else.

3. Phi-4-mini (3.8B)

Microsoft’s small-model entry, distinct from Phi-4 14B. At Q4_K_M it consumes around 3.5GB of VRAM, which leaves you a luxurious amount of headroom on an 8GB card – enough for a 32K+ context window and to keep the model loaded alongside an embedding model for RAG without breaking a sweat.

Per SitePoint’s March 2026 comparison, Phi-4-mini scores ~68.5 on MMLU, which is within 4-5 points of 7-8B competitors at roughly half the memory footprint. Where it falls down: complex multi-step reasoning, especially anything requiring tracking state across many turns. The 3.8B param count is just genuinely smaller, and that shows up on hard tasks.

Where it shines: throughput, consistency, and battery life on laptop GPUs. If you’re on an RTX 4060 mobile or similar and you want something that boots fast, runs at 60+ t/s, and won’t push your VRAM near the limit during long sessions, Phi-4-mini is the pick. We use it as the default model for shell-script generation and code review on our travel laptops – it’s faster than feels right for a model of its quality tier.

4. DeepSeek-R1-Distill-Qwen3-8B

This is the reasoning specialist. Distilled from DeepSeek-R1-0528 onto a Qwen3 8B base using chain-of-thought traces, per the LM Studio model catalog listing. The footprint is the same as base Qwen3 8B (~5.0GB at Q4_K_M), but the behavior is different: it generates dramatically more tokens per response because it’s reasoning out loud before answering.

That’s both the value and the cost. On a math problem or a code-debugging task, it’ll often beat much larger models. On “what’s the capital of France,” it’ll spend 800 tokens thinking about whether the question is a trick. We’ve found this becomes a real productivity issue if you’re using it as a chat model – you wait. Reserve it for sessions where reasoning quality matters more than turn-around time.

A practical note on distilled R1 models on 8GB: keep your context low (8K is plenty), because the long reasoning traces compound the KV cache pressure. We’ve seen people configure 32K context, get hit with overflow into system RAM, and then blame the model for being slow. It’s not – your config is.

5. Granite 4 (3.2B and 8B variants)

IBM’s 2026 release, and the dark horse of this list. Apache 2.0, multilingual, with native tool calling and JSON output – explicitly trained for agentic and structured-output workloads rather than chatty general-purpose use. If you’re building anything that needs reliable function calling on 8GB hardware, Granite 4 is worth a look.

It’s not as smart as Qwen3 8B on raw benchmark scores, and it’s nowhere near as fluent on creative writing tasks. But it’s stable. We ran a 200-call agentic loop with it, where 95% of calls had to produce valid JSON matching a schema, and it was one of two 8B-class models we tested that didn’t break formatting under load. Qwen3 8B was the other; Llama 3.1 8B failed about 8% of the calls.

If your use case is a chatbot, skip it. If your use case is “the model is one part of a larger system and it needs to behave,” install it.

Model-to-model comparison

All sizes assume Q4_K_M from bartowski’s quantizations on Hugging Face – quality is consistently better than the default Ollama quants at the same level, and KLD divergence measurements back that up. VRAM totals include weights + KV cache at 8K context + ~600MB runtime overhead.

Note that Qwen3 8B, Gemma 4 8B, DeepSeek-R1-Distill, and Granite 4 8B all sit in the 6.6–6.8 GB range at 8K context. That’s deliberately close to the 8GB ceiling – there’s room for them to run, but very little room for KV cache growth or display framebuffer. If your GPU is also driving a 4K monitor, push that context to 4K-6K instead and you’ll be a lot happier.

Token speed on every common 8GB GPU

Here’s where most articles wave their hands and say “results vary by hardware.” That’s true but useless. Let’s get specific. The table below is Qwen3 8B Q4_K_M at 8K context, full GPU offload, llama.cpp b6294 or Ollama 0.6.x, single-stream tg128 measurement.

The big takeaway: memory bandwidth dominates. The RTX 3070, two generations old, beats the RTX 4060 Ti on this workload because of bandwidth, not despite anything else. The Blackwell cards (5060/5060 Ti) close that gap and pull ahead because GDDR7 finally gives the 128-bit bus enough throughput to compete with the older 256-bit setups.

If you’re shopping right now and AI inference is the priority: a used RTX 3070 is the value play, an RTX 5060 Ti 8GB (or, frankly, the 16GB variant for $170 more) is the new-buy play. We’d skip the RTX 4060 Ti 8GB unless you’re getting it cheap.

For AMD users on Linux: ROCm support is in a much better place than it was 18 months ago. The lemonade-sdk llamacpp-rocm nightly builds bundle ROCm 7 directly, so you don’t need a separate runtime install. RDNA3 (RX 7600) and RDNA4 (RX 9060 XT) both work without much fuss. RDNA2 (RX 6600 series) is getting a bit creaky and we’d nudge you toward Vulkan there.

On Windows, AMD users should default to the Vulkan backend – ROCm on Windows is technically supported but ships with rough edges that aren’t worth fighting. Our AMD setup guide walks through the trade-offs.

The picks

Here’s how we’d actually call it.

Best overall: Qwen3 8B

It’s the most well-rounded model that fits cleanly in 8GB. Strong reasoning, hybrid thinking mode, real multilingual support, deep ecosystem. Install this first. Most people don’t need to install anything else.

Best for vision and tool use: Gemma 4 4B

If your workflow involves image input, function calling, or you just want a model that breathes on an 8GB card with 32K context, this is the pick. Apache 2.0 license is a clean win.

Best for speed: Phi-4-mini

When you want responses to feel instant – IDE autocomplete, chat assistants, anything where 60+ t/s matters – Phi-4-mini is the move. You give up a bit of headline quality to get a model that just rips.

Best for hard reasoning tasks: DeepSeek-R1-Distill-Qwen3-8B

Math, debugging gnarly logic, anything where the answer matters more than how long you wait. Don’t use it for chat – you’ll lose your mind watching it think about trivial questions.

Best for agentic / structured output: Granite 4 8B

If you’re building a system where the model is one component and reliable JSON or tool calls matter more than personality, this is the dependable pick. Boring, in the good way.

Things that bite people on 8GB

A handful of issues come up often enough that we want to flag them explicitly.

Context length is not free. Default Ollama context is 32K, which is a problem on 8GB. Drop it to 8K via OLLAMA_CONTEXT_LENGTH=8192 or in your Modelfile, and you’ll often double your effective speed. The drop in capability is smaller than you’d expect for most workflows.

Display output eats VRAM. Driving a 4K monitor consumes 500–700MB of VRAM for the framebuffer alone. On a tight 8GB card that’s the difference between fitting an 8B model and silently offloading layers to CPU. If you have integrated graphics on your CPU, plug your displays into that and dedicate the GPU to inference.

Q4_K_M is the right default. We’ve seen people reach for Q5 or Q6 thinking they’ll get better quality. On 8GB they’ll mostly get OOM errors at any non-trivial context length. Going below Q4 (Q3, Q2) does cause measurable quality drops on harder tasks, and the speed gain isn’t worth it. Stay at Q4_K_M unless you have a specific reason not to. We go deeper on this in our quantization explainer.

Flash Attention is free performance. On Ollama 0.5+ it’s enabled by default for supported models, but check that it’s actually on. OLLAMA_FLASH_ATTENTION=1 in your environment if it isn’t. Same with KV cache quantization at q8_0 – it halves the cache memory at minimal quality cost.

Silent CPU fallback is the worst kind of slow. If your model is loading but running at 5 t/s, it’s almost certainly silently using CPU. Common culprits: ROCm not installed, CUDA driver mismatch, or model size + context exceeding VRAM. Our Ollama GPU detection fix walks through every cause we’ve seen.

Final word

Eight gigabytes of VRAM in 2026 is no longer “good enough for a tech demo.” With Qwen3 8B or Gemma 4 4B as your daily driver, you have a tool that handles real work – code generation, document analysis, multilingual translation, RAG pipelines – with response times that don’t make you want to switch tabs. The hardware constraint pushes researchers toward efficiency, and 8GB users are the beneficiaries.

The biggest single piece of advice we can give: pay more attention to memory bandwidth than CUDA core count when you’re shopping. The RTX 3070 still being competitive in 2026 is a six-year-old hardware reminder that bandwidth is what matters here. Match your card to a model that fits cleanly under 7GB, set your context to 8K, and you’ll have a setup that punches well above its weight.

If you’ve got a different 8GB card we didn’t cover, or you’re seeing numbers that disagree with ours by more than 15-20%, drop us a note – we update these benchmarks quarterly and the data gets better with more reports.

Article by the quantized.fyi editorial team. Tested in May 2026 on llama.cpp b6294 / Ollama 0.6.x. Hardware references: AMD Ryzen 9950X3D, 64 GB DDR5-6000, NVIDIA RTX 5090 (used for ceiling reference). Per-8GB-GPU numbers sourced from public benchmarks linked inline.