Historical measurement · not a current runtime claim

The 16 GB Local AI benchmark

Every number here is measured on one real card — an NVIDIA GeForce RTX 5060 Ti with 16 GB of memory — not estimated from a spec sheet. We own exactly one consumer GPU, so instead of guessing at hardware we don't have, we go deep on the tier most people actually run: what fits in 16 GB, at what speed, at the quant a buyer would use.

This dataset was measured with Ollama on August 10, 2026. Shadowfetch Linux 5.0.1 ships an on-device provider, but it bundles neither Ollama nor these models. You supply the model, and it is reached from a network-less sandbox over one granted unix socket, not a localhost TCP port an `ollama serve` opens by default — until a model is reachable that way the installation reports its local AI state as installed-unavailable. These figures describe the historical measurement setup only.

Last measured August 10, 2026 UTC · driver 580.173.02 · Ollama 0.23.2

What fits your card?

Speed is measured on the 16 GB card above. Whether a model fits is mostly its memory footprint, which travels between cards far better than its speed does — so filter by the VRAM you have and see which of these measured models fit inside it.

Filter by VRAM

The measured ladder

Model Params Quant VRAM used On GPU Context Generation Prompt Cold load
Gemma 3 · 1B ollama pull gemma3:1b 999.89M Q4_K_M 1.3 GB 100% GPU 32768 263 tok/s 7931 tok/s 2.61s
Qwen2.5 · 3B Instruct ollama pull qwen2.5:3b-instruct 3.1B Q4_K_M 3.1 GB 100% GPU 32768 175 tok/s 9768 tok/s 2.66s
Gemma 3 · 4B ollama pull gemma3:4b 4.3B Q4_K_M 4.6 GB 100% GPU 131072 120 tok/s 3697 tok/s 3.14s
Llama 3.2 · 3B ollama pull llama3.2:3b 3.2B Q4_K_M 4.8 GB 100% GPU 131072 174 tok/s 8985 tok/s 2.92s
Qwen2.5 · 7B Instruct ollama pull qwen2.5:7b-instruct 7.6B Q4_K_M 6.3 GB 100% GPU 32768 88 tok/s 4960 tok/s 8.55s
Qwen2.5 · 14B ollama pull qwen2.5:14b 14.8B Q4_K_M 13 GB 100% GPU 32768 45 tok/s 2548 tok/s 9.97s

Where to start

  • On a 16 GB card, start with Qwen2.5 · 14B — the largest measured model that fits, at 13 GB and 45 tok/s.
  • Want it snappier? Drop to Qwen2.5 · 7B Instruct at 6.3 GB and 88 tok/s — about 2× the generation speed.
  • On an 8 GB card, run Llama 3.2 · 3B — 4.8 GB and 174 tok/s.

For what actually ships on the device, see the on-device provider and project workspaces.

How these were measured

  • One NVIDIA GeForce RTX 5060 Ti (16 GB). Generation rate is eval tokens ÷ eval time from the Ollama API, taken as the median of 3 timed runs of 200 tokens after a warm-up.
  • VRAM used and “on GPU” are what ollama ps reports for the resident model — the model's real footprint, and whether all of it sits on the GPU or spills to the CPU (which is the moment speed falls off a cliff).
  • Params, quantisation and architecture are read from ollama show — the model file's own metadata, not typed by us. Everything shown is Q4_K_M-class weights, the quant most people actually run.
  • Speed is specific to this card; your tokens/sec will differ on other hardware. Footprint and fit travel much better — that's why the filter above keys off memory, not speed.