The small models got good — and that changes where government AI can run

For a couple of years, "run the model locally" meant "accept a worse result." If you wanted output you could actually use, you sent your request to a frontier model in someone else's data centre. On-device was a hobbyist's compromise.

That trade-off has quietly collapsed, and it collapsed on a timeline you can put dates on.

Two releases that moved the line

In August 2025, OpenAI released gpt-oss — its first open-weight models, under an Apache 2.0 licence. The smaller of the two, gpt-oss-20b, delivers results in the neighbourhood of o3-mini on common benchmarks and runs within 16GB of memory. Not 16GB of exotic server memory — the amount of RAM in a normal work laptop.

In April 2026, Google released Gemma 4. Alongside the big variants it ships two edge models, E2B and E4B, built specifically to run on phones, tablets and other constrained hardware — multimodal, with a 128K-token context window, small enough to sit on a device in your pocket.

Read those two sentences again with a public-sector hat on. The capability that used to require a cloud round trip now fits inside the machine a civil servant has already been issued.

Why this is a government story, not just a tech story

The Government Digital Service has long designed for a deliberately humble baseline: the "government standard laptop" — roughly an Intel i5 with 16GB of RAM, no dedicated GPU. If something works there, it works for everyone. If it only works on a top-spec machine, it doesn't really work.

For years that baseline was the reason you couldn't do useful local AI in government. A 16GB, CPU-only laptop wasn't going to run anything worth running, so the only path to capable output was the cloud — which brought the data-residency problem straight back.

The small-model releases of the last year change the premise. A 16GB machine is now precisely the target that gpt-oss-20b and the Gemma 4 edge models are built for. The constraint that used to force you online is the constraint these models were designed to meet.

The licence matters as much as the size

There's a second reason these particular releases matter, and it's easy to miss next to the benchmark numbers: they're open-weight. gpt-oss is Apache 2.0. The Gemma 4 weights are downloadable and runnable. You can hold the model as a file, inspect it, run it in an air-gapped environment, and — licensing permitting — redistribute it inside a tool.

Joe Lanman made this point about his own local-model experiment last year: the value isn't only efficiency, it's that you're "not stuck with a large, closed commercial model." For a government-adjacent tool that has to think about procurement, data handling and long-term maintenance, "we own the artefact and it runs on our hardware" is a fundamentally different risk profile from "we rent access to an API that can change under us."

Proof, not theory

I'm making this concrete because I built on it. Prompt to Page is a desktop app for Windows and macOS that generates GOV.UK Design System pages from a plain-English prompt, entirely on-device — and it ships a small catalogue of exactly these models, including the Gemma 4 edge variants. The "standard laptop" baseline is an Intel/Windows machine, and that's now a device the tool runs on directly. On first run it looks at your hardware and recommends a model that will actually fit: a lighter one on an 8GB machine, a balanced default on 16GB, a heavier option if you've got the headroom.

The honest framing is important here. These models are not as good as the frontier cloud models, and they don't need to be. The job is narrow — generate valid, accessible GOV.UK HTML — and the tool wraps the model in deterministic validation and accessibility checks that catch its mistakes. A small model doing a focused task behind good guardrails is genuinely useful. A small model asked to be a general genius is not. Knowing which one you're building is most of the work.

The line is still moving

The thing about a trend you can date is that you can extrapolate it. gpt-oss in August, Gemma 4 in April, each cheaper to run and closer to the cloud than the last. Every few months, another category of task that "obviously needed" a frontier model turns out to run fine on a laptop.

For most of the consumer world, that's a nice efficiency. For government — where the data is sensitive, the hardware is standardised, and the accessibility bar is non-negotiable — it's the difference between "we can't use AI for this" and "we can, and the data never leaves the building."

That gap is closing. I'd rather build for where it's heading.