Smaller models need smaller prompts

I had a tidy intuition. A small local model is weaker than a big one, so it needs more help: more component documentation, more worked examples, more scaffolding crammed into the prompt. Give the struggling model a bigger crib sheet and it'll do better.

I was wrong, and I can show you by how much. Trimming the prompt — cutting the single most expensive part of the context I was so carefully assembling — took a small model from passing 1 of 7 test cases to passing 4 of 7. Adding help was hurting it.

This post is how I found that out, because the method matters as much as the result.

First, build the ruler

You cannot have this conversation on vibes. "The model feels better with examples" is how you ship a regression you can't see. So before tuning anything, I built a deterministic scorer.

It takes a generated GOV.UK page and scores it 0–100 against real rules: are the govuk-* classes valid, is the page structure correct (the width container, the main wrapper, the grid), are form labels and input types accessible, are the journey routes and field names right. An error costs 15 points, a warning 5. A case passes at 80 or above with no errors. No human in the loop, no taste, same answer every time.

Then seven test cases, chosen to span the real work: a single component, a start page, a question page, radios, a check-answers page, a confirmation page, and a multi-page journey. Not a big evaluation set — I'll come back to that, hard — but a consistent one. The same seven pages, scored the same way, every run.

With a ruler in hand, "is more context better?" stops being an opinion and becomes a measurement.

The constraint nobody warns you about

Here's the thing that makes small models different. Every generation has to fit inside the model's context window, and that window has to hold everything at once:

context window ≥ system prompt + retrieved docs + your request + the whole page it writes

On a frontier cloud model with a vast window, you never feel this. On a small model on a 16GB laptop, the window is the budget for the entire job — and I was spending most of it before the model wrote a word.

I learned this the embarrassing way first. Early on, the assembled prompt overflowed a too-small context window, the generated page got truncated, structure scores cratered, and it looked for all the world like "the model is bad." It wasn't. The prompt was eating the room the page needed. Fixing the budget, not the model, fixed the output. That experience reframed everything: the prompt isn't free advice you bolt on, it's a cost you pay out of a fixed account.

So what's in the account? Roughly: the system prompt (the 473-class list and house rules) costs about 1,900 tokens. The retrieved component documentation costs about 2,500 more. And within that retrieval block, the worked HTML examples — one per component — are about 70% of the cost. That last number is the whole story.

The experiment

Vary one thing at a time, hold everything else fixed, run the seven cases, read the score. My weak-model stand-in was Gemma-4-E4B, run locally. (An honest wrinkle I'll own below.)

Read the fourth row. Removing the per-component HTML examples — the most expensive 70% of the retrieval budget — lifted the model from 1/7 to 4/7 and the average from 59 to 83, over the passing line. It helped five of the seven cases and made none worse. It was also about 40% faster, because the model had less to read.

Now read the bottom two rows, because they're the half people skip. When I also cut the component class lists, quality collapsed back to 2/7. The examples were costing more than they were worth; the class lists were load-bearing. "Smaller prompt" was right. "Strip the context" was very wrong. The skill is knowing which token is signal and which is ballast.

I re-ran the headline result across three random seeds to make sure it wasn't a fluke: baseline averaged 58.3, examples-off averaged 80.7, with no overlap on any seed. The win was real, not a lucky roll.

Why "more help" backfired

The mental model I started with — weak model, so feed it more — had the causation backwards. For a small model on a tight budget, every worked example is a stretch of text it has to wade through before it can start, and a stretch of window it can no longer use for the page itself. The class list is irreducible: it's the only thing telling the model which classes exist. The examples are mostly redundant with the class list, and on a small model they crowd out the work. Take them away and you've given the model both a clearer signal and more room to write.

The neat part is what this says against an obvious comparison. When Joe Lanman built his single-component tool last year, he added a Design System example to his prompt to fix a date-input bug — and he was right to. But his task was different from mine: configure one component, where the example is the signal. Mine is generate a whole page, where five examples become noise against a tight budget. We did opposite things and we were both correct, because the task and the budget were different. The only reason I can say that with a straight face is that I measured it. Without the scorer, I'd just have two people with opposite anecdotes.

The caveats, because you should distrust this a little

I'd be breaking my own rule if I sold this as a law. It isn't. It's a strong hypothesis that shipped a conservative change.

The sample is tiny. Seven cases, a handful of seeds. That's a smoke test, not an evaluation. Expanding the held-out set to a hundred-plus cases is the single most important thing I still owe this work, and until I do, every number here is provisional.

It wasn't even a true small model. I couldn't pull a genuine "Fast"-tier model in the build environment, so I used Gemma-4-E4B — a balanced-tier model — as the closest stand-in. The finding needs re-confirming on the real small models.

One runtime, one shot. This was GGUF via Ollama — the same model format the app uses on Windows and Intel Macs — rather than the MLX path it uses on Apple Silicon. So it's closest to the Windows experience; the Apple Silicon path still needs its own confirmation. It also scores the model's first attempt, with no validation-retry. Both could move the numbers.

The scorer rewards what it checks. It measures valid classes and correct structure, not whether the page is genuinely well-designed. A high score is necessary, not sufficient.

So what shipped?

Only the safe version: the example-dropping trim applies to the small "Fast"-tier models, where it's measured to help. The larger models get the full prompt, byte-for-byte unchanged. The change can only help where it was tested and can't regress where it wasn't.

The takeaway

If you're putting a small model to work, your instinct to over-help it is the trap. Give it the irreducible signal — the part it genuinely can't infer — and give it room to work. Cut the rest.

And don't trust the instinct, including this one. Build the ruler first. The reason I can tell you "smaller models need smaller prompts" instead of merely believing it is that I measured it, on the same seven pages, the same way, every time.

Prompt to Page is an independent project by Courtney Allen. It is not affiliated with, endorsed by, or connected to the Government Digital Service, the Crown, or any UK government body. "GOV.UK" and the GOV.UK Design System are referenced descriptively.