Adding a model is not the same as recommending it

Last week I shipped a new model in Prompt to Page: Qwen3-Coder 30B-A3B. It is the first entry in the catalogue to score 4 out of 5 on the GOV.UK generation suite. A median of 89, on the same seven pages I use for every other model.

Then the requests arrived, as they always do when a new Qwen lands: add 3.5 (the 9B, or the 4B). Add 3.6 (the 35B mixture-of-experts).

I ran them. Version 0.3.2 adds Qwen3.6 35B-A3B to the picker.

It scored a 3. The 30B coder in the same 32 GB bracket still scores a 4. I added the new model anyway, as a comparison, and I did not make it the recommendation. That is not a slight on a strong general model. It is how a catalogue is supposed to work.

The ruler, briefly

I wrote earlier about how the scorer works. Short version, because the ranking is meaningless without it:

A generated page is scored 0–100 against real design-system rules — valid classes, page structure, form labels, journey routes, carried field names. A case passes at 80 with no errors. The catalogue's 1–5 rating is just that average, bucketed: 80 unlocks a 3 (good enough to be the automatic recommendation); 88 unlocks a 4.

Seven cases. A handful of seeds. A smoke test, not a hundred-page evaluation. I will keep saying that until the held-out set is bigger. It is still the only way I will let a model wear a number in the picker.

The job is also narrower than "coding." Public leaderboards reward repository patches and agentic tool use. This tool asks a local model to emit one design-system page, first shot, from a retrieved class list. Those are different sports. Models that look brilliant on LiveCodeBench can still invent a class that doesn't exist.

What I already ship, ranked

Twenty downloadable entries. I have collapsed GGUF and MLX twins to the better score, because you don't pick a file format — the app does.

Two splits in that table are load-bearing.

Coders still beat general models at this task. Qwen2.5-Coder 7B scores 85. Qwen3 8B, newer and in the same size band, scores 61 on the Windows/llama.cpp path. Eight billion parameters of general reasoning lost to seven billion parameters of "write the HTML." That is why I have not thrown out the 2024 Qwen2.5-Coder line. There is still no smaller Qwen3 coder to replace it.

The same weights are not the same model on every runtime. Qwen3 8B is a 3 on Apple Silicon (81) and a 1 as a GGUF (61). Gemma 4 E4B does the same trick the other way (72 vs 60). A download page that quotes one number for both is averaging away the thing that matters.

One asterisk on the 4: Qwen3-Coder 30B was scored on a stricter version of the suite than most of the 3s. The comparison likely understates how far ahead it is, not the other way round.

What your machine actually gets

I wrote in July about matching a model to the machine — hide anything that will not fit, recommend one that will. That post left an open question about whether to show the premium row greyed-out. The ranking does not settle the UX. It does settle what "premium" now means.

8 GB Windows. The automatic pick is still Qwen2.5-Coder 1.5B (rating 2). The 7B is on the list with a "this will be slow" confirm. Qwen3.5-4B, measured, would be competing with the score-1 and score-2 fast models, not replacing the 1.5B.

16 GB, either platform. This is the machine I optimise for. Qwen2.5-Coder 7B is the ceiling you can actually reach. The 14B is the Mac upgrade if you will tolerate memory pressure. The 35B does not appear here.

24 GB. Gemma 3 12B becomes visible and still is not a 3. Both 32 GB models stay hidden. After Devstral, I do not "caveat" a model that can take the machine down. I hide it.

32 GB. Two models appear: the 4-rated coder and the new 3-rated Qwen3.6. Neither is the automatic first-run pick until 48 GB — the recommender treats premium models as too slow on a 32 GB class machine, on purpose. You can choose either. The app will not surprise you with a 17 GB or 19 GB download.

What the evals actually said

I thought the interesting request was Qwen3.5-9B. A 4-bit copy is about 5–6 GB and wants a 16 GB machine — the band where I currently ship a Qwen3 8B GGUF that scored 61 and a Gemma 4 E4B GGUF that scored 60. A 9B that wrote clean design-system HTML would have changed the 16 GB Windows story.

It did not. On a seed-1 sweep it scored 60, with invented classes and a missed form. Its 4B sibling scored 74. The 7B coder already on that laptop scores 85. I was wrong about where the gap was. I am not growing the catalogue for a 2 that loses to a model I already ship.

Qwen3.5-4B is a possible replacement for the existing Qwen3 4B (50 as a GGUF, 63 on Apple Silicon). Possible. The seed-1 74 is a 2, not a 3, and it still invents classes. There is an older 4B instruct refresh (Qwen3-4B-Instruct-2507) that is non-thinking and uses the runtime I already ship. I would rather try that as a swap than add a fourth 4B.

Qwen3.6-35B-A3B is one model, not two. Thirty-five billion total parameters, three billion active. A 4-bit copy is a 19 GB download and a 32 GB catalogue entry — the same picker row as the 4-rated coder.

Its public numbers look strong. LiveCodeBench v6 at 80.4, SWE-bench Verified at 73.4, and Qwen's own frontend-generation bench well ahead of the 3.5 mixture-of-experts. Those benches are real. They measure a different job. Agentic repository repair is not "emit a check-answers page whose routes and field names match the journey plan."

On this job, the five-seed median was 82: a solid 3. Markup was clean — zero invented classes across the seven pages. The deductions are the same journey-semantics misses I see on every Qwen3-era candidate, a bit deeper on question pages (median 60) and check-answers (median 55) than the coder's. Letting it think did not help; it made the pages worse.

That is not a bad model. It is a good general model that did not overtake a specialist on a specialist task. The gate I had written before the run asked it to tie the champion (median 87 or better) before I would make it a 4. It didn't. So I shipped it as what it measured: a 3, in the same bracket, so testers can put the two side by side.

The 0.3.2 release also had to teach the app a new trick. These families think by default, and they dropped the /no_think switch the older Qwen3 entries honour. Thinking tokens on a laptop eat the output budget and stretch the wait. The new entry launches the bundled server with reasoning switched off, so the whole page budget goes on markup. Existing models launch exactly as before.

I also measured Qwen3.8-27B (median 83, another 3). Same quality band, slower on the Windows machines this catalogue has to serve. It stays out.

What I am not claiming

The 3.5 numbers are a seed-1 sweep, not a five-seed blessing. That is enough not to add them. It is not enough to write them into the picker. The 35B was five-seeded, on the same macOS sidecar the app ships, which is why it has a rating.

The seven-case suite is still too small. A 4 is "cleared 88 on this ruler," not "the best local model in the world." Check-answers remains the floor even for the 30B (median 70 on that one case). The failures are almost always journey semantics — invented routes, placeholder links — not missing buttons. That is useful, and it is not the same as a designer signing the page off.

And the 35B has no Apple Silicon MLX twin yet. Both platforms run the GGUF. If a later runtime makes it faster on a Mac, I will measure that path separately. The rating is for the path that exists.

The principle

A catalogue is not a leaderboard. It is a set of promises about a specific machine doing a specific job.

Shipping a model is one more option you can choose. Recommending it is the promise the app makes on first run. Those are allowed to diverge.

The 16 GB laptop is still the job. The 7B coder is still the best measured promise I can make on that laptop. The 30B coder is still the measured pick on 32 GB. Qwen3.6 35B now sits next to it, rated honestly, so you can see the difference yourself.

I added the newest model. I did not make it the default. The numbers, not the release date, get to decide that.

Prompt to Page is an independent project by Courtney Allen. It is not affiliated with, endorsed by, or connected to the Government Digital Service, the NHS, the Crown, or any UK government body. "GOV.UK" and "NHS.UK" are referenced descriptively.