Migrating Between Models

New model, better benchmarks, worse output. Prompts are tuned to models, and migration is a real project rather than a config change.

A better model is released. You change the model identifier. Output gets worse in ways you did not anticipate.

This is normal, and the reason is that prompts are tuned to models. Every constraint you added to fix a specific failure was fixing that model’s specific failure. The new model has different ones.

What changes

Verbosity. The most common surprise. A model producing noticeably longer or shorter answers breaks length-sensitive UI and shifts cost in either direction.

Format adherence. A model that reliably produced clean JSON may start wrapping it in prose, or vice versa. If you built parsing around one model’s habits, that parsing is now load-bearing on the wrong assumption.

Instruction interpretation. The same wording gets read differently. Constraints that were implicit for one model need stating for another.

Refusal boundaries. Alignment tuning differs between models and between versions, so a request handled fine before may now be declined — or the reverse.

Reasoning behaviour. Newer models may reason before answering by default. That changes latency, cost, and output shape, and your prompt asking for step-by-step work may now be redundant or actively interfering. See How Reasoning Models Work.

Tokenization. Different vocabulary means different token counts for identical text, so your cost estimates and context window headroom both shift.

Why your prompt is model-specific

Look at any mature production prompt and it contains lines added to fix particular failures: do not include a preamble, always use double quotes, if the answer is not in the context, say so.

Each was a patch for one model’s tendency. On a new model some are unnecessary — harmless but wasteful — and some now cause the problem they were preventing. Over-specified constraints can make a capable model worse by fighting its defaults.

Which is why the comment habit matters: a note saying added because it invented citations tells you whether the line is still needed. Without it, migration means guessing at every constraint.

The migration sequence

1 · Baseline the current model on your eval set. You cannot detect a regression without a number to compare against.

2 · Run the new model, unchanged prompt. Sometimes it is simply better. Frequently it is worse in one specific dimension, and that dimension is your work.

3 · Diagnose by category. Format failures, verbosity changes, refusals, and reasoning differences need different fixes. Group them rather than treating each case individually.

4 · Remove constraints before adding them. Try deleting patches that addressed old failures. This is counterintuitive and often the whole fix — a shorter prompt frequently works better on a newer model.

5 · Re-baseline both. Confirm the adjusted prompt on the new model beats the old prompt on the old model. Confirm it also has not broken the old model, if you need a fallback.

6 · Shadow, then percentage rollout. Real traffic finds what evals miss.

7 · Keep rollback trivial. A config change. Do not delete the old prompt version.

Silent migration

The version you did not choose to change: providers update models under stable version names.

Your prompts were tuned against behaviour that shifted without notice. This is why scheduled eval runs matter even when nothing on your side changed — they are the only detection mechanism, and the symptom otherwise appears as user reports weeks later.

Practically: run your eval on a schedule, alert on score drops, and treat the results as a change-detection signal rather than only a pre-deploy gate.

Reducing future pain

Pin versions explicitly rather than using aliases that float to the latest.

Keep prompts minimal. Fewer constraints means less to re-tune. Over-specification is a migration liability.

Use schema-constrained generation rather than prompt instructions for format. Enforcement transfers between models; wording does not.

Maintain the eval set as an asset. It is what makes migration a measured project instead of a guessing exercise, and it pays for itself the first time.

Abstract the provider call behind one interface, so switching is a configuration change rather than a code change scattered across features.

What to remember

• Prompts are tuned to specific models; a version change is a real project, not a config edit.
• Expect changes in verbosity, format adherence, instruction interpretation, refusals, reasoning behaviour, and token counts.
• Try removing constraints first — old patches can actively hurt a newer model.
• Baseline, test unchanged, diagnose by category, re-baseline, shadow, roll out with easy rollback.
• Providers update models silently; scheduled eval runs are the only detection mechanism.
• Constrain format by schema rather than by wording, since enforcement transfers across models.

Next: Self-Hosting Models