Synthetic Data

Generating training data with models. It works when you can verify or filter the output, and fails quietly when you cannot.

Good training data is the constraint on most fine-tuning projects. Collecting and labelling it is slow and expensive.

So: have a model generate it. This works better than it sounds — and it fails in a specific way that is worth understanding before relying on it.

Where it genuinely works

The pattern that separates success from failure is verification. Synthetic data is reliable exactly where you can check the output.

Verifiable domains. Generate a math problem, generate a solution, check the arithmetic. Generate code, run the tests. Keep what passes, discard what fails. The generator can be unreliable because the filter is not — this is the strongest form of the technique, and it is what drove recent gains in reasoning models.

Format and style transfer. You have real inputs but no outputs in your desired format. Have a strong model produce them, spot-check a sample, use the rest. Format is easy to verify by inspection.

Augmenting real examples. Take genuine cases and generate variations — different phrasings, edge cases, adversarial versions. Anchored in real data, which limits drift.

Filling coverage gaps. Your real data has three examples of an important edge case. Generate thirty more.

Privacy-constrained work. When real data cannot be used for training, synthetic substitutes carry no personal information.

Where it fails

Unverifiable factual content. A model generating facts produces confident errors, and those errors become training targets. The student learns to be wrong in the teacher’s specific ways.

Distribution narrowing. This is the subtle one. Model-generated text is cleaner, more uniform, and less varied than human text. Train on it and the student learns that narrow distribution. Real input — with typos, truncation, mixed languages, and unusual phrasing — falls outside what it saw.

The failure is quiet: evaluation on synthetic held-out data looks excellent, production performance disappoints, and the gap is invisible unless you evaluate on real inputs.

Inherited bias. Whatever the generator prefers becomes what the student learns, amplified.

Recursive degradation. Training on outputs of models trained on outputs, repeatedly, narrows the distribution further each round. Diversity collapses. Real data anchoring each generation is what prevents this.

Making it work

Verify whatever can be verified. Run the code, check the math, validate the schema. This single practice separates useful synthetic data from harmful synthetic data.

Anchor in real inputs. Generate outputs for real inputs rather than generating both. Real inputs preserve the real distribution, which is the thing most easily lost.

Force diversity explicitly. Ask for variation in length, tone, and difficulty; vary the generating prompt across the set. Left alone, a model produces a narrow band.

Use a stronger generator than target. Distilling downward works; a same-size model teaching itself mostly reinforces existing tendencies.

Filter aggressively. Deduplicate near-identical examples, drop anything failing validation, and read a sample yourself. Discarding half the generated set is normal and usually improves the result.

Always hold out real data for evaluation. Non-negotiable. Evaluating only on synthetic data hides the exact failure this technique causes.

Mixing

Pure synthetic training data is rarely the best choice. Mixtures work better: real data establishes the true distribution, synthetic data fills gaps and adds volume.

Where real data exists in quantity, use it. Where it is scarce, synthetic augmentation around it beats either alone.

What to remember

• Synthetic data is reliable exactly where output can be verified — math, code, schemas, format.
• It fails on unverifiable facts, where generator errors become training targets.
• The quiet failure is distribution narrowing: model text is cleaner than reality, so the student underperforms on real input while scoring well on synthetic tests.
• Generate outputs for real inputs, force diversity explicitly, use a stronger generator, and filter hard.
• Always evaluate on real held-out data.

Next: Reading Model Benchmarks