Synthetic data is a complement, not a replacement

The argument for synthetic data is straightforward and largely correct: it is fast, it is cheap at the margin, it can be generated in exactly the distribution you want, and it sidesteps privacy constraints that make real data difficult to obtain in health, finance and other regulated domains.

The argument gets overextended when it becomes a claim that real-world collection is a solved problem you can generate your way past.

What generation is genuinely good at

• Rare classes that are expensive to wait for in production data
• Balancing a distribution that is heavily skewed
• Privacy-constrained domains where the real records cannot leave a boundary
• Simulation of conditions that are dangerous or impractical to stage
• Rapid iteration on a schema before committing to real collection

In each of those, synthetic data is not a compromise. It is often the better instrument.

What it cannot do on its own

A generator produces the distribution it was taught. It reproduces the regularities of its source data faithfully — including that source's blind spots — and it does not contain information about the world that was absent from its input.

The cases you most need are frequently the ones nobody anticipated. Those are, by construction, the cases a generator will not invent, because it does not know they exist. Real-world collection remains the only mechanism that introduces genuinely new information.

Blended is the normal case

In practice most teams end up somewhere sensible: real data as the anchor and the evaluation set, synthetic data for augmentation and for classes that are hard to source. The proportions vary by domain, and the important discipline is keeping the evaluation set real.

If your benchmark is generated by the same process that generated your training data, it will report the model doing well at reproducing that process. That is not the same as the model working.

A useful question

Before generating, ask what specifically is missing. If the answer is "more of what we already have", generation will help. If the answer is "we do not know what we are missing", it will not — that is a collection and error-analysis problem, and no generator can solve it from the inside.

Keep the evaluation set real

This is the rule worth defending hardest. If both training and evaluation data come from the same generator, the benchmark measures how well the model reproduces that generator — which will look excellent and tell you almost nothing about deployment.

A real held-out set, however small, is what keeps the number honest. It is also usually the cheapest part of a data budget, because evaluation sets do not need to be large. They need to be representative.

Label the origin

Every example should record whether it was collected or generated, and if generated, by what process and version. Without that field, the ratio of real to synthetic drifts silently across batches and nobody can reconstruct which model saw what.

It also makes ablation possible. The question of whether the synthetic portion is helping is answerable in an afternoon if the provenance is recorded, and effectively unanswerable if it is not.