Imagine teaching a simple sorter to distinguish circle marks from square marks. You generate the training records yourself. Every circle happens to have a blue background, and every square has a yellow one. A rule that reads only the background can now look perfect, even though background colour is irrelevant to the task you meant to teach. Adding more records from the same recipe can preserve that convenient clue.

Synthetic training data is produced through a generation process, which can include a simulator, explicit rules or a learned model. One published example is Tremblay and colleagues’ 2018 object-detection work: its authors describe varying simulator settings such as lighting, pose and textures [1]. That study provides context for varying how examples are made; its results are separate from the small exercise here.

We will construct records containing only three bits: the correct class, an imperfect shape signal and a background signal. There are no pictures or real sorters. We will compare exactly two simple rules and count every prediction. This restricted setup makes it possible to see precisely how changing a generator changes which rule wins.

A synthetic dataset inherits a joint structure from its generation mechanism. Controlling the class frequencies does not automatically control the relationship between the label and every input feature. A generator can make a nuisance variable perfectly predictive, allowing a learner to satisfy its training objective through a relationship that the intended application does not preserve.

Tremblay et al.’s 2018 publisher abstract describes domain randomization over simulator parameters, including lighting, pose and texture, for a particular object-detection study [1]. We use that documented idea as context, without reproducing the network, images or reported benchmark. Our construction is a deterministic binary-feature exercise with a hypothesis family containing exactly two classifiers.

Let Y denote the target label, S an observed shape bit and B a background bit. Only S and B are available to a classifier at prediction time. The generation rules below specify all records, including observation errors. Exhaustively comparing the two candidates constitutes the entire selection procedure. There is no neural-network optimization, image rendering, stochastic sampling or model-provider call hidden behind the percentages. The result will be a property of these declared records and rules.

Use 0 for a square and 1 for a circle. The label says which mark the imaginary record represents. The shape bit is a separate observation that can be wrong. For example, a square has label 0, but its shape bit might be 1 because we deliberately inserted an observation error. The label has not changed merely because the signal is imperfect.

The background bit also uses 0 and 1: think of yellow and blue. Background is declared irrelevant to the target. Our first generator nevertheless assigns background 0 to every square and background 1 to every circle. This creates a relationship in the training table that was never part of the definition of a correct answer.

Name the two candidate rules Shape and Background. Shape returns the observed shape bit. Background returns the background bit. Neither reads the correct label when making a prediction. We use the label afterwards to count successes. Keeping these roles separate prevents a misleading demonstration in which a supposed classifier simply receives the answer as an input.

Define Y=0 for square and Y=1 for circle. The observed feature S is not ground truth: selected records deliberately have S=1−Y. B is a nuisance feature whose semantic meaning does not determine Y. The first generator sets B=Y, creating perfect empirical association despite that declared task semantics.

The candidate family is H={h_S,h_B}, with h_S(S,B)=S and h_B(S,B)=B. Labels are used only for scoring predictions. Row identifiers support bookkeeping and are not inference features. There is no third candidate, ensemble, learned feature extractor or access to the generator’s hidden label assignment. The comparison is therefore exhaustive within H, not an assertion about the best possible classifier.

This distinction also explains the observation errors. An incorrect S value is a feature error in the fictional measurement, not an incorrectly assigned Y value. We stipulate correct labels for this exercise. If a real generator produced incorrect labels, that would introduce another failure mechanism requiring separate checking. The current numbers isolate the relationship between a useful but imperfect feature and a perfectly correlated nuisance feature.

Build 20 square records and 20 circle records. For each class, let the shape bit be correct on 18 records and wrong on two. Every square receives background 0; every circle receives background 1. The table contains 40 records, with equal numbers of the two labels and equal numbers of the two backgrounds.

Those equal totals sound reassuring until you cross the two fields. Square with background 0 appears 20 times, while square with background 1 never appears. Circle with background 0 also never appears, while circle with background 1 appears 20 times. Two of the four possible class-and-background combinations are absent. Looking at each field’s total separately hides that structure.

The missing combinations are meaningful because our declared task permits either shape on either background. This is an original design assumption, not a statement about the frequency of real photographs. The example shows why a generation plan needs to identify which combinations matter. A balanced list of labels alone cannot tell you whether those combinations were represented.

The initial training manifest has N=40 records, twenty for each Y. Within each class, two S values are flipped and eighteen match Y. B equals Y throughout. The marginal counts are n(Y=0)=n(Y=1)=20 and n(B=0)=n(B=1)=20.

The joint counts are n(Y=0,B=0)=20, n(0,1)=0, n(1,0)=0 and n(1,1)=20. Thus both marginals are balanced while the joint support omits half of the four declared combinations. In this finite table, P_empirical(B=Y)=1. That empirical relationship follows from the recipe; it is not evidence that background determines shape in an external environment.

The task definition treats both background values as valid for either class. Under that definition, the empty off-diagonal cells expose a generation choice worth examining. They do not establish how common those cases are outside the exercise. Coverage analysis needs a declared space of relevant cases and a unit of counting. Here the space is only the two-by-two label/background grid; it says nothing about camera positions, textures or other unrepresented variables.

Score each rule on the 40 training records. Shape gets 18 squares and 18 circles right, making 36 correct predictions out of 40, or 90 percent. Background gets all 40 right because its input was copied from the label by the generator. Its training accuracy is 100 percent.

Our selection procedure chooses whichever of the two rules has the larger number of correct training predictions. It therefore chooses Background. The procedure has followed its instructions exactly. It was never told that colour should be ignored, and its candidate rules were judged only on records where colour and class always agreed.

That perfect score establishes agreement with this training table. It does not establish correct shape recognition under other background arrangements. Google’s introductory machine-learning lesson describes the distinction between training fit and performance on new examples, including problems caused by unrepresentative training data [4]. We can inspect one such mismatch directly here because the rules and every record are small enough to enumerate.

For candidate h, define training accuracy A_D(h)=N^−1 times the sum of indicators that h(S_i,B_i)=Y_i. Since both candidates are evaluated on the same N, maximizing the correct-count numerator is equivalent to maximizing A_D. The stipulated selector uses this criterion alone.

For h_S, the count is 18+18=36, giving A_D(h_S)=0.9. For h_B, the count is 20+20=40, giving A_D(h_B)=1. The selected rule is unambiguously h_B. No tie-breaking rule, optimization instability or sample-order effect is needed to obtain that result. Within the defined family, h_B is the empirical accuracy maximizer.

The interpretation remains local to D. Google’s teaching material separates training fit from new-data performance and identifies unrepresentative training data as a possible cause of failure [4]. In our fixture, the mismatch is specifically a changed background-label relationship. Calling it merely a bad algorithm would miss the mechanism: the objective and candidate set produce their stated result on the data supplied.

Now repeat the entire initial table ten times. This produces 400 rows: 200 squares and 200 circles. Shape is correct 360 times, and Background is correct 400 times. Their percentages remain 90 and 100. The same selection procedure still chooses Background.

This comparison means literal repetition of the 40 records. It is not a claim that 400 independent observations were collected. The purpose is to separate a larger row count from a changed training relationship. Multiplying all the counts by the same number leaves the relative scores unchanged. The two absent class-and-background combinations are still absent.

A new generator could produce additional variation, so this calculation does not prove that generating more data is generally useless. It shows what must change for this particular problem to move: the data must stop rewarding the background rule so strongly, or the task’s learning setup must change. Simply reporting a bigger total leaves that question unanswered. The recipe, including what it holds fixed, belongs beside the total.

Let D_repeat contain ten identical copies of every record in D. Its size is 400. The correct-count numerators become 10×36=360 for h_S and 10×40=400 for h_B, so both empirical accuracies are invariant. The selected hypothesis remains h_B.

Uniform duplication multiplies every candidate’s summed zero-one loss by ten and leaves its mean loss unchanged. Therefore it cannot alter the ordering of these two candidates under the stated selector. The joint counts become 200,0,0,200 in the same cell order as before. No new label/background combination enters the support.

This is an exact duplication result, not a claim about arbitrary synthetic scaling. A different generator, sampling distribution, feature set or learning objective could produce another outcome. Nor do the additional rows provide independent evidence about a population parameter. Their parent relationships are fully known. The useful diagnostic is whether the proposed increase changes the relevant distributional structure, rather than whether the file contains more lines.

Create a separate challenge table with ten records in each of the four class-and-background combinations. Each group has nine correct shape bits and one wrong shape bit. There are 40 challenge records in total. They use new identifiers, but they are still invented records, not photographs collected from a real deployment.

Shape gets nine right in each of four groups: 36 out of 40, or 90 percent. Background succeeds in the two groups where background equals the label and fails in the other two. Its result is 20 out of 40, or 50 percent. The initially selected rule therefore falls from perfect training agreement to half correct on this declared challenge.

The 50 percent figure is not a prediction of how a real sorter will perform. We deliberately built this table to expose dependence on background. It answers a narrow question: what do the two fixed rules do when both backgrounds occur equally often with both labels, while the shape signal retains its stated error count? Every part of that question is visible in the construction.

Define challenge manifest C with n_C(Y=y,B=b)=10 for every pair (y,b). In each cell, one S value is flipped and nine equal Y. C therefore has forty rows, thirty-six shape matches and twenty background matches. Its identifiers are disjoint from the training identifiers, but the construction is deterministic and does not assert statistical independence.

Evaluating the fixed candidates gives A_C(h_S)=36/40=0.9 and A_C(h_B)=20/40=0.5. The initial selector’s output h_B receives the latter score. The background marginal and label marginal remain balanced; it is their joint relationship that differs from D. This makes the mechanism more specific than an unexplained change in a headline score.

C is a designed stress fixture. It is not a randomly sampled holdout from a known deployment distribution, and its percentages carry no population confidence interval. The construction was chosen to illustrate the failure mode, not to provide an unbiased final evaluation after model development. A real application would need its own appropriate evaluation evidence beyond this finite calculation.

Change the training recipe. For each original record, make one version with background 0 and one with background 1. Keep that record’s label and observed shape bit unchanged in both versions. The two versions share a parent, so their relationship is explicit. This produces 80 training rows from the original 40.

Each square now appears in both background colours, as does each circle. The four class-and-background cells each contain 20 rows. Within a pair, the background rule is right once and wrong once. The shape rule makes the same prediction for both versions because its input has stayed the same.

We are allowed to preserve the label because background colour was declared irrelevant to this fictional task. That condition matters. The recipe changes a nuisance feature while holding the target meaning fixed. It is an exhaustive pairing operation, not a random draw. It also creates related records, so 80 rows should not be described as 80 separately observed objects. The change that matters here is the relationship between background and label.

Construct D_pair={(i,Y_i,S_i,b): i in D, b in {0,1}}. Retain parent_id=i for both descendants. The cardinality is 80, with twenty rows in each (Y,B) cell. This is equivalent to retaining each initial record and adding its background-swapped counterpart, because initial B_i=Y_i.

For each parent, one descendant has B=Y and one has B≠Y. By contrast, S=Y either holds for both descendants or for neither. The operation therefore breaks the initial perfect empirical label/background association while preserving the observed shape error rate. These are algebraic properties of the construction, not observations of a rendering engine.

Label preservation is an explicit semantic premise: changing B does not change the target mark. Parent pairing also establishes dependence between records. The procedure should not be described as eighty IID samples or a randomization experiment. It is a controlled enumeration of a declared nuisance variation. Its usefulness for another task would depend on whether the transformation actually preserves that task’s target and represents relevant conditions.

Score the two rules on the paired table. Shape was right on 36 original records, and each now has two versions, so it is right on 72 of 80 rows: 90 percent. Background is right on one row per pair, giving 40 out of 80: 50 percent. The same selection procedure now chooses Shape.

Evaluate that selected rule on the unchanged 40-row challenge. It gets 36 right, or 90 percent. The initial training recipe selected a rule with 100 percent training accuracy and 50 percent challenge accuracy. The paired recipe selects one with 90 percent training accuracy and 90 percent challenge accuracy. The lower training number is therefore not evidence that this change made the selected rule worse on the stated challenge.

Nothing about Shape itself improved. Its predictions were fixed from the beginning. Changing the recipe changed which of the two candidates the selector preferred. This is the central result of the exercise. It demonstrates a change in selection caused by a change in training relationships, with no need to invent a hidden model improvement.

On D_pair, h_S has 2×36=72 correct predictions and h_B has one correct prediction for each of forty parents. Their accuracies are 72/80=0.9 and 40/80=0.5. The empirical maximizer within H changes from h_B to h_S.

Holding C fixed, the selected-rule score changes from A_C(h_B)=0.5 to A_C(h_S)=0.9. The corresponding selected-rule training accuracy changes from 1 on D to 0.9 on D_pair. These are different training datasets, so the comparison should retain each denominator and recipe. The forty-percentage-point challenge difference is a descriptive difference between two deterministic outcomes, not an estimated treatment effect in a population.

Neither candidate’s function was updated. The entire effect operates through hypothesis selection: the data changes the empirical ranking. This distinction prevents the exercise from being mistaken for evidence that an unspecified neural network has learned a robust representation. Our conclusion is exact for the two fixed rules and the supplied manifests. More complex hypothesis families can use additional relationships absent from this construction.

Our paired recipe works on paper because changing a background leaves the intended shape unchanged. Other transformations may not have that property. Imagine a task that classifies whether an arrow points left or right. Reflecting the whole arrow horizontally changes the answer. Keeping the old label after that operation would create an incorrect training example, even if the transformed picture looked perfectly clear.

The same distinction applies to generated explanations or question-and-answer records. A neatly formatted answer can still fail the task it is meant to demonstrate. Check the target meaning as well as the appearance or file structure. In the circle-and-square exercise, we deliberately separated observation errors from label errors so that the source of each disagreement remained identifiable.

This gives a useful order for reviewing a proposed variation: describe what is being changed, state which part of the target should remain fixed, and check that this is true for the resulting record. If the answer should change, update it according to the task definition. Producing another valid file is only one part of producing another valid example.

A transformation T requires a label relation, not just a syntactic output specification. Our background substitution satisfies the stipulated invariance Y(T(x))=Y(x). A horizontal reflection in a left-versus-right arrow task instead requires label equivariance: the transformed input has the opposite direction label. Reusing the original label in that example would corrupt supervision.

The arrow case is an original semantic counterexample, not a measured augmentation study. It shows why transformations should be classified by their effect on the target. A generator may preserve a record’s schema while violating its meaning. Conversely, an input perturbation can legitimately introduce observational difficulty without changing the underlying target, as our deliberate S flips do.

Checking generated data therefore needs to distinguish feature validity, label correctness and the relationship between the two. For this exercise, label correctness is assumed and all feature values are binary by definition. Those assumptions make the count calculation exact. They would need evidence in an actual image or text pipeline. A review system’s agreement should not silently replace the underlying task-specific validation.

After pairing, every class-and-background combination appears. That is complete coverage of the four cells we chose to represent. It says nothing about blur, camera angle, partial obstruction or an unfamiliar kind of mark. Those variables do not exist in our table. A claim that the sorter now handles every real condition would go beyond the example.

Google Research’s April 2026 account of Simula distinguishes broad topic coverage, variation within a topic, difficulty and quality checks. It also reports that the effects of generation choices differed across its evaluated tasks [2]. Those distinctions are useful when planning a collection: adding variety, making examples harder and checking their answers are separate decisions.

For our recipe, the first concrete question was whether a circle could appear on either background. A larger task needs its own relevant questions. Record which conditions are deliberately varied, which are held fixed and which have not been represented. A long list of generated examples can still be narrow along an important dimension, while uncontrolled variation can make the intended labels difficult to justify.

Coverage is relative to the represented variable space. D_pair fills the support of the binary (Y,B) grid, but it adds no variable for occlusion, illumination or acquisition geometry. It does not prove robustness to those factors. Even within this grid, uniform cell counts encode a chosen weighting rather than an observed deployment prevalence.

The Simula authors’ April 2026 report separates global coverage, local variation, complexity and quality checking, with task-dependent outcomes for design choices [2]. We use those distinctions as planning context. Our exhaustive two-background pairing does not implement Simula, its critics or its evaluation metrics, and no reported performance gain is transferred to this exercise.

A generation specification should make its varied and fixed factors explicit, together with any constraints on combinations. For the present fixture, the permitted combinations and uniform counts are declared in advance. In another application, some combinations may be impossible or irrelevant. Filling a combinatorial grid indiscriminately could then produce invalid data. Coverage has to be defined in relation to the target task and supported conditions, not merely maximized as a row-count objective.

A generated record can look believable, differ from other records and still be unhelpful for the intended learner. It can also repeat a source record closely. These are different questions. Alaa and colleagues’ 2022 paper distinguishes fidelity, diversity and copying-related generalization in its proposed evaluation framework [3]. Our small exercise does not calculate those formal metrics.

What we do calculate is narrower: correct predictions for two named rules on two training recipes and one declared challenge. This is a downstream outcome for an artificial task. It does not establish image realism, originality, privacy or the performance of a real model. The numbers become useful because their question is clear, not because they can stand in for every other quality measure.

For a real project, decide what the generated data is meant to improve and evaluate that use with suitable evidence. Keep the generation process, the learner and the evaluation conditions visible. A successful check of file format can coexist with a failed label check; a strong training result can coexist with a weak result under changed conditions. Each deserves its own recorded outcome.

Alaa et al.’s 2022 abstract separates fidelity, diversity and generalization associated with copying training data as dimensions of generative-model evaluation [3]. Those dimensions provide context for resisting a single undifferentiated quality score. We do not implement their alpha-precision, beta-recall or authenticity estimators.

The present utility calculation is conditional on H, the selection objective, D or D_pair, and C. Changing any of those objects could change the conclusion. Neither high candidate accuracy nor complete support of the chosen binary grid establishes fidelity to a physical image distribution. No privacy property follows from the fact that these records were generated, and no privacy metric is being reported.

An actual assessment should distinguish properties of the generated sample collection from properties of a model trained with it. The former can include valid labels and relevant coverage; the latter requires measured outputs under a stated protocol. Because C was constructed to explain a known mechanism, it cannot stand in for independent external validation. Its value here is transparent diagnosis: the two rules’ predictions can be traced directly to every declared input.

The result is reproducible when the record includes the task definition, the three fields, the generation rules, the two candidate rules and the scoring method. Keep the original 40 rows, their 80 paired descendants and the 40 challenge rows distinct. For the ten-copy comparison, say plainly that the rows were repeated. No random seed is needed because every assignment is deterministic.

The supported conclusion is specific: under the declared selector, the initial recipe chooses Background, while pairing both backgrounds chooses Shape. On the stated challenge, those selected rules score 20 out of 40 and 36 out of 40. A reader can verify the change without trusting an unexplained claim that the data became better.

That is a useful standard for a synthetic-data experiment report. Describe what changed in the recipe and which result changed with it. Preserve the assumptions about valid labels and relevant conditions. Then identify the evidence still needed for the intended real use. The larger file is easy to count; the meaningful question is what new relationship its examples actually teach.

A minimal reproducible artifact for this exercise contains the schema, target semantics, deterministic record construction, parent relation, hypothesis family, selection criterion and evaluation manifest. Record the versions of those objects together. There is no stochastic seed, external generator version or physical acquisition setting to invent.

The exact findings are argmax_H A_D=h_B, argmax_H A_D_pair=h_S, A_C(h_B)=0.5 and A_C(h_S)=0.9. Uniform tenfold duplication preserves the first selection. The paired construction changes the empirical label/background relationship while retaining the specified shape error rate. These statements exhaust the demonstrated selection effect; they do not establish universal synthetic-data efficacy.

For a later real experiment, retain the actual generator and learner configurations, label-validation procedure and evaluation conditions required to interpret its measured outputs. A changed recipe should have an identifiable revision, allowing a result to be associated with the data that produced it. The purpose of the record is to expose the mechanism and the evidence boundary. It makes a claim about improvement inspectable instead of leaving dataset size to carry more meaning than its counting unit supports.