Learning from generated images · Robot simulation
Can made-up images help a robot learn?
Train with different image sources, then try placing a carrot in a new simulator layout.
All four training methods failed on the new layouts. This recipe didn't show a benefit; that doesn't mean generated training data never helps.
Clip files & individual measurements
Cosmos 3 Nano training images
Rather controlled policy study
- Held-out placement success
- 0.0 %
Cosmos 3 Edge training images
Rather controlled policy study
- Held-out placement success
- 0.0 %
Each clip shows one registered input and seed. The full study includes every seed and failure.
Look for thisThe carrot needs to land on the plate and stay there after the robot lets go.
Every run, including failures (data file) (opens in a new tab)
Choose a scene or change the models
Examples stay in their registered order. Every seed and any failed runs remain in the full results.
Compare the numbers across every run
Cosmos 3 Nano training images − Cosmos 3 Edge training images. Uses every jointly scored case and seed for these two configurations, across the full study.
| Measurement | Left − right | 95% interval | Paired coverage |
|---|---|---|---|
| Held-out placement successpercentage points | 0.0 | No variation in resamples | 72 matched pairs6 groups · 0 excluded pairs |
A positive difference means the left model has a higher value; a negative difference means a lower value. Whether that is better depends on the metric. Excluded pairs include inapplicable or unscored measurements. These differences use matched runs and can differ from subtracting the displayed averages. Intervals that cross zero do not establish a clear direction of difference.
How we tested it — results, method & every run
Do generated training images improve robot control?
Does replacing half of reference-image training episodes with world-generated observations improve closed-loop placement on unseen native spatial layout groups?
24 cases · 3 seeds · 4 compared configurations · completed 2026-10-16
What we found. All four methods scored 0/72 on unseen layouts, versus 22–24/24 on familiar validation resets. Neither generated-image nor ordinary augmentation rescued layout generalization in this small policy setup. These tied results do not identify a winning method or establish that synthetic data never helps.
These clips show actual simulator control after training. Compare the same starting layout across training methods. A successful rollout must release the carrot onto the plate and leave it supported through the final second.
| Model configuration | Held-out placement success% · higher is better | Run errors / runs |
|---|---|---|
| Reference imagesbaseline | 0.0No variation in resamples6 groups · 72 scored runs | 0 / 72 |
| Ordinary image augmentationbaseline | 0.0No variation in resamples6 groups · 72 scored runs | 0 / 72 |
| Cosmos 3 Edge training images | 0.0No variation in resamples6 groups · 72 scored runs | 0 / 72 |
| Cosmos 3 Nano training images | 0.0No variation in resamples6 groups · 72 scored runs | 0 / 72 |
Means weight each seed and scene/group equally. Descriptive 95% percentile intervals use 2,000 crossed seed and group bootstrap draws, seed 202610290; cases remain nested within groups and paired differences use the same draws. An unscored seed/group cell withholds the balanced aggregate instead of silently reweighting it. Scored observations, excluded pairs and failures remain visible. Small cohorts and evaluator bias limit these intervals.
When every resample gives the same result, the interval cannot describe unseen successes or failures. No observed variation does not establish certainty.
