{"author":{"@type":"Person","name":"Alexander Bock","url":"https://abxk.ai/en/autor/alexander-bock/"},"categories":["Physical AI"],"content":{"markdown":"Robots that learn by trial and error need a lot of trials, often millions. In the real world, that would take years, wear out the hardware and break things along the way. That is why most robot learning today happens in **simulation**, where a computer can run thousands of practice attempts per minute. The problem: no simulation matches reality exactly. Motors are a bit weaker, sensors a bit noisier, signals a bit later. A strategy that works perfectly in the simulator can fail on the real robot. This is called the **sim-to-real gap**.\n\nThis is part 4 of Level 2 in our learning path Physical AI. We built the classic balancing task, a pole on a moving cart, and let two strategies learn it with the same method: one in a single, fixed simulation, the other in simulations whose masses, lengths, delays and sensor noise changed randomly from attempt to attempt. Then we tested both in {{\u003c num \"physical_sims\" \"sr_tests\" \u003e}} different \"real\" worlds per condition. You will learn what this **domain randomization** does and why it works.\n\n## The task: keep a pole upright\n\nA cart can move left and right on a track, and a pole stands on it, held only by a hinge. Like balancing a broom on your hand, the cart has to move under the pole whenever it starts to fall. Fifty times per second, the strategy gets four measurements, the position and speed of the cart and the angle and turning speed of the pole, and decides how hard to push. An attempt counts as a success if the pole stays upright for ten seconds without the cart leaving the track.\n\n![Sketch of a cart on a track with a pole standing on it; arrows show the four measurements going to the strategy and the force going back to the cart; a second panel shows the same cart with a longer pole, a delay clock and a noisy signal as examples of differences in the real world](/images/learn/en/sim-to-real-domain-randomization-task.webp \"The balancing task and the differences between simulation and reality we tested.\")\n\n| What changes | Fixed simulation | Random practice worlds |\n|---|---|---|\n| Mass of the cart | Always 1 kg | 0.5 to 2 kg |\n| Mass of the pole | Always 0.1 kg | 0.05 to 0.4 kg |\n| Length of the pole | Always the same | From a bit shorter to twice as long |\n| Delay of the measurements | None | Up to 40 milliseconds |\n| Sensor noise | None | From none to strong |\n\n## How the strategies learned\n\nBoth strategies are simple: four numbers that say how strongly to react to each measurement. They learned with the **cross-entropy method**: try 48 random strategies, keep the best eight, try new ones that resemble those, and repeat. After {{\u003c num \"physical_sims\" \"sr_iters\" \u003e}} rounds, both kept the pole upright for the full ten seconds in their own training worlds. In training, you could not tell them apart.\n\n## The test in the \"real world\"\n\nThe difference showed up in the test. As long as the test world matched the fixed simulation, both strategies succeeded in every attempt. A longer or heavier pole caused hardly any problems either. But when the measurements arrived 60 milliseconds late, the strategy from the fixed simulation succeeded in {{\u003c num \"physical_sims\" \"sr_delay60_nominal\" \u003e}} of {{\u003c num \"physical_sims\" \"sr_tests\" \u003e}} attempts, the strategy from the random worlds in {{\u003c num \"physical_sims\" \"sr_delay60_randomized\" \u003e}}. With strong sensor noise, it was {{\u003c num \"physical_sims\" \"sr_noise_nominal\" \u003e}} against {{\u003c num \"physical_sims\" \"sr_noise_randomized\" \u003e}}. In a random mix of all deviations, the closest thing to a real robot, the strategy from the fixed simulation succeeded in {{\u003c num \"physical_sims\" \"sr_mixed_nominal\" \u003e}} of {{\u003c num \"physical_sims\" \"sr_tests\" \u003e}} worlds, the one from the random worlds in {{\u003c num \"physical_sims\" \"sr_mixed_randomized\" \u003e}}.\n\n{{\u003c data \"sr-tests\" \u003e}}\n\n![Line chart: share of successful attempts against the delay of the measurements from 0 to 80 milliseconds; the strategy from the fixed simulation drops sharply from 40 milliseconds, the strategy from the random practice worlds stays at 100 percent up to 60 milliseconds and only fails at 80](/images/learn/en/sim-to-real-domain-randomization-delay.webp \"Success against measurement delay. Both strategies fail beyond what they ever practiced.\")\n\nWhy? The strategy from the fixed simulation learned to rely on perfect, instant measurements and reacts sharply. Each sharp reaction to a late measurement comes too late, and, as in the feedback loop of Level 1, the corrections start to swing. The strategy from the random worlds never had that luxury. It learned to react more calmly, which costs nothing when the world is perfect and saves it when the world is not.\n\n## The limits of randomization\n\nThe chart also shows the limit. At a delay of 80 milliseconds, twice as long as anything in the practice worlds, the strategy from the random worlds failed too. Randomization makes a strategy robust within the range it practiced, not beyond. It also has costs: training needs more attempts, and a strategy that has to work everywhere may be slower or more careful than one tuned for a single, well-known robot.\n\nIn practice, robot builders therefore combine several tools:\n\n- **Measuring the real robot** to set the ranges of the simulation sensibly, for example by timing the real delays.\n- **Randomizing what matters**, such as friction, masses, delays, lighting and camera positions for robots that see.\n- **Fine-tuning on the real robot** with a small number of real attempts after the training in simulation.\n- **Learning from many robots** at once, an idea behind the robot foundation models that this learning path covers in Level 0.\n\n## How we ran the experiment\n\nWe implemented the classic equations of the cart and pole in Python with numpy, with time steps of 20 milliseconds and a maximum force of 10 newtons. Each strategy is a linear rule with four numbers. Training ran for {{\u003c num \"physical_sims\" \"sr_iters\" \u003e}} rounds with 48 candidates of five attempts each. For every test condition, we ran {{\u003c num \"physical_sims\" \"sr_tests\" \u003e}} attempts with slightly different starting positions and a fixed random seed. The next part returns to the brain: what makes a decoder for brain signals better or worse.","text":"Robots that learn by trial and error need a lot of trials, often millions. In the real world, that would take years, wear out the hardware and break things along the way. That is why most robot learning today happens in simulation, where a computer can run thousands of practice attempts per minute. The problem: no simulation matches reality exactly. Motors are a bit weaker, sensors a bit noisier, signals a bit later. A strategy that works perfectly in the simulator can fail on the real robot. This is called the sim-to-real gap.\nThis is part 4 of Level 2 in our learning path Physical AI. We built the classic balancing task, a pole on a moving cart, and let two strategies learn it with the same method: one in a single, fixed simulation, the other in simulations whose masses, lengths, delays and sensor noise changed randomly from attempt to attempt. Then we tested both in 100 different \u0026ldquo;real\u0026rdquo; worlds per condition. You will learn what this domain randomization does and why it works.\nThe task: keep a pole upright A cart can move left and right on a track, and a pole stands on it, held only by a hinge. Like balancing a broom on your hand, the cart has to move under the pole whenever it starts to fall. Fifty times per second, the strategy gets four measurements, the position and speed of the cart and the angle and turning speed of the pole, and decides how hard to push. An attempt counts as a success if the pole stays upright for ten seconds without the cart leaving the track.\nThe balancing task and the differences between simulation and reality we tested. Trained in Simulation: Why Robots Need Messy Practice Worlds What changes Fixed simulation Random practice worlds Mass of the cart Always 1 kg 0.5 to 2 kg Mass of the pole Always 0.1 kg 0.05 to 0.4 kg Length of the pole Always the same From a bit shorter to twice as long Delay of the measurements None Up to 40 milliseconds Sensor noise None From none to strong How the strategies learned Both strategies are simple: four numbers that say how strongly to react to each measurement. They learned with the cross-entropy method: try 48 random strategies, keep the best eight, try new ones that resemble those, and repeat. After 25 rounds, both kept the pole upright for the full ten seconds in their own training worlds. In training, you could not tell them apart.\nThe test in the \u0026ldquo;real world\u0026rdquo; The difference showed up in the test. As long as the test world matched the fixed simulation, both strategies succeeded in every attempt. A longer or heavier pole caused hardly any problems either. But when the measurements arrived 60 milliseconds late, the strategy from the fixed simulation succeeded in 0 of 100 attempts, the strategy from the random worlds in 100. With strong sensor noise, it was 57 against 100. In a random mix of all deviations, the closest thing to a real robot, the strategy from the fixed simulation succeeded in 51 of 100 worlds, the one from the random worlds in 97.\nSuccessful attempts out of 100 in the “real world” Test conditionTrained in one fixed simulationTrained with random variations As in the simulation100100 Pole twice as long97100 Pole four times as heavy100100 Measurement 40 ms late20100 Measurement 60 ms late0100 Strong sensor noise57100 Random mix of all deviations5197 Source as for the first table on this page.\nSuccess against measurement delay. Both strategies fail beyond what they ever practiced. Why? The strategy from the fixed simulation learned to rely on perfect, instant measurements and reacts sharply. Each sharp reaction to a late measurement comes too late, and, as in the feedback loop of Level 1, the corrections start to swing. The strategy from the random worlds never had that luxury. It learned to react more calmly, which costs nothing when the world is perfect and saves it when the world is not.\nThe limits of randomization The chart also shows the limit. At a delay of 80 milliseconds, twice as long as anything in the practice worlds, the strategy from the random worlds failed too. Randomization makes a strategy robust within the range it practiced, not beyond. It also has costs: training needs more attempts, and a strategy that has to work everywhere may be slower or more careful than one tuned for a single, well-known robot.\nIn practice, robot builders therefore combine several tools:\nMeasuring the real robot to set the ranges of the simulation sensibly, for example by timing the real delays. Randomizing what matters, such as friction, masses, delays, lighting and camera positions for robots that see. Fine-tuning on the real robot with a small number of real attempts after the training in simulation. Learning from many robots at once, an idea behind the robot foundation models that this learning path covers in Level 0. How we ran the experiment We implemented the classic equations of the cart and pole in Python with numpy, with time steps of 20 milliseconds and a maximum force of 10 newtons. Each strategy is a linear rule with four numbers. Training ran for 25 rounds with 48 candidates of five attempts each. For every test condition, we ran 100 attempts with slightly different starting positions and a fixed random seed. The next part returns to the brain: what makes a decoder for brain signals better or worse.\n","words":915},"date":"2026-10-18","description":"Robots learn in simulation, but reality is messier. Our experiment shows how training in many random practice worlds helps a robot survive delays and noise.","image":"https://abxk.ai/images/cover/sim-to-real-domain-randomization.webp","json":"https://abxk.ai/en/physical-ai/learn/sim-to-real-domain-randomization/index.json","kind":"page","language":"en","learning_path":{"category":"physical-ai","level":2,"level_name":"Intermediate"},"modified":"2026-10-18","section":"physical-ai","seo_title":"Sim-to-Real and Domain Randomization in an Experiment","tags":["sim-to-real","domain-randomization","robot-learning","simulation","experiment"],"title":"Trained in Simulation: Why Robots Need Messy Practice Worlds","translations":[{"json":"https://abxk.ai/de/physical-ai/learn/sim-to-real-domain-randomization/index.json","language":"de","url":"https://abxk.ai/de/physical-ai/learn/sim-to-real-domain-randomization/"}],"url":"https://abxk.ai/en/physical-ai/learn/sim-to-real-domain-randomization/"}