Skip to content

Physical AI: how robots learn in simulation first, tested in MuJoCo

20 Nov 2026

·

8 minute read

Physical AI is AI that acts in the physical world through a robot or a machine, and most of its training happens in simulation first, because a simulator gives hours of practice in seconds with nothing to break. The risk is the reality gap: a controller learns the simulator, and the real machine is different. The standard fix is to randomise the simulator's physics during training so the real machine looks like one more variation (Peng et al., 2017). We tested both halves on a laptop with the MuJoCo physics engine. One training run gave a balancing controller about 7 hours of simulated practice in 21 seconds. Trained on default physics, it coped with a heavier pole, a weaker motor, sensor noise and friction, and fell over almost every time once its commands arrived 40 ms late. Trained on randomised physics, it handled 40 ms every time, and only some training runs survived 60 ms, beyond the range they had practised on.

We trained a balancing robot in MuJoCo on one laptop core

The robot is a cart-pole: a 1 kg cart on a rail with a 0.5 m pole hinged on top, pushed left and right by a motor of up to 10 N. The controller has to keep the pole within about 11 degrees of upright and the cart within 1 m of the centre for 10 seconds, starting from a small random tilt. It is a classic first task in robot control, small enough to run in seconds, which let us repeat everything five times.

The controller is four numbers that weigh the cart's position, the pole's angle and how fast each is changing, and it decides the motor force 50 times a second. We trained it with basic random search, a simple method that tries small random changes to the numbers and keeps the ones that balance longer (Mania et al., 2018). Each training run was 2,880 attempts.

We trained it two ways, five times each with different random seeds:

  • on default physics only;
  • on randomised physics, where every attempt draws a new pole mass (half to double), cart mass, motor strength, a control delay of 0, 20 or 40 ms, sensor noise and rail friction.

There is no real robot in this test. The "real machine" is stood in for by physics changes the controller never trained on, one at a time and then all together. Everything ran on one core of an Apple M4 laptop with MuJoCo 4.0.1, installed for this test only.

Simulation gave 7 hours of practice in 21 seconds

On one core, MuJoCo ran 368 ten-second balancing attempts in 3.01 seconds: about 1,200 seconds of simulated time for every second of real time. A whole training run on default physics, 2,880 attempts, simulated between 26,292 and 27,407 seconds, about 7.3 to 7.6 hours of practice, in 21 to 22 seconds of wall time. The randomised runs simulated 6.6 to 7.2 hours in 24 to 27 seconds. Attempts end early when the pole falls, which is why the simulated time is less than 2,880 times 10 seconds.

Those numbers are the case for simulation-first. Thousands of falls cost nothing, nothing wears out, and every run can be repeated exactly. On a physical rig the same 2,880 attempts would also need someone to reset the pole after each fall.

The controller trained on default physics fell over with a 40 ms delay

We tested every trained controller 100 times on each condition. The table adds up the five controllers of each kind, so each cell is out of 500.

Test conditionTrained on default physicsTrained on randomised physics
Training physics (default)500500
Pole 1.8 times heavier500500
Motor 30% weaker500500
Sensor noise500500
Rail friction 0.4 N500500
40 ms control delay6500
60 ms control delay, beyond the randomised range0206
All the changes together, with 40 ms delay254500

Bar chart from our physical AI simulation test in MuJoCo, successful balancing attempts out of 500: trained on default physics, 500 on its own physics but 6 with a 40 ms control delay; trained on randomised physics, 500 at 40 ms and 206 at 60 ms.

The heavier pole, the weaker motor, the noise and the friction made no difference to this task: every controller of both kinds balanced all 100 times. The delay was the gap. A controller that reacts to the pole as it was two control cycles ago keeps correcting for a fall that has already moved on, and every default-trained controller fell in at least 94 of its 100 attempts.

The combined test was the surprise. Three of the five default-trained controllers did better with every change at once than with the delay alone, at 100, 92 and 62 out of 100. Our reading, which we did not test further, is that the weaker motor softens the controller's corrections, and softer corrections tolerate delay better. A test that only ran the combined case would have hidden how fragile those controllers were to delay on its own. The one-change-at-a-time rows found it.

Randomising the physics fixed it, inside the range it trained on

Every controller trained on randomised physics balanced all 100 times with a 40 ms delay, and it gave up nothing on the default physics. The randomised controllers also settled on different numbers from the default ones: a stronger response to the pole's angle and a weaker response to how fast it is falling.

Randomisation covered what we randomised, and not much more. The training delays stopped at 40 ms. At 60 ms, two randomised controllers still balanced all 100 times and three managed 0, 2 and 4. Same method, same training budget, and the result depended on the random seed. If we had trained one controller and tested it once, we could have reported either answer.

This is the pattern the research describes at larger scale. Peng and colleagues randomised the dynamics of a simulated robot arm and moved a pushing skill to the real arm without training on it, and they report the policies stayed robust to calibration error (arXiv 1710.06537). Earlier, Tobin and colleagues randomised what a simulated camera saw and trained an object detector on simulated images alone that worked on real ones (arXiv 1703.06907).

What to do before a physical AI controller meets the real machine

Our test is small, but it points at the checks we would run on any project that trains a robot or machine controller in simulation first, from a cobot cell to the kind of robot we priced in our humanoid robot vs cobot comparison:

  1. Measure the real machine before setting the training ranges: the delay from sensor reading to motor response, the masses, the motor strength, the sensor noise. In our test the delay alone decided the outcome, and a simulator has no delay unless you add one.
  2. Randomise past what you measured, and test at and beyond the edge of the range. Our 60 ms row shows what happens just outside it.
  3. Train several seeds and test each one. Ours split from 0 to 100 on the same test.
  4. Test one change at a time as well as all together. The combined test hid the delay problem for three of five controllers.
  5. Treat the simulator as a filter. It rules out controllers that would fail; it does not prove one will work. The first runs on the real machine still need someone ready to stop it.

What this test does not show

This is a cart-pole with a four-number controller, a long way from a humanoid, a robot arm or a model that sees through a camera. The "real machine" is more simulation, so the test shows how a reality gap behaves, not how large any particular real gap is. It covers one task and one training method, and the conclusions about delay, seeds and edges are what we would check first, not a guarantee for other robots.

Frequently asked questions

What is physical AI? AI that senses and acts in the physical world through a robot or machine, such as a robot arm, a mobile robot or a humanoid, as opposed to AI that only produces text, images or decisions on a screen.

Why do robots learn in simulation first? Because simulated practice is fast, cheap and safe. In our test, one laptop core gave a controller about 7 hours of practice in 21 seconds, and the 2,880 falls along the way cost nothing. The real machine is then used to confirm the result rather than to learn from scratch, a split that work such as Peng et al., 2017 depends on.

What is domain randomisation? Training in a simulator whose physics or appearance changes on every attempt, so the real world looks like one more variation. It was used on simulated camera images by Tobin et al., 2017 and on robot dynamics by Peng et al. In our test it fixed a 40 ms control delay completely and carried only partly to 60 ms, outside the range it trained on.

AI tools

Ask ChatGPTAsk Claude

Start with the problem that matters