Tuning Isaac Lab and ONNX Policies for Microduck

Deploying reinforcement learning (RL) policies directly onto budget-friendly hardware is always a challenge. The gap between digital physics engines and real-world mechanical quirks is notoriously wide. Microduck, an 800-gram, 25-centimeter bipedal robot developed by Pollen Robotics and Hugging Face, bridges this gap using a specialized sim-to-real training pipeline. Built on NVIDIA Isaac Lab and Newton, the pipeline generates stable walking policies for the $399 hardware.

Platform Specifications and Compute Scaling

To move these policies from simulation to physical servos, Isaac Lab runs highly parallelized environments. Developers can scale these setups across local GPUs or cloud clusters, depending on their training targets.

The reported performance figures come from different Microduck training configurations. The 1–2 hour training time is associated with the original mjlab workflow, while the 1.25 million steps-per-second result comes from NVIDIA's distributed Isaac Lab setup. These figures should therefore not be interpreted as a direct performance comparison.

Low-Level Actuator and Backlash Modeling

The biggest failure point in sim-to-real transitions usually comes down to actuator limitations and mechanical tolerances. Standard physics engines tend to assume perfect conditions, like instantaneous torque and zero gear slack. When loading those policies onto a real robot, they can fail to behave as expected. The Microduck pipeline addresses this by injecting physical motor and gear constraints directly into the physics engine.

    [Raw Policy Action Command]
                 |
                 v
    [Dynamixel XL330 Servo]
      BAM M6: Voltage, Sag, Friction
                 |
                 v
    [Passive Backlash Hinge Joint]
      1° Play Each Direction
      (2° Total Slack)
                 |
                 v
    [Post-Backlash Encoder Readings]
                 |
                 v
    [Policy Input: 61 Observations]

Better Actuator Models (BAM M6)

Instead of using basic joint-motor approximations, the Microduck simulation models the robot's 14 Dynamixel XL330 servos using Better Actuator Models (BAM). The original Microduck RL implementation uses the BAM M6 model to account for:

• Motor performance under varying physical loads
• Current and voltage saturation limits
• Battery sag and voltage drops
• Internal friction within the servos

By simulating these physical constraints, the training environment more closely reflects how the robot's actuators behave in real-world conditions.

Backlash Emulation

Mechanical slack in physical gearboxes can disrupt a walking gait. The original Microduck RL pipeline models this by introducing an unpowered, passive backlash hinge at each of the 14 actuated joints. These hinges allow approximately 1 degree of play in either direction, resulting in 2 degrees of total slack per joint.

By incorporating the effects of backlash into joint-position observations, the simulation exposes the policy to mechanical delays and differences between commanded and actual movement. This helps the robot learn walking behaviors that are more tolerant of physical hardware limitations.

In the Isaac Lab implementation, backlash modeling is supported through a dedicated simulation configuration.

Domain Randomization and Asset Variations

To make the policies resilient across different real-world conditions, the pipeline varies physical parameters across parallel simulation environments. These variations include:

• Ground friction coefficients
• Robot mass and center-of-mass distribution
• Battery voltage levels
• Command and sensor latency
• Joint inertia, encoder offsets, and mechanical backlash

By training under different combinations of physical conditions, the policy becomes less dependent on a single idealized simulation and better prepared for deployment on real hardware.

The original Microduck RL implementation also uses different robot model configurations depending on the training task:

• robot_walk.xml: Used for standard walking tasks, with simplified collision handling to reduce simulation overhead.
• robot_groundcontact.xml: Used for behaviors involving more extensive ground contact, such as getting up or sitting down, where more detailed collision modeling is required.

These configurations illustrate how simulation complexity can be adjusted depending on the behavior being trained.

Compute Environment and Deployment Runtime

The software stack relies on Python 3.12, using the high-performance package manager uv to manage dependencies such as PyTorch and SciPy.

When installing the software on resource-constrained systems, downloading large dependencies can sometimes cause network timeouts. Setting the environment variable UV_HTTP_TIMEOUT=600 increases the allowed download timeout, which can help when installing large packages. If the local development machine lacks a compatible NVIDIA GPU, the original Microduck RL workflow also supports offloading training to Hugging Face Jobs using the --hf-jobs flag.

         [Policy Execution System]
                    |
                    v
       [61 Sensor Observations]
                    |
                    v
             [ONNX Runtime]
                    |
                    v
       [14 Joint Target Commands]
                    |
                    v
          [Dynamixel Servos]
                (50 Hz)

The finished policy operates through a fixed input-output interface at 50 Hz:

• Inputs: A 61-dimensional observation vector representing the robot's state and control-related information.
• Outputs: A 14-dimensional vector containing joint-position targets.

Because compatible policies share the same input-output interface, the onboard runtime can switch between different behaviors without requiring a system reboot. These can include walking, recovery, sitting, or ball-kicking policies, provided the models follow the expected observation and action conventions.

After training, the policy can be exported in TorchScript or ONNX format. The onboard computer loads the exported policy.onnx file and uses its predictions to generate servo commands.

Developers can also submit compatible ONNX policies of up to 64 MiB to Microduck Arena on Hugging Face for evaluation and comparison against other controllers. This provides a shared environment for testing different policy architectures and training approaches.