Atlas Learns to Move: How Reinforcement Learning Gave a Humanoid Robot Its Gait

Figure 1: Key milestones in the Atlas program, from the 2013 hydraulic DARPA platform to the 2024 all-electric redesign with reinforcement learning locomotion.

01 The Electric Atlas: A New Platform for Learning

The Atlas robot has undergone the most dramatic transformation in its decade-long history. The original Atlas, unveiled in July 2013, was a bipedal hydraulic humanoid developed by Boston Dynamics with funding and oversight from the U.S. Defense Advanced Research Projects Agency (DARPA). It was designed for search-and-rescue tasks and was initially tethered to an external power source, weighing over 150 kilograms. For over a decade, Atlas relied on hydraulically actuated joints—powerful, but heavy, noisy, and difficult to control with the precision that modern learning algorithms demand.

In April 2024, Boston Dynamics retired the hydraulic Atlas (internally designated HD Atlas) and announced a fully electric successor the very next day. The new electric Atlas is lighter, quieter, and designed from the ground up to be a platform for reinforcement learning. Its electric actuators offer finer torque control and higher bandwidth than hydraulic cylinders, making it possible for a learning algorithm to explore joint configurations and gaits that the older platform could not safely attempt. The video released by Boston Dynamics—titled "Walk, Run, Crawl, RL Fun"—showcases this new platform executing a range of locomotion behaviours that were learned rather than manually programmed.

02 Reinforcement Learning: The Engine Behind Autonomous Movement

Reinforcement learning is one of the three fundamental paradigms of machine learning, alongside supervised and unsupervised learning. In RL, an intelligent agent learns to take actions in a dynamic environment in order to maximise a reward signal. Rather than being told the correct answer for each situation—as in supervised learning—the agent discovers effective strategies through trial and error, balancing exploration of new actions against exploitation of known good ones. This exploration–exploitation dilemma is the central tension in every RL system.

For a humanoid robot, the "agent" is the control policy that maps the robot's observed state—joint angles, velocities, body orientation, contact forces—to motor commands. The "environment" is the physical world (or a simulation of it), and the "reward" is a scalar signal that encourages forward progress, balance maintenance, and smooth motion while penalising falls, energy waste, or joint-limit violations. Over millions of iterations, the policy learns to produce gaits that satisfy these competing objectives without any human explicitly coding the trajectories. The result is movement that can be remarkably natural and adaptive: the robot learns to recover from perturbations, transition between gaits, and even crawl when standing is not the best option.

03 From Simulation to Reality: The Sim-to-Real Challenge

Training a reinforcement-learning policy directly on a physical robot is impractical for humanoid locomotion. Each fall risks expensive hardware damage, and the thousands of iterations needed for RL to converge would take years of wall-clock time on a real robot. The standard solution is to train in simulation and then transfer the learned policy to the real robot—a process known as sim-to-real transfer. Boston Dynamics' video demonstrates that this pipeline now produces policies capable of walking, running, and crawling on the electric Atlas.

Sim-to-real transfer is not straightforward. Every simulation is an approximation of reality: friction coefficients are simplified, sensor noise is modelled rather than measured, and the dynamics of contact between a robot's foot and the ground are notoriously difficult to simulate accurately. If the simulation is too far from reality, a policy that performs perfectly in simulation fails catastrophically on the real robot—a problem known as the reality gap. Researchers narrow this gap through domain randomisation, where physical parameters such as mass, friction, actuator latency, and sensor noise are randomly varied during training. The policy that emerges is robust to a range of possible realities rather than overfit to one simulation's assumptions. The electric Atlas, with its clean torque-control interface and well-characterised dynamics, is a strong candidate for this approach.

Figure 2: The five-stage pipeline from physics simulation to physical robot deployment, illustrating the sim-to-real transfer workflow.

04 The Gait Spectrum: Walking, Running, and Crawling

The title of Boston Dynamics' video—Walk, Run, Crawl—signals a notable broadening of the locomotion repertoire. Walking and running are the canonical bipedal gaits, differing in whether there is a flight phase where both feet leave the ground. Crawling, however, is a fundamentally different mode: the robot lowers its centre of mass, uses its hands or arms for support, and moves with three or four contact points rather than two. A crawling gait is more stable and useful in confined spaces, on steep or slippery terrain, or when the robot needs to recover from a near-fall without fully collapsing.

What makes this repertoire notable is not that each gait exists in isolation—engineers have programmed walking and crawling robots for decades—but that a single learned policy can transition between them. A reinforcement-learning controller that has been trained on a reward landscape covering multiple locomotion modes can select the appropriate gait based on the current state: speed, terrain, perturbation, or command. This is qualitatively different from a finite-state machine that switches between hand-tuned controllers. The learned policy treats gait selection as a continuous control problem, producing smooth transitions that would be difficult to script by hand.

05 Why Humanoid Locomotion Is Harder Than It Looks

Humanoid bipedal locomotion is one of the hardest control problems in robotics. A biped is an inherently unstable system: its centre of mass sits well above a narrow base of support, and small disturbances—uneven ground, a sideways push, a momentary loss of traction—can quickly cascade into a fall. The dynamics are highly nonlinear, the contact between foot and ground is discontinuous, and the system has many coupled degrees of freedom. Classical control approaches, such as model predictive control with zero-moment-point balancing, can produce stable walking but require careful modelling and tend to produce stiff, conservative gaits that do not generalise well to unexpected terrain.

Reinforcement learning sidesteps some of these difficulties by not requiring an explicit model. Instead of solving the equations of motion, the learning algorithm discovers control strategies that work in practice, including strategies that a human engineer might not design. The trade-off is that RL policies can be opaque: it is difficult to guarantee safety, to understand why a policy chose a particular action, or to predict behaviour in situations the training distribution did not cover. For a humanoid robot operating near people, these guarantees matter. Boston Dynamics' long experience with model-based control provides a foundation for hybrid approaches that combine the adaptability of learned policies with the safety guarantees of classical methods.

06 The Broader Landscape: RL in Robotics Beyond Atlas

Boston Dynamics is not alone in applying reinforcement learning to legged locomotion. Academic and industrial labs worldwide have demonstrated RL-trained quadrupeds—four-legged robots like ANYmal, MIT Mini Cheetah, and Unitree platforms—that can walk, run, recover from falls, and traverse challenging terrain. Quadrupeds are easier than bipeds because they have a wider support base and can recover from disturbances by shifting weight across four legs. Humanoids represent the hard end of the spectrum, and Atlas is among the most capable platforms on which RL locomotion has been publicly demonstrated.

The broader trend is the migration of learning-based methods from simulation benchmarks into real-world robotic systems. RL has been used for dexterous manipulation, drone racing, autonomous driving, and robotic assembly. Each domain faces its own version of the sim-to-real gap and its own safety constraints. What the Atlas video suggests is that for at least one class of high-difficulty, high-stakes locomotion tasks—humanoid bipedal movement—learned policies have reached a level of competence that is worth showcasing publicly. This does not mean the problem is solved, but it does mean the frontier has moved.

07 Implications for Industry and Autonomous Labour

Boston Dynamics has been owned by the Hyundai Motor Group since 2021, and the company has stated its intent to commercialise robots beyond Spot, its quadruped. A humanoid robot that can learn to walk, run, and crawl autonomously has obvious implications for industrial settings: warehouses, construction sites, manufacturing floors, and logistics hubs where stairs, uneven surfaces, and confined spaces are common. A robot that can adapt its gait to the terrain without reprogramming could operate in environments that were previously too unstructured for a fixed-controller humanoid.

The economic significance is amplified by concurrent labour trends. Many industrial economies face persistent shortages in manual and physically demanding occupations. A humanoid that can be trained—rather than painstakingly programmed—for a new site or task could be deployed faster and at lower cost than a system requiring bespoke control engineering for each environment. Whether this translates into real deployment at scale depends on factors well beyond locomotion: battery life, durability, cost, safety certification, and the ability to perform useful manipulation tasks, not just movement. But the locomotion foundation that RL provides is a necessary precondition for the rest.

08 Limits, Risks, and the Road Ahead

Reinforcement learning for humanoid locomotion remains an active research frontier, not a mature technology. The policies shown in Boston Dynamics' video are demonstrations in controlled environments; their robustness in unstructured real-world settings—rain, ice, crowded factory floors, unexpected obstacles—has not been independently assessed. Sim-to-real transfer narrows the reality gap but does not eliminate it, and edge cases that fall outside the training distribution can produce unpredictable behaviour. For a 75-kilogram humanoid operating near people, an unpredictable behaviour is a safety hazard, not merely a performance limitation.

There are also broader questions about interpretability and accountability. If a learned policy causes a robot to fall or to strike a person, the policy is a neural network whose decision process is not transparent. Debugging a learned controller is fundamentally different from debugging a model-based one, where the equations and assumptions can be inspected. The robotics community is developing tools for policy verification, safe RL, and hybrid control that combine learned and model-based elements, but these are not yet standard practice in deployed systems. The Atlas video is a milestone in what RL can achieve for humanoid locomotion. The path from milestone to safe, reliable, everyday deployment is still long, and it will be shaped as much by engineering discipline and safety practice as by the next algorithmic breakthrough.