Reinforcement Learning for Quadruped Walking
Training a four-legged robot to walk smoothly across different terrain in simulation, then deploying the learned controller on a real Unitree Go1.
UMass course project
- Build a stable walking controller that works across different terrain and can transfer to a real robot.
- Train policies in simulation across varied terrain, using rewards that encourage smoother, symmetric walking.
- Stable walking on flat and rough terrain, followed by deployment on a Unitree Go1.
Baseline walking without depth sensing.
Harder surfaces where gait quality and balance matter more.
Depth observations give information about upcoming terrain that a blind policy cannot see.
Teaching the robot to walk
The controller is learned with reinforcement learning in Isaac Lab. Many parallel simulated robots train at once on GPU. Rewards push for velocity tracking, a steady body height, and smooth actuation so the robot does not default to bouncing or “pronking.”
Why terrain matters
Training only on flat ground often produces irregular jumping gaits. Adding structured terrain during training encourages more natural, symmetric walking.
Irregular jumping / less stable gait.
More symmetric, steadier walking.
From simulation to the Go1
After training in simulation, the policy was transferred to a physical Unitree Go1 Edu.
Technical details
Training used diverse terrains (flat, rough, rails, waves). Reward design emphasized velocity tracking, base height, and smooth actuation. Depth-camera trials explored stairs and obstacle navigation on top of the blind policy.