All projects

An Adversarial Multi-Agent Pursuit Game for Locomotor Agility Pretraining

November 2026

CoRL 2026 · RoboLetics 3.0 WorkshopRunner-up, Best Paper

Nathan A. C. Hogg, Matt Hyatt, Neil Klingensmith, George K. Thiruvathukal

Humanoid robots learn to lunge, dodge, and dive by playing competitive tag against each other through league-based self-play.

Reinforcement LearningSelf-PlayPPOMuJoCo WarpmjlabHumanoid LocomotionUnitree G1
An Adversarial Multi-Agent Pursuit Game for Locomotor Agility Pretraining

Most humanoid locomotion controllers are trained either with carafully shaped dense rewards or by tracking reference motions. Both plateau: once the reward is maximizedm, the controller stops learning new behavior. We asked whether competitive self-play, the recipe behind superhuman play in Go, StarCrafy, and hide-and-seek, could keep a locomotion controller improving from nothing more than a sparse game outcome.

Highlights

  • Accepted to the RoboLetics 3.0 workshop at CoRL 2026 and named Best Paper runner-up.
  • Built a multi-agent competitive league environment for humanoids with PFSP opponent sampling.
  • Designed an adaptive role-sampling strategy that fixes pursuer/evader difficulty imbalance and stabilizes training.
  • Emergent lunging, dodging, diving, and sprinting from a sparse win/loss reward — no reference motions.

Gallery

Two simulated humanoids mid-chase, the evader diving out of reach
Pursuer (red) lunges while the evader (white) dives away — one of the emergent strategies.
Pursuer humanoid lunging toward the evader
A pursuer lunge learned purely from the tag outcome.
Evader humanoid dodging a pursuer
The evader sidesteps a close-range tag attempt.
League win-rate heatmap and Elo score over training
Left: win rate between league checkpoints (red = pursuer wins, blue = evader wins). Right: Elo of the online learner over 40,000 rounds.
Planar velocity probability density for dense-reward, pursuer, and evader policies
Velocity distributions: a dense goal-following baseline vs. the pursuer and evader after league play — each role learns a distinct movement strategy.
Win-rate heatmaps without and with adaptive role sampling
Without role sampling (left) the pursuer dominates; with it (right) both roles stay competitive.