WM-LOCO World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain

  • Yuxi Liu1,*
  • Lijun Han1,*
  • Ziming Wang1,2
  • Ao Zhang1,3
  • Cong Yang4
  • Wei Sui1,†

1D-Robotics 2Beijing University of Posts and Telecommunications 3Harbin Institute of Technology 4Soochow University

*Equal contribution †Corresponding author

Demo · 2:12 Onboard runs on all four terrain classes, time-aligned REAL vs. SIM comparisons, simulation runs, and world-model open-loop prediction.

01

Framework

WM-LOCO framework: (a) a PPO policy with a mixture-of-experts actor–critic, (b) an RSSM world model, and the simulation that supplies on-policy data and task rewards
Fig. 2 WM-LOCO jointly trains an RSSM and a PPO actor–critic; the recurrent feature supplies predictive context to the policy. Proprioception and the velocity command enter the actor–critic directly, while depth enters only through the world-model feature fWMt. The PPO baseline omits only the world-model pathway and its auxiliary loss.

Abstract

Foothold-constrained terrain limits feasible footholds to sparse, discontinuous, or narrow support regions, as in gaps, stepping stones, stairs, and balance beams. In such settings, a single misplaced step can cause a fall. Reliable traversal therefore requires precise foot placement and anticipation of upcoming terrain.

To address this challenge, we present World-Model-Augmented Visual Locomotion (WM-LOCO), which augments a visual locomotion policy with predictive context from a jointly trained world model. Given onboard depth observations and proprioception, a recurrent state-space model (RSSM) learns action-conditioned latent dynamics that predict future proprioceptive states, egocentric observations, and task rewards. The resulting recurrent feature is then fed into the proximal policy optimization (PPO) actor–critic, providing predictive context for anticipatory foot placement on foothold-constrained terrain.

WM-LOCO achieves average success rates of 96.0% on gaps, 96.9% on stepping stones, 94.0% on stairs, and 92.1% on balance beams, outperforming the evaluated baselines while improving gait efficiency and smoothness. Furthermore, WM-LOCO achieves zero-shot sim-to-real transfer to a Unitree G1, with 92.5% success across these four terrain classes.

02

Real-World Demo

The policy runs onboard a Unitree G1, using proprioception and a single Intel RealSense D435 depth stream, without additional tuning. Ten trials per terrain class: 37/40 successful (92.5%).

Stepping stones

0.25 m stone tops, 0.45 m gaps

10/10

Gap

A 0.8 m gap between two platforms

9/10

Balance beam

3 m long, 0.10 m wide · front and side views

9/10

Stairs

0.15 m risers, 0.25 m treads, up and down

9/10

03

Simulation Results

Success rates in Isaac Lab across four terrain classes at Easy, Medium, and Hard difficulty.

Terrain Difficulty PPO GLAD CReF WM-LOCO Real
StairsEasy87.0%95.3%95.0%94.2%90%
Medium91.4%88.7%91.3%95.7%
Hard91.3%60.3%64.7%92.0%
Mean89.9%81.4%83.7%94.0%
GapEasy0.0%98.0%93.7%98.0%90%
Medium0.0%99.0%78.3%100.0%
Hard0.0%78.7%0.0%90.0%
Mean0.0%91.9%57.3%96.0%
Stepping stonesEasy0.0%76.0%94.3%97.1%100%
Medium0.0%64.7%94.0%97.3%
Hard0.0%7.7%88.7%96.4%
Mean0.0%49.4%92.3%96.9%
Balance beamEasy0.0%99.0%95.3%99.0%90%
Medium0.0%90.7%94.7%96.7%
Hard0.0%0.0%41.1%80.7%
Mean0.0%63.2%77.0%92.1%

Table 1 Success rates (%) for four methods and four terrains. Mean is the arithmetic average of Easy, Medium, and Hard; bold denotes the best simulation result in each row. Real reports WM-LOCO hardware success at one fixed difficulty per terrain, over 10 trials per terrain class.

Baseline Comparisons

WM-LOCO is the only method whose mean success exceeds 90% on every terrain class.

Radar chart of success rates averaged across difficulty tiers for PPO, GLAD, CReF, and WM-LOCO on stairs, gaps, stepping stones, and balance beams

Fig. 4 Success rates averaged across difficulty tiers. PPO achieves 0% success on gaps, stepping stones, and balance beams.

Ablation Study

Removing the world model reduces gap success to 0% for every width from 0.4–0.5 m onward.

Gap-traversal success versus gap width for the full model and for variants without the world model, history encoder, or MoE

Fig. 5 Gap-traversal success of the full model and of variants without the world model, history encoder, or MoE.

Combined Course

One episode chaining all four terrain classes: 95.4% success in simulation.

Keyframes of WM-LOCO on a combined simulation course: stair ascent, stepping stones, gap crossing, narrow beam, and stair descent

Fig. 1(d) Stair ascent, stepping stones, a gap, a balance beam, and stair descent in one simulated episode.

World-Model Validation

Given recorded actions, the world model predicts 15 updates (1.5 s) ahead without future observations.

Multi-step world-model predictions on stepping stones: ground-truth and predicted depth sequences, depth prediction error, and proprioception prediction error over a 1.5 s horizon

Fig. 3 Multi-step world-model predictions on stepping stones: (a) depth sequences; (b, c) depth and proprioceptive errors. Rollouts receive no future observations.

04

Citation

@misc{liu2026wmloco,
  title         = {World-Model-Augmented Visual Locomotion for
                   Humanoids on Foothold-Constrained Terrain},
  author        = {Liu, Yuxi and Han, Lijun and Wang, Ziming and
                   Zhang, Ao and Yang, Cong and Sui, Wei},
  year          = {2026},
  eprint        = {2609.02542},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO}
}