Recently, Prof. Jianyu Chen’s research team from the Institute for Interdisciplinary Information Sciences, Tsinghua University has collaborated with the tech company Robot Era and achieved a breakthrough in the field of humanoid robot. The team proposed a novel learning framework for humanoid robots that effectively alleviates real-world noise from observations, enhancing the robots' adaptability in complex environments. This research, titled Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning (DWL), has been accepted by Robotics: Science and Systems (RSS), one of the most prestigious conferences in the field of robotics, and received a perfect score of 4.0 from all reviewers.
Modern environments are designed according to human needs, making humanoid robots particularly suitable for these settings. These robots excel in mobility, especially on complex terrains. Yet, traditional humanoid robot gait development relies heavily on model-based control techniques, which require precise dynamics modeling and struggle with complex interaction scenarios, increasing control complexity.
Recent advances in model-free reinforcement learning (RL) show great potential in creating versatile legged locomotion controllers, allowing robots to learn and adapt to diverse environments, often outperforming traditional methods. However, achieving robust motion control in humanoid robots is challenging due to their higher center of gravity, instability during leg swings, increased leg inertia, additional weight, and larger size. Previously, RL applications for humanoid robots in real-world scenarios are limited to simpler terrains.
To address these challenges, Professor Jianyu Chen’s research team proposed Denoising World Model Learning (DWL) based on their previously developed Humanoid-Gym platform (https://arxiv.org/abs/2404.05695). DWL enhances humanoid robots' ability to navigate diverse and complex terrains. This technique has been validated on two different sizes of humanoid robots from Robot Era: XBot-S (1.2m) and XBot-L (1.65m). DWL is the first algorithm to enable humanoid robot to master various real-world challenging terrains with end-to-end RL and zero-shot sim-to-real transfer, using a single neural network.
Figure1: Jianyu Chen's research team proposed a method demonstrated in real-world experiments.
As shown in Figure 1, the humanoid robots demonstrate stable walking on various terrains, including snowy slopes, stairs, and irregular surfaces, while withstanding significant external disturbances. The same neural network policy is used across all scenarios, showcasing its robustness and generalization capability. DWL's success stems from its innovative representation learning framework, which effectively denoises factors that exacerbate the sim-to-real gap. Additionally, the research team introduced an active 2-DoF ankle control mechanism (closed kinematic chain ankle mechanism), as shown in Figure 2, significantly enhancing the robot's robustness.
Figure 2: The humanoid robots used in this work are the XBot-S and XBot-L from Robot Era.
“In future work, we will incorporate additional sensor information to enable more efficient navigation in challenging terrains while maintaining robustness, further enhancing the DWL framework,” Co-first author Yen-Jen and Xinyang said.
The corresponding author of this work is Assistant Professor Jianyu Chen. The co-first authors include Xinyang Gu (engineer at Robot Era), Yen-Jen Wang (third-year master's student at IIIS, Tsinghua), Xiang Zhu (third-year master's student at IIIS, Tsinghua), and Chengming Shi (first-year PhD student at IIIS, Tsinghua). Other co-authors include Yanjiang Guo (second-year PhD student at IIIS, Tsinghua) and Yichen Liu (second-year PhD student at IIIS, Tsinghua).
Introduction of the Research Team
Jianyu Chen is an assistant professor of the Institute for Interdisciplinary Information Sciences at Tsinghua University and the founder of the embodied AI company, Robot Era. He obtained his bachelor's degree from Tsinghua University and his doctor's degree from the University of California, Berkeley, under the supervision of Professor Masayoshi Tomizuka.
Jianyu Chen currently leads the Intelligent Systems and Robotics Laboratory (ISR Lab), which focuses on artificial intelligence and robotics, aiming to build general-purpose robotic systems with high performance and high intelligence. His research areas include reinforcement learning, robotics, control, and foundation models. He has published more than 50 papers in flagship international conferences and journals in the fields of robotics and artificial intelligence, and some of the papers have been selected as best paper finalists for L4DC 2022, IEEE IV 2021 and IFAC MECC 2021. He was selected into the Forbes 30 under 30 Asia list in 2021.
Editor: Yueliang Jiang
Reviewer: Xiamin Lv
