Learning to Adapt for Building Generalist Robots

演讲人: Mengdi Xu Carnegie Mellon University
时间: 2023-12-28 14:00-2023-12-28 15:00
地点:FIT 1-222

Talk Abstract: Deep robot learning unlocks exciting robot capabilities in the recent decade, but still struggles to generalize to unseen tasks despite large-scale pre-taining. This limitation arises from the highly unstructured real world, which encompasses an endless array of possible tasks, some extending beyond the robot training set. In this talk, I will discuss how to build generalist robots with strong data efficiency, parameter efficiency, and robustness. I will highlight my research on improving robot generalization through learning to adapt with different abstractions, including in-context robot learning conditioning on a few demonstrations, unsupervised continual learning discovering robot task structures, and embodied agents leveraging large foundation models. These approaches have shown significant promise in acquiring new motor skills in various applications and even solving long-horizon physical puzzles with creative robot tool use.


Mengdi Xu is a fifth-year Ph.D. student at Carnegie Mellon University, supervised by Prof. Ding Zhao. Her research focuses on building learning-based generalist robots that can interact with the world like humans. Mengdi has spent wonderful summers interning at Google DeepMind, MIT-IBM Watson AI Lab, and Toyota Research Institute. She holds a Master's degree in Machine Learning from Carnegie Mellon University, a Master's in Robotics from Johns Hopkins University, and dual Bachelor's degrees in Vehicle Engineering and Management from Tsinghua University. Mengdi was selected as one of EECS Rising Stars 2023, RSS Pioneers 2023, and Computational and Data Science Rising Stars 2023.