Learning Robot Control: From RL to Differential Simulation - (PhD Defense of Yunlong Song)

This thesis focuses on Learning Robot Control by integrating deep reinforcement learning (RL) and model-based control methods. It aims to develop advanced control methods that bridge the gap between data-driven learning and model-based control. The proposed methods enhance robot agility and robustness in real-world applications.
Key contributions are:
- Show that RL outperforms Optimal Control in autonomous racing because it directly optimizes a non-differentiable task-level objective.
- Propose a policy-search-for-model-predictive-control (MPC) framework, combining RL's ability to optimize high-level task objectives with MPC's precise actuation and constraint handling.
- Introduce a differentiable simulation framework to leverage robot dynamics for more stable and - efficient policy training.
- Develop a high-performance drone racing system outperforming optimal control methods and professional pilots.
- Develop Flightmare, a flexible modular quadrotor simulator for reinforcement learning and vision-based flight.
OUTLINE:
00:00 - Introduction
02:37 - Robot Control: An Optimal Control Perspective
03:14 - Robot Control: A Reinforcement Learning Perspective
05:06 - Project 1: Autonomous Drone Racing: Optimal Control vs. Reinforcement Learning
12:05 - Project 2: Flying Through Dynamic Gates: Reinforcement Learning for Optimal Control
16:04 - Project 3: Quadrupedal Locomotion: Differentiable Simulation
20:18 - Conclusions
23:05 - One More Thing

[@zebinhuang6352]

Congratulations

[@xiaotiandai]

Good job

[@leihe7188]

Congratulations! Very nice work!

[@kehanlong648]

Amazing work!

[@eeshiba8505]

Nice work! Congrats!

[@user-kt9kz5pe7w]

amazing work!

[@yumaoliu5319]

Congratulation! Dr. Song.

[@ashfaquekhan7282]

amazing work Dr. Song

[@ChanJoon]

Congratulation! Very impressive! I also want to be a researcher like you!

[@user-hw1ge9df9k]

Very impressive and excellent work! Congratulation! Dr. Song. May I ask you a simple question? How do you train the RL NN so that the drone know the order of gate to pass? 1st gate, 2nd gate, .. , how the drone trained to know the sequence of gates to pass? Is there any mark on the gate corner so that the sequence can be visually recognized?

[@QiWei_cs]

Congrats! BTW, can you share your PhD thesis?

[@xiaodaochen]

Congratulations