CS885 Module 1: Trust region & proximal policy optimization

[@geeklonewu7812]

Thanks! Best-ever lecture on TRPO and PPO!

[@yuniorzhang5643]

Excellent explaination, smooth and clear!

[@robosergTV]

thanks! Will there be more lectures on RL ?

[@Sebastian-fc5dc]

Very nice to follow, thanks

[@НиколайНовичков-е1э]

Thank you!

[@outtaspacetime]

good job, thank your sir!

[@marcotroster8247]

You missed the essence of PPO entirely. It's done because simulators are slow. And to make training feasible you wanna re-use the training data sampled with the old policy for multiple updates. But the ratio would just go crazy and cause unstable training. So a ratio bound gets introduced. Compute the policy gradients and have a look at how the clipping and min work. It's so genius that the gradients related to "bad" training examples become zero due to the constant rule.

[@yunpenghuang2246]

thanks a lot

[@ashishj2358]

Nice!