L5 DDPG and SAC (Foundations of Deep RL Series)

[@flopyarcade4407]

For those unaware: Peter Abbeel is a legend. Consider gratitude. ✌️

[@yihongliu7326]

Thank you, Pieter. More than 1 year back when I first learned RL I was trying to search for something like this online but couldn't. This is very helpful and amazing :)

[@hassaannaeem4374]

Awesome series. Thanks Pieter

[@dermitdembrot3091]

How widely is this "backprop through Q" used in continuous control? Does e.g. SAC use it?

[@sourjyasarkar]

In the learning step of Soft-Actor Critic Algorithm, shouldn't it be Q_{phi}(st,at) and pi_{\theta}(at | st) presuming the parameters for the Q value network in \phi and that of the policy network is \theta ?

[@dermitdembrot3091]

Yes it should, to keep the slides consistent, but no big issue. Maybe the papers use greek letters differently.

[@bashkirovsergey]

Hello! At 3:38 I cannot understand the Q(t) = r(t) + gamma * Q(t+1). Shouldn't it be Q(t) = r(t) + gamma * V(t+1)? I'd appreciate any hint! Thank you!

[@ioannis.g.tzolas]

My understanding is that the action value is equal to the state value since we are having a deterministic policy.

[@rezamaz8975]

In general, value function give us how much the state is good and it seems to me it's equal Q function

[@ky8920]

i think it uses a single sample to estimate weighted sum over q_t+1

[@aakashrana3036]

I think that the action is taken from the policy, have a look at the ddpg algorithm pseudocode to have more clarity.

[@arizmohammadi5354]

I appreciate your efforts

[@nguyenmanh466]

10:00 that flexing :"D

[@levizhou6726]

Thank you sir.

[@arisioz]

In comparison to the other banger lectures this one was very briefly analyzed :(

[@hadsaadat8283]

the voice quality is aweful