Why no Dreamer series?

🤩

The best!

👍👍👍👍👍👍

Thanks for the help!

tysm bro this helps me somuch u a real g ❤

like like like like like something something like like like * vocal fry * like like like like * vocal fry * Thanks for sharing btw. Excellent material.

Hi, Can some one confirm whether my understanding is correct or not? In Q-Learning, the agent takes an action in state 𝑠 and transitions to 𝑠′. The action it actually takes in 𝑠′ (𝑎′) might not be the optimal action 𝑎max (the one that maximizes 𝑄(𝑠′,𝑎′)). However, the Q-value for 𝑠,𝑎 is updated assuming the agent takes the optimal action in 𝑠′ , i.e., max 𝑎′ 𝑄(𝑠,𝑎′). Similarly, in 𝑠′ , the agent might take a suboptimal action 𝑎′ , transitioning to 𝑠′′. When updating 𝑄(𝑠′,𝑎′), Q-Learning still uses the action 𝑎max′′ that maximizes 𝑄(𝑠′′,𝑎′′), assuming optimal behavior in 𝑠′′ . But the actual action 𝑎′′ could still be based on the exploration policy. This shows how Q-Learning is off-policy, as it updates values assuming optimal actions, even though the agent might not follow the optimal policy during exploration.

42:28 that student almost stumped Pieter with that question. That was a nice question.

I don't understand 32:15. If the dynamics are stochastic, then the term P("firepit" | "left-of-firepit" , a) would be non-zero and I would expect V_2("left-of-firepit") would be negative. What am I missing here?

only guy on the internet who explained about bayesian sampling properly!!

I’ve realised that everything I want to achieve starts from my mind

The explanations on update policy are a bit quick. This video gives more insights : kzbin.info/www/bejne/ommao5qCnJ5jfqs Refers to the comment for more explanation, especially for n=3 and why V(3,3) = 0.78 "You need to consider the noise. 80% of the time, agent goes in the desired direction. But, 10% of the time it goest left relative to desired direction and 10% of the time it goes right. If agent hits a wall, it stays in place. So, at iteration 3, 10% of the time the agent goes left and hits a wall so ends up staying at (3,3). 10% of the time it goes right and ends up in (2,3). 80% of the time it goes where it wants and ends up in (3,4). Value at iteration 2 of (3,3) = .72, (2,3) = 0, and (3,4)=1. So, using the bellman equation, you get 0.8[0+(0.9)(1)]+0.1[0+(0.9)(0.72)]+0.1[0+(0.9)(0)] = 0.78"

may i suggest a basic course should at least introduce all terminology , it would be good to specify what is pi upfront

thanks Pieter! Amazing explanation. from Matteo and Lilli

I would say this series is a great shortcut for beginners to enter the realm of RL from scratch. Even the most minor details and associated definitions are covered with plain words for breakdown explanation. Notes are clear to understand and illustrations of computing are beautiful and intuitive. Thank you so much prof.

Thank you! I couldn't distinguish between A* using graph search and A* using tree search. And I get it now

I love youtube for this actually.

How is. a GAN an unsupervised model? it do get supervisory signal from the discriminator . Or Karpathys char RNN, next character ground truth is used during training. Is not it better to call them self-supervised or a case where labels can be found from the data itself

Is it just me being not a extremely professional ML practitioner? I found this lecture really hard to follow. Lots of points not being explained clearly and glossed over. I've watched some other advanced level DL courses on youtube and those are wayyyyy better in explaining the concept. I had high hope and felt really disappointed.

Amazing man!! The best video to really understand d-seperataion!! and only in 20 minutes!! Amazing

Garcia Daniel Anderson Elizabeth Clark Thomas

Thanks you for sharing. This course is what I am finding. Can't believe I can access the top quality course from my bed rooms.

The best lecture ever in this topic.

Hi, it seems the video failed for slides 14-17. Any chance there is any content to help with that part?

THANK YOU SO SO MACH

at 1:24:14 they mention that they have to double check the loss; could you tell us what they found?

Could you elaborate 8:54? is it really active?

What kind of book to read, please refer the book from start. give document for it.

What kind of book to read, please refer the book from start. give document for it.

What kind of book to read, please refer the book from start. give document for it.

What kind of book to read, please refer the book from start. give document for it.

What kind of book to read, please refer the book from start. give document for it.

What kind of book to read, please refer the book from start. give document for it.

Thanks for that! It's so much better than any RL intros I've watched on a graduate level.

Good explanation. Very clear. I did not understand the entropy part , could someone suggest a good resource to understand that part?

Can’t thank you enough. This is the only video people need to learn about d-separation. Thanks again for your time and effort!

I'm struggling to get a good intuition of what a NN trained for this task is learning to do exactly. This is my thought process: You have some process of going from x0 to xt, by sampling epsilon ~ N(0,I) and basically adding it to x0, with a couple of constants. Then you have a NN that given xt, it approximates the actual sample from N(0,I) that was used (epsilon). And you optimize this NN on this task. In the end, you can just give it pure noise, it will give you some other approximation of a sample from N(0,I), and you can substract it (with some constants) and get a novel image. If this is essentially true, I struggle to get an intuition of how by training a NN to do a great job at this, you get actually great-looking novel images. I can see how by training it to go from x_t+1 to x_t, the NN will be a denoising function. One can imagine the weights of this NN to encapsulate different patterns in the pixels of the NN to identify regions of pure noise vs regions of the original image. It makes sense that it would be able to do a good job when there are some patterns to detect. But how about when it sees a purely noisy image? It would seem to me that whatever patterns it's trying to match, that none of them would be there since it would all be noise. And you would get crap, basically. I hope I explained myself, any help is appreciated! Thanks!

very well explained

Which probability course to revise to understadn qickli all this? sampling ...(from youtube) or other

Garcia Mary Davis Deborah Taylor Karen

What if the nodes are adjacent?

10:00 that flexing :"D

Is the code for the examples available or is it possible to get?

At time 1:15:22 (slide 55) for likelihood ratio gradient on VAE, I did my calculation. And for the first term (the one that does not become 0 eventually), I have the term sum over all z, we start with the term 1/q * \triangledown_\phi (q * [ .... ] ) = 1/q * \triangledown_\phi (q * [.... ]) + 1/q * q * \triangledown_\phi [...], I see the expected value of the second term is 0. For the first term, when i take expectation, it becomes \sum_z q * 1/q \triangledown_\phi (q* [....]) = \sum_z \triangledown_\phi (q* [...]) not sure how to proceed... and it's not the same case as slide 56 because we don't have something of q_phi(z|x) f(z), in this case we have q_phi(z|x) [ logp_z(z) + log p_\theta(z|x) - log q_phi(z|x) ]

Thank you for the lecture!

Excellent explanation. I appreciate it so much. Now i finally understand this after so much struggling looking at some other resources. Thank you!

wow so many examples thanks!

Thank you very much. Youre really helping me learning for my exam thursday :)

The video is really interesting, thanks for uploading it. I hope audio was better, some times I couldn't understand what was said