Thanks for such amazing illustration for Diffusion. One question is about the equation in slice @ 13:16, how to get t-2 and t-3? x_t=sqrt(a_t)*x_t-1+sqrt(1-a_t)*e x_t-1=sqrt(a_t-1)*x_t-2+sqrt(1-a_t-1)*e x_t=sqrt(a_t)*[sqrt(a_t-1)*x_t-2+sqrt(1-a_t-1)*e]+sqrt(1-a_t)*e=sqrt(a_t*a_t-1)*x_t-2+[sqrt(a_t-a_t*a_t-1)+sqrt(1-a_t)]*e The rightmost term doesn't equal or close to sqrt(1-a_t*a_t-1)*e Dis I misunderstand something? Thanks again. @Outlier

Very Bad Video

Thanks for the explanation, it's awesome! But I have a question. In cross attention, if we set the text as V, the final attention matrix could be viewed as a weighted sum of each word in V itself (the "weighted" part comes from the Q, K similarity). If I understand correctly, the final attention matrix should contain the values in the text domain, why can we multiply by a W_out projection and get the result in the image domain (add it to the original image)? Will it make more sense if we set the text condition as Q, and the image as K, V?

U really liked that you showed the derivation in an understandable way

Amazing explanation thanks a lot! Minor improvement suggestion: add a pop-protection to your microphone :)

I think it is not exactly true statement that N(mu, sigma) = mu + sigma*N(0, 1). Just try that transformation, mu plays a role of translation in the value axis. However, what is correct, that if you sample from the left side, it acts the same as if you sample from the right side. I am pointing this out because I got stuck with that for a while. But I still also might got it completely wrong.

Love this - what a fantastic achievement!

Brilliant work!

11:57

This is a really great video, thanks for your big effort explaining!

Wonderful explanation! Not sure if this is in the original papers, but I find it very odd that there is no nonlinear function after V and before W_out. It seems like a waste to me since Attention@V is itself a linear function, so w_out wont necessarily change content of the data beyond what Attention@V already would have done through training.

The strange patten in the reconstructed image and the generated image is likely to be caused by the perceptual loss, I have no idea why but the disappears when I take the perceptual loss away.

Thank you so much . But please i prefer that you go to the maths and operations more detailly being training of Würstchen 🎉🎉 thank you

really nice video, helps me understand a lot❗

Great video!! At 13:34, does multiplying with a_0 correct?

best explanation

This is an amazing video 🔥

12:12 does anyone understand here why when \beta is linear, \sqrt{1-\beta} is linear as well?

the statement at 15:33 isnt right ... is it ? cause i have a counter f(x) = x^2 , g(x) = -x^2 here f(x) >= g(x) but thier derivatives are negatives of each other. Please help i dont really understand the concept of ELBO

At inference. Input of State A( VQGAN decoder) is discrete latents. Continuous latents needs to be quantize to discrete latents( discrete latents is also choosen from codebook, by which vector in Continuous latents nearest to vector in codebook). But Output of State B is Continuous latents. And Output of State B is directly for Input of State A..... if it right ? How State A( VQGAN decoder) handle Continuous latents . I check VQGAN paper and this Wurchen paper. That is not clear. Please help me that. Thank you

So, apparently, it sounds like Wurchen is exactly at Stage C. am i right?

Hey great video. Can you tell me why random sampling of codebook vectors doesn't generate a meaningful images. In Vae we random sample from std gaussian, why the same doesn't work for vq auto encoders.

How did you manage to get 128 A100 for 3 weeks? I think the cost is about 100k USD for one run. Assuming you did multiple iterations, the overall cost can be easily 200k for this project.

Just completed writing a 24 paged handwritten note based on this video and a few other sources

Amazing explanation and thank you for taking the time to properly visualize everything

Sorry if I am misunderstanding, but at 19:10, shouldn't the code be: "uncond_predicted_noise = model(x, t, None)" instead of "uncond_predicted_noise = model(x, labels, None)" Also, according to the CFG paper's formula, shouldn't the next line be: "predicted_noise = torch.lerp(predicted_noise, uncond_predicted_noise, -cfg_scale)" under the definition of lerp? One last question: have you tried using L1Loss instead of MSELoss? On my implementation, L1 Loss performs much better (although my implementation is different than yours). I know the ELBO term expands to essentially an MSE term wrt predicted noise, so I am confused as to why L1 Loss performs better for my model. Thank you for your time.

Hey @Outlier , on 12:44 looks like you mistakenly use "a" instead of "alpha" notation symbol inside 'product notation (Pi Notation)'. Since you mentioned the example multiplication below using "alpha notation". e.g. t = 8 "alpha_8" = "alpha_1" x "alpha_2" x "alpha_3" x "alpha_4" x "alpha_5" x "alpha_6" x "alpha_7" x "alpha_8" ======================================================= Is it intentional, though ? Please let me know. Thanks

This is dumb, but can anyone explain why when beta increases linearly, the square root of 1 - beta decreases linearly, at 12:13? Shouldn't it have some curve to it, given the square root?

@22:32 ur missing a bar over the alpha on the bottom line. Plz correct me if im wrong

Hi! Can you please explain why the output is getting two stitched images?

How did the KL divergence expand to log(q/p)? You yourself mentioned it to be the integral of q*log(q/p) for D_kl(q||p) Perhaps too much of a simplification.

I really love to understand this Diffusion thing, but I literally says "WTF" ...

what a wonderful and thoughtful way to deliver the whole langscape of the diffusion model! Nice video! 👍

Just out of curiosity... Did they derive everything and then get amazing results or the other way around. I am pretty sure someone got amazing results with this and then thought how should I put it in a paper without being rejected 😂😂😂

As a PhD student who also struggles with notations, THANK YOU!!

@outliier in the 12:49 wondering how in the second line beta in beta*I got under-root ? please explain the assumption taken here...

` x = torch.randn((n, 3, self.img_size, self.img_size)).to(self.device) predicted_noise = model(x, t) ` in the deffusion class why you create an noise and pass that noise into the model to predict noise ... please explain

Great content

Nice !

Thank you so much!!! I was finally able to understand Attention with your video!!!

Incredibile explanation. My Bayesian analysis course would be so much better if it is explained this way. *Little suggestion: use \left( blabla ight) for brackets such that it will fit vertically.

Thank you so much for this

why you use self-attention in the unit architecture instead of other attention unet? how it works here

Thanks for the crystal clear explanation!

Thanks, the video was really helpful, it gave me such a great time in understanding diffusion models, kudos and keep on making such quality content!

This is really great. make it so easy to grasp such a complex stuff

So the process of adding noise and removing it happens in a loop

I have two feature x--> text feature , y--> image_feature and rag --> rag_feature (This is Extra feature) I want to apply the cross _attention between (rag and (x/y)) How should I apply

This was the best ML paper review I have ever seen. You stopped making videos but I would really love to see you go through more of this for more research in the field man! Hatsoff to you.

19:57 how can it be KL divergence? there has to be a term outside of log for it to be KL divergence?? can you explain this?