Variational Autoencoder - Model, ELBO, loss function and maths explained easily!

[@alisalimi5300]

After following your content for a while, I come to believe that if there is a concept, you will explain it better than anybody else here.

[@Koi-vv8cy]

It's the clearest explanation about VAE that I have ever seen.

[@umarjamilai]

If you're up to the challenge, watch my other video on how to code Stable Diffusion from scratch, which also uses the VAE

[@lucdemartre4738]

I would pay so much to have you as my teacher, that's not only the best video i've ever seen on deep leanring, but probably the most appealing way anyone ever taught me CS !

In ML topics most of the videos go to the beef of the math or the code implementation and do not care about a whole picture understanding. Your video fills that gap so nicely! Watching this video is a strong first step to understanding the topic deeply! very well done!

[@chenqu773]

The peps starting from 06:40 are the gem. Totally agree.

[@egoworks5611]

fing masterpiece thank you man, you outdid stanford and other papers with this explanation.

[@OlenaHlushchenko]

I love this so much, this channel lands in my top 3 ML channels ever

[@kranthikumar9998]

@Umar Jamil, you have explained VAE so much better compared to others. You are one of my favorite KZbin teachers.

[@lucdemartre4738]

PLATO MENTIONED PLATO MENTIONED I LOVE YOU THAT'S THE BEST VIDEO I'VE EVER SEEN !!!

[@vipulsangode8612]

This is the best explanation on the internet!

[@miladheydari7916]

this is the best video on the Internet

[@JohnSmith-he5xg]

Getting philosophical w/ the Cave Allegory. I love it. Great stuff.

[@isaz2425]

Thanks, this video have many explanations that are missing from other tutorials on VAE. Like the part from 22:45 onwards. I saw a lot of other videos that didn't explain how the p and q functions were related to the encoder and decoder. (every other tutorial felt like they started talking about VAE, and then suddenly changed subject to talk about some distribution functions for no obvious reason).

[@umarjamilai]

Glad you liked it!

[@shuoliu3546]

You solved my confusion since long! Thank you !

[@xm9086]

You are a great teacher.

[@sandahk7173]

It was the best video I found to explain VAE. Thank you so much!

[@greyxray]

so clear! so on point! love the way you teach!

[@anantmohan3158]

Thank you for creating such a nice explanation of VAE. When are you bringing the practical implementation video of VAE? please make a video for this as well.

[@charlieawesome5986]

Thank you so much for this bro

[@ricardogomes9528]

Great video on VAE Umar, but I keep with a question: At 11:49, the KL divergence is written as "D_{KL}({numerator}||{denominator}". While in 11:34 you wrote KL divergence as "D_{KL}{denominator}||{numerator}". Why is that? I guess it is not because of the "\minus" sign

[@nassimlamrini-b5i]

🎯 Key points for quick navigation: 00:13 *understand fundamental ideas* 00:41 *explain autoencoder concept* 02:36 *issues with traditional autoencoders* 03:02 *introduce variational autoencoder* 04:13 *sampling from latent space* 25:02 *Model and prior choice.* 25:17 *Element-wise noise multiplication.* 25:32 *Learning log variance.* 26:00 *Maximizing ELBO for space learning.* 26:15 *Derivation of loss function.* Made with HARPA AI

[@dannyjiujitsu]

Phenomenal teaching.

[@pauledam2174]

great video ( am about half way through). I think at minute 13 there is a misstatement (I think). in general if g(x) is greater than h(x), if we find the max of h it doesn't mean we have located the max for g

[@desmondteo855]

Incredible explanation. Thanks for making this video. It's extremely helpful!

[@TheBlackNight971]

Both VAE and AE map the input over a latent space, the difference lies on the **structure** of this latent space. The AE latent space is not "well-organized" as well as the VAE's latent space.

[@shajidmughal3386]

Great explanation. Clean!!! Reminds me of school where our physics teacher taught everything practical and it felt so simple. subs+1👍

[@oiooio7879]

Wow thank you very informative

[@vikramsandu6054]

Simply amazing. Thank you so much for explaining so beautifully. :)

[@morgancredib-ai2501]

A normalizing flow video would complement this nicely

[@nadajonidi9691]

Would you please give the url for normalizing flows

[@darkswordsmith]

Variational methods is a class of methods ive always struggled to understand. I think your explanation addresses some key questions but I thunk you jumped into the math too quickly. Questions im still wondering about: - what are the "variational" parameters? They are represented by phi, but what do they do/mean? - why is having the noise source necessary? What happens if i set epsilon to be 0 all the time?

[@umarjamilai]

Link to the slides: github.com/hkproj/vae-from-scratch-notes

[@RomanLi-y9c]

thx for the video, this is awesome!

[@samcal740]

This actually very good

[@awsom]

Great Explanation!!

[@mirkovukovic6517]

At the very end of the video Umar mentions that in "the next video" he will explain the coding of the VAE along with some examples. I could not find that video. Was it made? What is its title? Thanks 🙂

[@umarjamilai]

There's a video on coding stable diffusion, which includes the VAE

[@mahsakhalili5042]

Appreciate it! it helped me a lot

[@吴腾-q7j]

Thanks for sharing . In the chicken and egg example, will p(x, z) be trackable? if x, z is unrelated, and z is a prior distribution, so p(x, z) can be writen in a formalized way?

[@waynewang2071]

Hey, thank you for the great video. Curious if there is any plan to have a session for code for VAE? Many thanks!

[@rishidixit7939]

What is the meaning of Latent Space? How is it different from the code that AutoEncoder have ?

[@hainguyenthien4225]

thank you very much

[@danialramezani-c5k]

You are awesome; cheers

[@GrifinsBrother]

Sad that you have not released video "Hot to code the VAE"(

[@shashankgsharma0901]

thanks UMAR!

[@oiooio7879]

Can you do more explanations with coding walk through that video you did on transformer with the coding helped me understand it a lot

[@umarjamilai]

Hi Oio! I am working on a full coding tutorial to make your own Stable Diffusion from scratch. Stay tuned!

[@huuhuynguyen3025]

@@umarjamilai i hope to see it soon, sir

[@DikkeHamster]

Don't AEs also learn a latent space representation? I'm not sure this is the largest difference with VAEs. The sampling is, however.

[@umarjamilai]

They do, but nobody forces the latent space of AEs to be a gaussian or any other known distribution.

[@Kiara-h2o]

Thanks a lot

[@martinschulze5399]

14:41 you dont maximiye log p(x), that is a fixed quantity.

[@sohammitra8657]

Hey can you do a video on SWin transformer next??

[@gabrielpetersson3416]

Banger again

[@prateekpatel6082]

why does learning distribution via a latent variable capture semantic meaning. ? can you please elaborate a bit on that

[@quonxinquonyi8570]

Latent variable is of low dimension compare to input which is of high dimension…so this low dimension latent variable contains features which are robust, meaning these robust features survive the encoding process coz encoding process removes redundant features….imagine a collection had images of cat and a bird image distribution, what an encoder can do in such a process is to outline a bird or cat by its outline without going into details of colours and texture….these outlines is more than enough to distinguish a bird from a cat without going into high dimensions of texture and colors

[@prateekpatel6082]

@@quonxinquonyi8570 that doesnt answer the question. Latent space in autoencoders dont capture semantic meaning , but when we enforce regularization on latent space and learn a distribution thats when it learns some manifold

[@quonxinquonyi8570]

@@prateekpatel6082 learning distribution means that you could generate from that distribution or in other words sample from such distribution…but since the “ sample generating distribution “ can be too hard to learn, so we go for reparametrization technique to learn the a standard normal distribution so that we can optimize

[@quonxinquonyi8570]

I wasn’t talking about auto encoder,I was talking about variational auto encoder…

[@quonxinquonyi8570]

“ learning the manifold “ doesn’t make sense in the context of variational auto encoder….coz to learn the manifold, we try to approach the “score function” ….score function means the original input distribution….there we have to noised and denoised in order to get some sense of generating distribution….but the problem still holds in form of denominator of density of density function….so we use log of derivative of distribution to cancel out that constant denominator….then use the high school level first order derivative method to learn the noise by using the perturbed density function….

[@frommarkham424]

VAEs are awesomeeeeeee they make ai art generators work

[@OlenaHlushchenko]

Thanks!

[@MooseOnEarth]

1:21 - The file name of the original file zebra dot jpg should be replaced by zebra dot ppm (a truely uncompressed format). Then the zebra dot zip in the latent space be replaced by zebra dot jpg. And the result by zebra.ppm. Then the analogy would be way better to cover the lossy nature of the encoder but an enormous compression potential. It would be a comparison to the wavelet compression performed by JPEG, which is never lossless essentially.

[@lifeisbeautifu1]

You rock!

[@na_haynes]

The Cave Allegory was overkill lol

[@umarjamilai]

I'm more of a philosopher than an engineer 🧘🏽

[@shashankgsharma0901]

I lost you at 16:00

[@anjalikatageri1550]

fr me too

[@zlfu3020]

Missing a lot of details and whys.