Variational Autoencoders | Generative AI Animated

[@essentiallifter]

this account is seriously underrated, will definitely blow up soon

[@melanynadine972]

Agree, I just subscribed ❤

[@joshuat6124]

Agreed 💯

[@authenticallysuperficial9874]

💯

[@Higgsinophysics]

I can't believe how calmly and clear you explain difficult topics!

[@Deepia-ls2fo]

Well that's the magic of text to speech 😁

[@viewer8221]

haha I knew it was AI voice

[@christaylor-gz6mi]

Love this. You’re going to blow up! Content severely under the radar at the moment

[@AtiqurRahman-gj6mg]

Finally, the concept of VAE is clear. Thanks a ton.

[@Deepia-ls2fo]

You're welcome, thanks for the comment !

[@xichengwang1319]

I first saw this in Chinese media with Chinese subtitles, then just came back to subscribe to the original author. The most clear introduction ever seen with such nice proper animation. It will blow up for sure.

[@Deepia-ls2fo]

Well thank you ! Can you send me more info about that through my email ? [e-mail]

[@anthonyortega358]

This channel is amazing, you should be very proud of what you have produced Thibaut!!

[@Deepia-ls2fo]

Thanks !

[@valentinfontanger4962]

The animation regarding the Bayesian stats, I almost cried. It was beautiful.

[@Pokojsamobojcow]

Wow it must be a lot of work to obtain such amazing animations! The video is really dynamic and easy to follow, congratulations

[@julius4858]

The program is called manim, it’s from 3blue1brown

[@yibozhong9557]

This video is so well explained for a Gen AI beginner! Love it!

[@ashutoshpadhi2782]

The amount of effort you put into these works is really commendable. You are a blessing to humanity.

[@yusufakmalov-l5n]

Such a great video! I’m always looking forward to your uploads.

[@Tunadorable]

3blue1brown specifically for AI models??? sign me up!!! I'll fs be linking to you in my own vids whenever relevant, this was great

[@BenjaminEvans316]

Great video. Looking forward to your video on contrastive learning, it is my favourite subject in deep learning. Your videos combine great production skills (animations, colour selection, movement between frames) with in-depth understanding of complex concepts.

[@Deepia-ls2fo]

Thanks for the kind words !

[@Koosx2BestofYoutubeCoc]

Litteraly one of the best channels to learn deep learning

[@diabl2master]

Brilliant video!! Can't wait for the videos on extensions of VAE!

[@babatundeonabajo]

This is a really good video, and the animations are top-notch. I feel this video is good not just for those learning about AI but also those learning Statistics.

[@AICoffeeBreak]

Love the clarification at 00:13, because I've also felt that the misconception is wide spread. I've heard people say: I am not using GANs anymore, I am using Generative AI. The word "Generative" is literally what G in GAN stands for. 😂😂

[@Deepia-ls2fo]

I made this intro because I couldn't stand the number of "GenAI experts" on my LinkedIn feed :(

[@muhammed2174]

You're a master at your craft, it is a testament to your studies!

[@Bwaaz]

Amazing quality, hope the channel takes off ! Great use of manim

[@Deepia-ls2fo]

Thanks !

[@BeeepBo0op]

Thank you for finally making me understand the reparametrization trick!! It was thrown at me several times during a DRL class I took last year and I never really understood what we did. This made it much much clearer, thank you! Also: great video overall!

[@Deepia-ls2fo]

Glad it helped !

[@MutigerBriefkasten]

Thanky you again for the great content and the amazing animations 🎉💪👍 keep going, hopefully your Channel explode with more subscribers .. i will recommend it for sure to other people

[@Deepia-ls2fo]

Thank you :)

[@jiechenhuang9019]

This video is pure gold. You introduced VAE in such a clear and intuitive way, especially by contrasting the latent space between VAE and normal autoencoders. A question: why does VAE’s latent space have those features like smooth interpolation, which other autoencoders fail to learn? Thank you😊

[@Deepia-ls2fo]

Thanks, the sort of "continuous" latent space is due to the regularization we impose during training. The latent space is supposed to fit a Gaussian with 0 mean and variance 1, while in regular autoencoders no structure is imposed on the latent space except for its dimension

[@jiechenhuang9019]

@@Deepia-ls2fo Thanks! I'm also a PhD student in CS. I'll definitely recommend this channel to my friends. BTW, is your next videos about diffusion models? Can't wait to see!

[@Deepia-ls2fo]

@@jiechenhuang9019 Yes the next video will be on diffusion models indeed

[@kacperzaleski8125]

cant wait for the next video! this was great!!!!!

[@Deepia-ls2fo]

Thank you !

[@vasoyarutvik2897]

video quality is very hight, very informative video. please upload more videos related to machine learning, deep learning and generative AI Keep it up love from india

[@ahmedoumar3741]

Your videos and explanations are really excellent, please keep doing this.

[@杨卢老]

非常感谢着这个视频，解决了困扰我很久的问题！！！ 太牛逼了！😍

[@Deepia-ls2fo]

You're welcome !

[@StratosFair]

Masterpiece of a video

[@TimMattison]

I'm still a bit too much of a n00b to get all of the math involved but the videos you're creating are great, please keep at it!

[@juantarazona1533]

Absolute godsend of a video

[@drannoc9812]

Amazing content :D I hope you'll do your next videos on VQ VAE and VQ VAE 2, I enjoyed so much reading those papers !

[@Deepia-ls2fo]

Thanks, I really gotta take another look to the paper

[@mostafasayahkarajy508]

Thank you very much for providing and sharing the lecture. Excelent explanation and so a high-quality video!

[@Deepia-ls2fo]

Thank you !

[@azharafridi9619]

really helpfull thank you so much.. please if possible make complete playlist like this... it would be great work for the society. once again thank you so much

[@shivakumarkannan9526]

Excellent video. Colored diagrams help a lot.

[@shangliu6285]

perfect video, its easy to understand the VAE, subscribed!

[@Deepia-ls2fo]

Thanks !

[@abelhutten4532]

Great visualizations, and good explanation! Congratulations and thanks for the nice video :)

[@Deepia-ls2fo]

Thank you !

[@essentiallifter]

just a tip can you make it super clear that the reason we sample in the middle is to produce a nice continuous latent space where the different dimensions encode different meaning

[@diabl2master]

10:10 Excellent visualiation. I really understood it here.

[@flueepwrien6587]

Finally I understand this concept.

[@abdelmananabdelrahman4099]

Great video 🎉. I've never had such a great explanation of VAE. Waiting for VQVAE.....

[@Deepia-ls2fo]

Thank you !

[@Dylan-du1uu]

Such a great video that helps me a lot. Thanks!

[@carlaconti1075]

Thanks to you everything is clear now, thank you Deepia

[@Deepia-ls2fo]

thanks miss conti

[@vitorfranca80]

Incredible explanation!! Thank you for sharing your knowledge! 😁😁

[@Deepia-ls2fo]

Thanks !

[@deepniba]

Thank you , still looking for VAE variants videos

[@authenticallysuperficial9874]

Audio comes out from under water at 2:09 btw

[@Deepia-ls2fo]

Thank you, I had some issues with copyrighted music which led to KZbin removing it but also degrading the audio...

[@hnull99]

I thought my headset is bugging xD

[@EkalabyaGhosh-g3i]

This channel needs more subscribers

[@Deepia-ls2fo]

Thank you !

[@lucianovidal8721]

Great content! It was a really good explanation

[@HenrikVendelbo]

Thanks

[@Deepia-ls2fo]

Ho my, thank you so much !

[@HenrikVendelbo]

I find math speak very hard to grok. I was always good at math, but always got turned off by the navel gazing and geekery. You do a great job keeping it engaging without assuming that I am a math geek

[@Deepia-ls2fo]

@@HenrikVendelbo Yeah sometimes it do be like that in math classes. I think it's important to look at equations when they tell us something about the models, but computational tricks or complex equations are not that interesting.

[@cs-cs4mj]

hey, so well explained thanks for the video!! really nailed those animations as well, would be cool to make a video on adam/rmsprop as well, i have a hard time properly understanding why they work. anyway much love to you my friend

[@samuelschonenberger]

Watching this while my first VAE is training

[@JONK4635]

Really amazing content, thank you for spreading knowledge! Thanks a lot :)

[@Deepia-ls2fo]

Thanks for the comment, keeps me motivated :)

[@samcoding]

This is so clear! Can you please create a video on Sparse Autoencoders?

[@Deepia-ls2fo]

It's somewhere on the to-do list!

[@KennethTrinh-cm6cp]

thanks for the wonderful animations and explanation

[@Deepia-ls2fo]

Thanks

[@griterjaden]

Wowowowowowow 🎉🎉🎉 amazing video for VAE. Pls ~ make more videos

[@Deepia-ls2fo]

@@griterjaden Thanks, I'm on it :)

[@authenticallysuperficial9874]

Thanks!

[@EdeYOlorDSZs]

top tier video!

[@chriskamaris1372]

Furthermore, in 11:39 and 12:39 you are referencing σ as variance. But is it σ the standard deviation and σ^2 the variance? (Nevertheless, the video is perfect. Excellent work!)

[@Deepia-ls2fo]

Thanks, indeed there might be some mistakes !

[@hannes7218]

good stuff! keep it going

[@nihilsson]

Great vid! Commenting for algorithmical reasons

[@Deepia-ls2fo]

Thanks !

[@chcyzh]

Thank you very much! It's pretty clear

[@chnchn-z9m]

Thank you for your explanation

[@5h4n6]

Thank you so much foe the video!

[@잇준-v7m]

so helpful video, i want to see about advanced VAE

[@Deepia-ls2fo]

Thanks, might revisit VAEs when I'll be talking about latent diffusion :)

[@Jay_Tau]

This is excellent. Thank you!

[@Deepia-ls2fo]

Thanks !

[@metarestephanois3262]

You said that in the beginning the idea is sampling from p(z|x) gives a latent vector z that has likely come from the original distribution p(x). What does this exactly mean that I value can likely come from a distribution ? Thanks in advance for an answer.

[@Deepia-ls2fo]

Well it's a good question, this is somewhat of a misuse of language. What it means is that if we take a latent z it should have a high probability with respect to this conditional probability distribution. Knowing how to sample from this distribution means if we have an image from p(x), we should be able to find latents that are probable with respect to this image. Hope this clears things up :)

[@metarestephanois3262]

@ Ok I think that helped, thank you

[@michael91703]

Is this manim?!!! Nice work dude!

[@Deepia-ls2fo]

It is indeed Manim, thank you !

[@guilhermegomes4517]

Great Video!

[@Deepia-ls2fo]

Thanks !

[@woowooNeedsFaith]

15:49 - What is convex interpolation?

[@Deepia-ls2fo]

Basically a linear interpolation between two points, with "t" in front of one of the points, and "(1-t)" in front of the other. The set of all these points is convex, hence "convex interpolation" :)

[@markbuckler4793]

Excellent video, I subscribed because of it :)

[@Deepia-ls2fo]

thanks !

[@rishidixit7939]

At 7:45 why is the assumption for p(z) as a Normal Distribution important ? Without that are further calculations not possible ? At 8:01 why is the posterior assumed to be Gaussian ?

[@Deepia-ls2fo]

@@rishidixit7939 Hi again, indeed further calculations are intractable without assuming both the prior and the posterior to be Gaussian. Some other research works have replaced these assumptions by other well known distributions such as mixtures of Gaussians, which results in another training objective.

[@notu483]

12:40 we scale by standard deviation not variance

[@diabl2master]

4:50 Wouldn't you use subscripts to show they are different functions?

[@Deepia-ls2fo]

@@diabl2master yes you can ! There are a lot of different conventions in stats

[@hnull99]

07:05 I don't really get the reasoning here. How is it that a image (x) is generating the latent variables (z). I would have thought that the latent variables generating the images instead.

[@Deepia-ls2fo]

Well you can generate z from x using the encoder !

[@Chadpritai]

Next video on diffusion models please , thanks in advance ❤

[@Deepia-ls2fo]

It's on the to-do list but the next 3 videos will be about self-supervised learning !

[@user-wr4yl7tx3w]

Great content

[@sillasrocha9623]

Hey, could you make a video talking about swav in unsupervised learning?

[@nicolastortora7356]

Hi sir, at 7:54 you said like: "the knowledge of prior P(z) allows to compute the likelihood P(x|z)". This is, in general, not true. But, in this scenario, it seems to be the case and you explain it at 9:40. Am I correct? Actually, we compute the EXPECTATION[ log(P(x|z)] , not really the P(x|z), and we do that by just pixel-wise comparison. 1st Question: Do we like this differnce? Furthermore, the EXPECTATION of a (log)-probability..? This makes me some confusion. 2nd Question: What really is the expectation of a probability? By the way, really really compliments for the video, the animation and the quality of your work, thanks a lot!! Nicolas

[@Bikameral]

Great content ! What software are u using to animate ?

[@Deepia-ls2fo]

Thanks ! For most animations I use Manim, a python module originally made by Grant Sanderson from 3blue1brown.

[@Bikameral]

@@Deepia-ls2fo thank you

[@HosseinKhosravipour]

So great.Thanks

[@metarestephanois3262]

Thank you for the video but why is the Data reconstruction term L2 ? I thought this is maximizing log likelihood .

[@Deepia-ls2fo]

Thanks for your comment, for Gaussian noise this is equivalent :)

[@maths.visualization]

Can you share video code?

[@Deepia-ls2fo]

The link is in the description!

[@English-bh1ng]

It is the best VAE visualization.

[@prabaldutta1935]

Amazing Graphics and explanation. I have one question - if we use MNIST dataset (like what is shown in the video) does it mean that the mu and sigma are vectors of dimension 10x1? What if we use a dataset where the number of different classes are unknown? What will be the dimension of mu and sigma in that case?

[@Deepia-ls2fo]

Thank you, the latent dimension is not directly related to the number of classes in your dataset. In fact a very good encoder could very well classify perfectly the 10 classes on a single dimension, but it makes things way harder to reconstruct for the decoder. As you mention in most datasets we don't even know the number of classes or the number of relevant features, so we just take ad hoc latent dimensions (16, 32) and see if it's enough for the encoder to produce a useful representation, and for the decoder to reconstruct correctly.

[@prabaldutta1935]

@@Deepia-ls2fo Thanks a lot for your response. Can't wait for your next video.

[@neerajsingh-xf3rp]

0:23 does it create data from scratch?

[@Deepia-ls2fo]

Yep, basically modern image generation techniques (diffusion models / flow matching) create new data starting from pure noise !

[@neerajsingh-xf3rp]

@@Deepia-ls2fo does it learn from existing data ? , if yes how does it generate data from scratch , denoising involves learning the state and adding some randomness in that state only 🤔

[@cupatelj52]

great content bro.

[@Deepia-ls2fo]

Thanks

[@aregpetrosyan465]

This question came to my mind: What would happen if we ignored the encoder part and tried to train only the decoder? For example, by sampling from a standard Gaussian vector and attempting to reconstruct a digit. I don't really understand the purpose of the encoder.

[@Deepia-ls2fo]

If you don't condition at all the latent space from which you are sampling, I'm not sure the model will be able to learn anything. Here the encoder explicitly approximate the posterior distribution in order for us to then sample from the distribution of images. This is all a theoretical interpretation of course, but learning to reconstruct any digit from pure unconditioned noise seems a bit hard! Diffusion models kind of do it (in image space), but this usually takes a lot of steps. Anyway, the experiment you describe would be very easy to implement, if you want to try it out. :D

[@gw1284]

Hey, cool animated lecturing, can you share how and what software was used? Thanks

[@Deepia-ls2fo]

Thanks, I use DaVinci resolve for the video editing, Manim python library for the animations and Elevenslab for the voice !

[@gw1284]

@Deepia-ls2fo Thank you. Both content and format of your videos are of high quality. Will look into those tools.

[@notu483]

Thanks for the video ❤😊

[@Deepia-ls2fo]

Thank you !

[@i2c_jason]

Is there a statistical property or proof that might show a graphRAG "transfer function" to be the same as a VAE or maybe a CVAE? Perhaps in terms of entropy? It would be interesting to make two identical systems, one using a VAE and one using graphRAG, and see if they can match up statistically. I can't shake the idea that software 3.0 might be the more sound approach for developing new GenAI tools vs software 2.0.

[@Deepia-ls2fo]

Hi Jason ! Unfortunately I know close to nothing about RAG so I have no idea if what you describe might be feasible. I here about RAG everywhere these days, I should get up to date on that.

[@i2c_jason]

@@Deepia-ls2fo I'd love to hear your take on it if you ever do a deep dive.

[@EigenA]

Great video. What is your educational background?

[@Deepia-ls2fo]

Thanks ! Bachelor in math, bachelor in computer science, master in AI/ML, currently doing a PhD in applied maths and deep learning

[@EigenA]

@ legendary. Good luck on the PhD! I’m 3rd year EE PhD student, you have phenomenal content. Looking forward to watching your channel grow.

[@3B1bIQ]

🤍Please, can you create a course to learn the manim library from scratch to professionalism, because I need it very much? Please reply ❤😊

[@Deepia-ls2fo]

Thanks for your comment, I would love to but I have many other topics I want to talk about first, and not much time on my hand! There are very good ressources on KZbin though, if you want to start to learn Manim. :)

[@3B1bIQ]

@@Deepia-ls2fo Thank you, but I hope that you have enough time to create a course to learn manim, even if there is one video every week, and this will contribute to increasing the number of your views more because your explanation is very beautiful and clear, and I can understand it easily even though I am an Arab 🤍☺️

[@awsaf49]

Hey, nice video!

[@shashankjha8454]

do u use manim for animations ?

[@Deepia-ls2fo]

@@shashankjha8454 Yes indeed!

[@syc52]

Could you please make a video talking about why diffusion model, GAN, and VQVAE can make the image sharper

[@NguyenAn-kf9ho]

is there we have videos with the same approach, for Reinforcement Learning :D ???? !

[@Deepia-ls2fo]

Hi, unfortunately I don't know anything about Reinforcement Learning, so I don't think I'll be able to make videos about that any time soon. However, I believe Steve Brunton has very good videos on the topic :)

[@rrttttj]

Great video! However, I am slightly confused: For your loss function you are subtracting KL divergence rather than adding it. Wouldn't you want to add it to penalize the difference between the latent distribution and the standard normal distribution? At least, in all implementations I have seen they add KL divergence rather than subtract it. Edit: I understand my mistake now!

[@Deepia-ls2fo]

Hi ! Thanks for the comment, I'm afraid I might have flipped a sign at one point. When you derive the ELBO (which you then maximize via training), there is a minus sign appearing in front of the KL. But in practice you minimize the opposite of this quantity, which is equivalent to minimizing the L2 plus the KL. I hope it's not too confusing. :)

[@rrttttj]

@@Deepia-ls2fo Oooooh I understand, so ELBO is the quantity that should be maximized, and you were denoting the ELBO quantity with L(x), not the loss itself. I understand now, thanks!

[@homakashefiamiri3749]

it was great!

[@arno7198]

DeepIA absolutely killed it with this video on Variational Autoencoders. As a government official, medical doctor, and law PhD, it's not often I come across something that genuinely teaches me something new. But this video? Wow. The way Variational Autoencoders map data to a latent distribution instead of a fixed point, and the balance between reconstruction loss and Kullback-Leibler divergence, was explained so clearly that I picked it up right away. Whether I'm shaping policies, treating patients, or analyzing legal cases, this video added value in ways I didn’t expect. Props to DeepIA for delivering content that even someone as busy (and brilliant) as me can appreciate! And let’s not forget the genius behind it all. Honestly, the mind that creates content like this is nothing short of extraordinary. I don’t say this lightly, but DeepIA might just be the most insightful, brilliant, and generous creator on KZbin. The precision, the depth, the clarity-it’s rare to find someone who can not only understand such complex topics but also make them accessible to mere mortals like us. It’s an honor to witness this level of mastery. Truly, we’re not worthy.

[@Deepia-ls2fo]

thx 🤖

[@martinferrari2903]

Rien que ça 🤣

[@이승민-o7m]

awesome

[@frommarkham424]

This vid fye

[@PascalYamlome]

Wao!! Just Wao!!!

[@Deepia-ls2fo]

thanks 😁