Variational Autoencoders | Generative AI Animated

[@essentiallifter]

this account is seriously underrated, will definitely blow up soon

[@melanynadine972]

Agree, I just subscribed ❤

[@joshuat6124]

Agreed 💯

[@authenticallysuperficial9874]

💯

[@Higgsinophysics]

I can't believe how calmly and clear you explain difficult topics!

[@Deepia-ls2fo]

Well that's the magic of text to speech 😁

[@viewer8221]

haha I knew it was AI voice

[@xichengwang1319]

I first saw this in Chinese media with Chinese subtitles, then just came back to subscribe to the original author. The most clear introduction ever seen with such nice proper animation. It will blow up for sure.

[@Deepia-ls2fo]

Well thank you ! Can you send me more info about that through my email ? [e-mail]

[@anthonyortega358]

This channel is amazing, you should be very proud of what you have produced Thibaut!!

[@Deepia-ls2fo]

Thanks !

[@AtiqurRahman-gj6mg]

Finally, the concept of VAE is clear. Thanks a ton.

[@Deepia-ls2fo]

You're welcome, thanks for the comment !

[@ashutoshpadhi2782]

The amount of effort you put into these works is really commendable. You are a blessing to humanity.

[@BenjaminEvans316]

Great video. Looking forward to your video on contrastive learning, it is my favourite subject in deep learning. Your videos combine great production skills (animations, colour selection, movement between frames) with in-depth understanding of complex concepts.

[@Deepia-ls2fo]

Thanks for the kind words !

[@CharlottePecolo]

Wow it must be a lot of work to obtain such amazing animations! The video is really dynamic and easy to follow, congratulations

[@julius4858]

The program is called manim, it’s from 3blue1brown

[@muhammed2174]

You're a master at your craft, it is a testament to your studies!

[@EkalabyaGhosh-g3i]

This channel needs more subscribers

[@Deepia-ls2fo]

Thank you !

[@babatundeonabajo]

This is a really good video, and the animations are top-notch. I feel this video is good not just for those learning about AI but also those learning Statistics.

[@Bwaaz]

Amazing quality, hope the channel takes off ! Great use of manim

[@Deepia-ls2fo]

Thanks !

[@BeeepBo0op]

Thank you for finally making me understand the reparametrization trick!! It was thrown at me several times during a DRL class I took last year and I never really understood what we did. This made it much much clearer, thank you! Also: great video overall!

[@Deepia-ls2fo]

Glad it helped !

[@Tunadorable]

3blue1brown specifically for AI models??? sign me up!!! I'll fs be linking to you in my own vids whenever relevant, this was great

[@MutigerBriefkasten]

Thanky you again for the great content and the amazing animations 🎉💪👍 keep going, hopefully your Channel explode with more subscribers .. i will recommend it for sure to other people

[@Deepia-ls2fo]

Thank you :)

[@hichamaniba9041]

Thank you , still looking for VAE variants videos

[@mostafasayahkarajy508]

Thank you very much for providing and sharing the lecture. Excelent explanation and so a high-quality video!

[@Deepia-ls2fo]

Thank you !

[@AICoffeeBreak]

Love the clarification at 00:13, because I've also felt that the misconception is wide spread. I've heard people say: I am not using GANs anymore, I am using Generative AI. The word "Generative" is literally what G in GAN stands for. 😂😂

[@Deepia-ls2fo]

I made this intro because I couldn't stand the number of "GenAI experts" on my LinkedIn feed :(

[@kacperzaleski8125]

cant wait for the next video! this was great!!!!!

[@Deepia-ls2fo]

Thank you !

[@abelhutten4532]

Great visualizations, and good explanation! Congratulations and thanks for the nice video :)

[@Deepia-ls2fo]

Thank you !

[@abdelmananabdelrahman4099]

Great video 🎉. I've never had such a great explanation of VAE. Waiting for VQVAE.....

[@Deepia-ls2fo]

Thank you !

[@drannoc9812]

Amazing content :D I hope you'll do your next videos on VQ VAE and VQ VAE 2, I enjoyed so much reading those papers !

[@Deepia-ls2fo]

Thanks, I really gotta take another look to the paper

[@carlaconti1075]

Thanks to you everything is clear now, thank you Deepia

[@Deepia-ls2fo]

thanks miss conti

[@vitorfranca80]

Incredible explanation!! Thank you for sharing your knowledge! 😁😁

[@Deepia-ls2fo]

Thanks !

[@shangliu6285]

perfect video, its easy to understand the VAE, subscribed!

[@Deepia-ls2fo]

Thanks !

[@samuelschonenberger]

Watching this while my first VAE is training

[@lucianovidal8721]

Great content! It was a really good explanation

[@cs-cs4mj]

hey, so well explained thanks for the video!! really nailed those animations as well, would be cool to make a video on adam/rmsprop as well, i have a hard time properly understanding why they work. anyway much love to you my friend

[@JONK4635]

Really amazing content, thank you for spreading knowledge! Thanks a lot :)

[@Deepia-ls2fo]

Thanks for the comment, keeps me motivated :)

[@authenticallysuperficial9874]

Thanks!

[@flueepwrien6587]

Finally I understand this concept.

[@KennethTrinh-cm6cp]

thanks for the wonderful animations and explanation

[@Deepia-ls2fo]

Thanks

[@nihilsson]

Great vid! Commenting for algorithmical reasons

[@Deepia-ls2fo]

Thanks !

[@chcyzh]

Thank you very much! It's pretty clear

[@griterjaden]

Wowowowowowow 🎉🎉🎉 amazing video for VAE. Pls ~ make more videos

[@Deepia-ls2fo]

@@griterjaden Thanks, I'm on it :)

[@Chadpritai]

Next video on diffusion models please , thanks in advance ❤

[@Deepia-ls2fo]

It's on the to-do list but the next 3 videos will be about self-supervised learning !

[@English-bh1ng]

It is the best VAE visualization.

[@Jay_Tau]

This is excellent. Thank you!

[@Deepia-ls2fo]

Thanks !

[@michael91703]

Is this manim?!!! Nice work dude!

[@Deepia-ls2fo]

It is indeed Manim, thank you !

[@markbuckler4793]

Excellent video, I subscribed because of it :)

[@Deepia-ls2fo]

thanks !

[@Bikameral]

Great content ! What software are u using to animate ?

[@Deepia-ls2fo]

Thanks ! For most animations I use Manim, a python module originally made by Grant Sanderson from 3blue1brown.

[@Bikameral]

@@Deepia-ls2fo thank you

[@hannes7218]

good stuff! keep it going

[@EdeYOlorDSZs]

top tier video!

[@chriskamaris1372]

Furthermore, in 11:39 and 12:39 you are referencing σ as variance. But is it σ the standard deviation and σ^2 the variance? (Nevertheless, the video is perfect. Excellent work!)

[@Deepia-ls2fo]

Thanks, indeed there might be some mistakes !

[@EigenA]

Great video. What is your educational background?

[@Deepia-ls2fo]

Thanks ! Bachelor in math, bachelor in computer science, master in AI/ML, currently doing a PhD in applied maths and deep learning

[@EigenA]

@ legendary. Good luck on the PhD! I’m 3rd year EE PhD student, you have phenomenal content. Looking forward to watching your channel grow.

[@guilhermegomes4517]

Great Video!

[@Deepia-ls2fo]

Thanks !

[@notu483]

Thanks for the video ❤😊

[@Deepia-ls2fo]

Thank you !

[@syc52]

Could you please make a video talking about why diffusion model, GAN, and VQVAE can make the image sharper

[@sillasrocha9623]

Hey, could you make a video talking about swav in unsupervised learning?

[@cupatelj52]

great content bro.

[@Deepia-ls2fo]

Thanks

[@prabaldutta1935]

Amazing Graphics and explanation. I have one question - if we use MNIST dataset (like what is shown in the video) does it mean that the mu and sigma are vectors of dimension 10x1? What if we use a dataset where the number of different classes are unknown? What will be the dimension of mu and sigma in that case?

[@Deepia-ls2fo]

Thank you, the latent dimension is not directly related to the number of classes in your dataset. In fact a very good encoder could very well classify perfectly the 10 classes on a single dimension, but it makes things way harder to reconstruct for the decoder. As you mention in most datasets we don't even know the number of classes or the number of relevant features, so we just take ad hoc latent dimensions (16, 32) and see if it's enough for the encoder to produce a useful representation, and for the decoder to reconstruct correctly.

[@prabaldutta1935]

@@Deepia-ls2fo Thanks a lot for your response. Can't wait for your next video.

[@awsaf49]

Hey, nice video!

[@3B1bIQ]

🤍Please, can you create a course to learn the manim library from scratch to professionalism, because I need it very much? Please reply ❤😊

[@Deepia-ls2fo]

Thanks for your comment, I would love to but I have many other topics I want to talk about first, and not much time on my hand! There are very good ressources on KZbin though, if you want to start to learn Manim. :)

[@3B1bIQ]

@@Deepia-ls2fo Thank you, but I hope that you have enough time to create a course to learn manim, even if there is one video every week, and this will contribute to increasing the number of your views more because your explanation is very beautiful and clear, and I can understand it easily even though I am an Arab 🤍☺️

[@rrttttj]

Great video! However, I am slightly confused: For your loss function you are subtracting KL divergence rather than adding it. Wouldn't you want to add it to penalize the difference between the latent distribution and the standard normal distribution? At least, in all implementations I have seen they add KL divergence rather than subtract it. Edit: I understand my mistake now!

[@Deepia-ls2fo]

Hi ! Thanks for the comment, I'm afraid I might have flipped a sign at one point. When you derive the ELBO (which you then maximize via training), there is a minus sign appearing in front of the KL. But in practice you minimize the opposite of this quantity, which is equivalent to minimizing the L2 plus the KL. I hope it's not too confusing. :)

[@rrttttj]

@@Deepia-ls2fo Oooooh I understand, so ELBO is the quantity that should be maximized, and you were denoting the ELBO quantity with L(x), not the loss itself. I understand now, thanks!

[@aregpetrosyan465]

This question came to my mind: What would happen if we ignored the encoder part and tried to train only the decoder? For example, by sampling from a standard Gaussian vector and attempting to reconstruct a digit. I don't really understand the purpose of the encoder.

[@Deepia-ls2fo]

If you don't condition at all the latent space from which you are sampling, I'm not sure the model will be able to learn anything. Here the encoder explicitly approximate the posterior distribution in order for us to then sample from the distribution of images. This is all a theoretical interpretation of course, but learning to reconstruct any digit from pure unconditioned noise seems a bit hard! Diffusion models kind of do it (in image space), but this usually takes a lot of steps. Anyway, the experiment you describe would be very easy to implement, if you want to try it out. :D

[@미비된동작-p4g]

13:05 That’s so funny VAE and Adam both are proposed by same person, Kingma..

[@Deepia-ls2fo]

He's quite the man, also co-author on some key diffusion models paper :)

[@i2c_jason]

Is there a statistical property or proof that might show a graphRAG "transfer function" to be the same as a VAE or maybe a CVAE? Perhaps in terms of entropy? It would be interesting to make two identical systems, one using a VAE and one using graphRAG, and see if they can match up statistically. I can't shake the idea that software 3.0 might be the more sound approach for developing new GenAI tools vs software 2.0.

[@Deepia-ls2fo]

Hi Jason ! Unfortunately I know close to nothing about RAG so I have no idea if what you describe might be feasible. I here about RAG everywhere these days, I should get up to date on that.

[@i2c_jason]

@@Deepia-ls2fo I'd love to hear your take on it if you ever do a deep dive.

[@notu483]

12:40 we scale by standard deviation not variance

[@shashankjha8454]

do u use manim for animations ?

[@Deepia-ls2fo]

@@shashankjha8454 Yes indeed!

[@NguyenAn-kf9ho]

is there we have videos with the same approach, for Reinforcement Learning :D ???? !

[@Deepia-ls2fo]

Hi, unfortunately I don't know anything about Reinforcement Learning, so I don't think I'll be able to make videos about that any time soon. However, I believe Steve Brunton has very good videos on the topic :)

[@asteriskman]

'now that we've got the basics down' ... lol yea ok, professor.

[@arno7198]

DeepIA absolutely killed it with this video on Variational Autoencoders. As a government official, medical doctor, and law PhD, it's not often I come across something that genuinely teaches me something new. But this video? Wow. The way Variational Autoencoders map data to a latent distribution instead of a fixed point, and the balance between reconstruction loss and Kullback-Leibler divergence, was explained so clearly that I picked it up right away. Whether I'm shaping policies, treating patients, or analyzing legal cases, this video added value in ways I didn’t expect. Props to DeepIA for delivering content that even someone as busy (and brilliant) as me can appreciate! And let’s not forget the genius behind it all. Honestly, the mind that creates content like this is nothing short of extraordinary. I don’t say this lightly, but DeepIA might just be the most insightful, brilliant, and generous creator on KZbin. The precision, the depth, the clarity-it’s rare to find someone who can not only understand such complex topics but also make them accessible to mere mortals like us. It’s an honor to witness this level of mastery. Truly, we’re not worthy.

[@Deepia-ls2fo]

thx 🤖

[@martinferrari2903]

Rien que ça 🤣

[@HenrikVendelbo]

Thanks

[@Deepia-ls2fo]

Ho my, thank you so much !

[@HenrikVendelbo]

I find math speak very hard to grok. I was always good at math, but always got turned off by the navel gazing and geekery. You do a great job keeping it engaging without assuming that I am a math geek

[@Deepia-ls2fo]

@@HenrikVendelbo Yeah sometimes it do be like that in math classes. I think it's important to look at equations when they tell us something about the models, but computational tricks or complex equations are not that interesting.

[@rishidixit7939]

At 7:45 why is the assumption for p(z) as a Normal Distribution important ? Without that are further calculations not possible ? At 8:01 why is the posterior assumed to be Gaussian ?

[@Deepia-ls2fo]

@@rishidixit7939 Hi again, indeed further calculations are intractable without assuming both the prior and the posterior to be Gaussian. Some other research works have replaced these assumptions by other well known distributions such as mixtures of Gaussians, which results in another training objective.

[@authenticallysuperficial9874]

Audio comes out from under water at 2:09 btw

[@Deepia-ls2fo]

Thank you, I had some issues with copyrighted music which led to KZbin removing it but also degrading the audio...

[@zyansheep]

Now if only the latent space could be a variable size, and be discrete, then maybe we could do effective ai lossy/lossless compression 🤔

[@Deepia-ls2fo]

Hi, I don't know about variable dimension latent space, but discrete sure sounds like VQ-VAE :)

[@أمديعماد]

You need that music I dont remember it

[@alexm9606]

Comment to up this channel

[@fedorzhdanov6085]

There are so many trashy channels with AI generated nonsense, while some channels (like this) has clear explanation and just few views. I think KZbin should add some "peer-review" feature, and while there is no such tool I encourage support such good channels with likes and comments and hit dislikes for useless AI "blah-blah" channels. I'm not against AI as helper tool (like script writing/voice generation), but if there is no fact checks from authors, that make it garbage and the platform doesn't have proper garbage collector yet.

[@MursaleenFayyaz-lr7hu]

Please create videos on Auto Regressive Models, Particularly RNN, LSTM, PixelCNN as soos as you can. I have a mid exam in third week of October which will cover these topics.

[@duncanwarukira4348]

Is this voice AI

[@Deepia-ls2fo]

Yes I cloned my voice using a text to speech service called elevenlabs

[@timewasting7574]

3:25 - 6:20 is so distracting. Just assume your audience knows these. No need to conform your target group to general public. Just assume senior-year undergraduate please.

[@sheevys]

The auto-generated voice-over is super annoying. Any chance a real human can narrate it?

[@Deepia-ls2fo]

No to be honest that would take way too much time on my side, so it's probably never going to happen. Hopefully text to speech services get better over time!

[@ruslanbip7333]

Thanks