From Autoencoders to Variational Autoencoders: Improving the Encoder

Autoencoders have a number of limitations for generative tasks. That’s why they need a power-up to become Variational Autoencoders. In this video, I explain the first step to transform a vanilla autoencoder into a VAE. Specifically, I discuss how VAEs use multivariate normal distributions to encode input data into a latent space and why this is awesome for generative tasks. Don’t worry - I also explain what multivariate normal distributions are!
===============================
Slide deck:
github.com/musikalkemist/gene...
Join The Sound Of AI Slack community:
valeriovelardo.com/the-sound-...
===============================
Interested in hiring me as a consultant/freelancer?
valeriovelardo.com/
Follow Valerio on Facebook:
/ thesoundofai
Connect with Valerio on Linkedin:
/ valeriovelardo
Follow Valerio on Twitter:
/ musikalkemist
===============================
Content
0:00 Intro
0:32 Issues with vanilla AEs
1:03 From AEs to VAEs
1:46 Encoder mapping: AEs vs VAEs
2:40 Univariate normal distribution
11:37 Multivariate normal distrivution
16:13 Usage of multivariate normal distribution in VAEs
22:06 How multivariate normal distribution solves discontinuities in VAEs

[@yashtrivedi2702]

I am a master's student doing research in AI. After watching 1000 videos on KZbin this is the simplest and the easiest way to learn and understand VAE. Salute to your level of knowledge.

[@ValerioVelardoTheSoundofAI]

Thanks!

[@vladimirbosinceanu5778]

I would have to agree. Some people are just more natural than others at teaching.

[@dhruvmatani7824]

This is probably the clearest explanation of this concept (yet) that I have found online - I love your style! :)

[@ratkapna]

Definitely interested in a probability/statistics course!

[@samuelgbashi8969]

Wow Valerio, this is a great piece. I must admit that i have never come across a good teacher like you. Thumbs up.

[@ValerioVelardoTheSoundofAI]

Thanks!

[@hackercop]

Sir, I've watched many videos on Variational Autoencoders and this was the best explanation thanks

[@hermanhjaramillo]

Excellent Course. Thanks for sharing!

[@asheeshmathur]

Wonderful. you with basic and gradueryally reach top effortlessly. Fortunate to catch you. God Bless You

[@vaitom6078]

love the trick about making variance a non positive only value 🙂

[@mahmodaldahol]

You are amazing valerio. Thank you for you great channel

[@paedrufernando2351]

your videos are as clear as a dew drop in the ocean of videos on this topic out there...thanks to your efforts.. and really well explained.. your one of the best ML experienced teacher's out there among the research intellects.

[@ValerioVelardoTheSoundofAI]

Thank you Paedru!

[@vaitom6078]

Same for me, probability and statistics course is a must in ML field, thank you for your work :)

[@SteveNiafas]

Can't wait for the next. Exceptional work!

[@ValerioVelardoTheSoundofAI]

Thanks!

[@markusbuchholz3518]

The math is always correct. The outliers represents the minority of general population (observations), meaning very far away from the the mean value as this exceptional (outlier) YT channel! Thanks Valerio for spreading the beauty of knowledge!

[@ValerioVelardoTheSoundofAI]

Thank you Markus :)

[@souravsanyal6183]

Fantastic....thanks a ton. Keep it up.

[@jamalseyedmohammadi6681]

You are amazing valerio. Thank you for you great channel ;)

[@ValerioVelardoTheSoundofAI]

Thanks!

[@haroldsu1696]

great video!

[@user-cc3eu4ng7f]

Amazing video Thank you so much

[@fatimadarbandsari1870]

It was great, thank you

[@dhruvmatani7824]

Valerio, at around the 20 minute mark, you mention that the capital Sigma vector can be considered to be the variance vector. I was assuming this to be the vector of standard deviations. Could we consider it it to be a vector of standard deviations instead? Also, why would it be a problem if the standard deviations are always positive? Won't there be negative values in small epsilon vector (which represents a point in the higher dimensional space), causing the values to be spread on both sides of the means in their respective dimensions?

[@drjenschn]

At 12:22 - the two distributions don't really look like they are independent: the standard ellipsoid is not aligned with the axes, indicating non-vanishing off-diagonal elements in the covariance matrix of p(X), p(Y)... The equation following the picture is correct, since it includes the general covariance case. At 14:05, it's not proportionate to the Mahalanobis distance, it is exactly the *squared* Mahalanobis distance. At 19:13, I think the epsilon should be multiplied with L, where L^TL = Sigma comes from a Cholesky decomposition (if my mind serves correctly)... The way it continues with pow(.../2) is correct again for diagonal matrices Sigma.

[@phamquang5535]

thanks dude

[@Jononor]

Would love to hear some audio samples generated with these methods. Would AudioMNIST be a suitable maybe (speech of characters 0-9)? Or maybe a dateset of notes with different timbre?

[@alonalon8794]

Interested in probability and statistics course too