The KL Divergence : Data Science Basics

understanding how to measure the difference between two distributions
Proof that KL Divergence is non-negative : • Jensen's Inequality : ...
My Patreon : www.patreon.com/user?u=49277905
0:00 How to Learn Math
1:57 Motivation for P(x) / Q(x)
7:21 Motivation for Log
11:43 Motivation for Leading P(x)
15:59 Application to Data Science

[@szymonk.7237]

Wow... 😳 I've never seen more genius, easy and intuitive explanation of KL-div 😳👏👏👏👏👏 Big thanks good man ! ❤️

[@ritvikmath]

Glad you liked it!

[@anilaxsus6376]

I agree,

[@murkyPurple123]

Your bottom-up (instead of top-down) approach that you mentioned in the beginning of the video would be really great to see for all kinds of differrent concepts!

[@ritvikmath]

Great idea!

[@DS-vu5yo]

That was the best description of why we use log that I have ever seen. Good work, man.

[@zafersahinoglu5913]

I am a research scientist. You provide a clear and concise treatment of KL-Divergence. The best I have seen to date. Thanks.

[@AdeOlubummo]

Just fantastic! Even if I forget the formula for KL divergence, I can "re-engineer" it on demand.

[@varadpuntambekar8895]

I don't think I'm ever going to forget this. Thanks so much.

[@user-li5lh1qs6s]

I'm in the middle of a $2,500 course, BUT → KZbin → your video... 👏🏻👏🏻👏🏻👏🏻👏🏻 Thank you for starting with the "why", and appealing to my brains desire to understand, not just do.

[@tanvirazhar]

Amazing. The pace you have explained, the approach...everything is just top-notch.

[@asimosman3428]

This video is absolutely mind-blowing! The way it breaks down such a complex concept into an intuitive understanding is truly remarkable. Thank you!

[@JBoy340a]

That was great. I have struggled to understand certain aspects of KL Divergence, and this is a great way to think about it without getting bogged down in symbology.

[@ritvikmath]

Glad it was helpful!

[@markozege]

Thank you for this, the best explanation of KL divergence that I have seen. Love how you approach it building gradually, really inspiring for how to learn math.

[@eagermage3157]

Best Math Teacher ever. So clearly explained the design and thinking process of how the algo comes out. Many video just explain the formula which confused me why we should do this way... Thank you!

[@mrcaljoe1]

I think you're channel and teaching style is brilliant. I wish I knew about this channel when I was doing my undergrad.

[@anujadassanayake6202]

Great explanation, this is the first time I'm learning about KL divergence and it was very easy to grasp because of the way you taught it

[@thankyouthankyou1172]

I found out this professor is very good at explaining every tough concept! respect and many appreciations!

[@trungphan9137]

This is mind blowing.... I love the way you go from the problem to the solution, it's clever way to understand this KL divergence

[@ritvikmath]

thanks!

[@tampopo_yukki]

I love how you approach to the KL divergence!

[@julianwebb9222]

That was great! Not just dumping the formula on you but walking you through its logic with simple steps. Loved it! ❤

[@marka5968]

Wow. This is the best explanation of KL-divergence I've ever heard. So many over-complicated stuff out there but yours is absolutely genius.

[@ritvikmath]

Glad it was helpful!

[@andrashorvath2411]

Fantastically clearly explained, congrats.

[@trentbolt2006]

You've really made my day with ur explanation. Thank you so much :D

[@Mars.2024]

Everytime i have a math question your hannel is my first choice! Amazing ✅ thanks a million 🎉

[@tayyibulhassan6227]

One of the BEST tutorials for sure

[@godlyradmehr2004]

The best explanation I've ever seen about KL divergence ❤

[@paigecarlson1742]

Outstanding. Really helping me through this info retrieval course!

[@somdubey5436]

superb...I believe this is the best explanation I have ever come across for K L Divergence. Thanks a tonne.

[@ChocolateMilkCultLeader]

This is the perfect video in Math. Love it. Shared with all my readers

[@shamarbauyrzhan7997]

Let's celebrate a new video on this amazing chanel!!! Love your work!

[@ritvikmath]

🎉

[@winstongraves8321]

This was awesome. Really helpful to think through it backwards and “redevelop” our own function

[@midnightwanders5876]

Great work! I've been a fan of your ,material for some time and in this video you have truly mastered your craft.

[@ritvikmath]

Wow, thank you!

[@danscherb4130]

Another amazing video! Please keep them coming!

[@yhoang6674]

In the 'Motivation for log,' you said that taking a simple average is not the right way to go, and then you try to find a function that makes f(4) and f(1/4) have opposite signs. That means you are trying to make two very different distributions have the smallest distance possible (canceling each other out), which is contradictory to what we expected. We expected them to be large.

[@joesavage9077]

Wow!!! This approach to explaining was mind "opening". I got it! Thanks so much

[@Hobbies_forkids]

Excellent way to explain it. Makes maths sounds logical and approachable 🎉

[@mehmetozkan1479]

I have never seen complex math explained this good Thank you very much!

[@momcilomrkaic2214]

Your videos are great just keep going, I watched you for few years already

[@user-vb1no5lq1e]

me not know some of the fundamentals after listening to your explanation made a lot of sense, and I felt I understood the concept well. I am willing to watch your videos more often.

[@0hexe]

Amazing video, love the format!

[@kasyaci]

That was one of the best explanations I have ever heard! Great job and many thanks!

[@ritvikmath]

Thanks!!

[@MagmaMusen]

Thanks!

[@tom199520000]

wahh.. i am studying computer science master degree. Your video really helps me a lot! please keep on doing such great work for us!

[@manducchuc915]

Thanks, exactly the explanation I have been looking for!

[@seansullivan6986]

Excellent intuitive explanation!

[@vzinko]

Thank you as always for sharing your brilliant teachings, Ritvik. Could you please do a video on the Gram-Schmidt process and how orthonormal basis matrices are relevant to data science?

[@vorushin]

Thank you for the great explanation! I totally agree that math is not given from above, but invented by people. And showing how the invention can be done is the best way to teach the new concepts. Thanks a lot!

[@hpp496videos]

This was incredibly illustrative!

[@Justin-zw1hx]

dude, the explanation is so good, you rock!

[@ritvikmath]

Glad it helped!

[@brandonkim4675]

I recently got interested in learning machine learning and stumbled upon the stable diffusion, the current state of art open source image generation ai. That's where I encountered the KL divergence. The more I try to understand it, more complicated concepts and formulas are thrown at me. I managed to find some videos that explains how to derive it, but none of them explained why the hell logarithm is present in it for gods sake! And here you are, explaining every missing details from other videos and blog posts in a way that the person who knows very little about the subject can understand in a very satisfying and easy to follow way. Hats off to you, sir. I wish every teachers are like you.

[@ritvikmath]

Thanks and godspeed for your journey through machine learning !

[@Andy-qi5nh]

Amazing teaching. It helps a lot in my process of data shift covariate detection project. Thanks

[@ritvikmath]

Glad it was helpful!

[@sandipmehta2950]

amazing explanation. not many can do this. well done.

[@RakshithReddy5555]

Blew my mind, I wanted to understand what kl divergence is to understand the recent Gen AI papers and couldn't. This video helped me a lot.

[@user-co6pu8zv3v]

Thank you! This is the best explanation of KL divergence wich i've seen

[@ritvikmath]

Glad it was helpful!

[@chenqu773]

Thank you very much! Besides the "norminal" category in your example, I am wondering if this can also be used in "ordinal" category. For example, if I make a questionare from "dislike" to "like very much" and get poll from 2 groups, can I use the KL-divergence to calculate the difference between these 2 groups, and whether there is a even better way to discribe this difference, for example, group2 shows "significant" higher interest than group1 ?

[@SSJVNN]

The comments didn't lie you actually explained this so well. I watched the ads all the way through btw.

[@PrajwalSingh15]

Thank you so much for this explanation and also got a new insight about the log :)

[@ritvikmath]

Happy to help!

[@razgaon3680]

Best video I've seen in a while!

[@ritvikmath]

Thanks!

[@akhileshpandey123]

very nice explanation. Thanks for the work.

[@yb801]

This is an amzing explanation, thanks!

[@ritvikmath]

Glad it was helpful!

[@ringo8530]

you are way more better than my school's professor. thank you

[@aliksargsyan2035]

Thank you. As usual, great and very intuitive explanation.

[@ritvikmath]

No problem !

[@vaibhavnakrani2983]

Awesome! Very intuitive

[@sparkgin]

That was amazing. Thank you so much!

[@s.prakash7869]

Awesome explaination. Thanks for this video

[@ritvikmath]

Glad it was helpful!

[@keyvan4680]

thank you for the clear explanation.

[@andrew-qf4xl]

the thing you said in the first minute, is something ive been saying for a while now. As students we arent told what problem drove scientists or engineers into constructing new formulas or ways of thinking.

[@fh3652]

Great stuff... Learning a way to teach maths to my kid... A constructivist method... While learning about stats... I really appreciate your work.

[@ritvikmath]

Glad it was helpful!

[@jackritwik09]

mad respect for Ritvik from Ritwik for acing the subtle art of intuitive explanation:)) If only professors could master the same art.

[@gingerderidder8665]

Taking the MITx Stats class, but I find that you explain the concepts so much better!

[@ritvikmath]

Glad to hear!

[@aisniper4095]

Amazing explanation!

[@kaanefe4266]

God level explanation thank you!!!

[@barbaraalexandrova6680]

thank you for the best expanation on this topic

[@komuna5984]

Thanks a lot for sharing the underlying motivation behind the K-L divergence! I really needed such deep insights! JAJAKALLAH...

[@ritvikmath]

You're so welcome!

[@devindoinmonkmode]

Wonderful man. Thank you so much.

[@s.m.tahsinzaman_2720]

Thank you SO much! God bless you Sir, keep up the great work 😊

[@ritvikmath]

You are very welcome

[@rishi2504]

You just got a subscriber. Thank You! 😊

[@talsveta]

Great explanation

[@jtkklb]

Very good explanation

[@seyyedmahdihoseini3084]

very very very very well explained, Thanks

[@mariusschmidt6883]

Wow. Just wow! This is brilliant🤩

[@ritvikmath]

Thanks!

[@ingenierocivilizado728]

Thank you very much for your valuable videos!!

[@ritvikmath]

Glad you like them!

[@afsanarabeya4417]

thank you. this really helped !!

[@sikalee415]

You are great in explaining this! Thanks!

[@ritvikmath]

Glad it was helpful!

[@mantische]

It was the easiest explanation I’ve ever seen.

[@qiguosun129]

Thanks for the lecture, your work is always so intuitive.

[@ritvikmath]

You are very welcome

[@abironnoy3115]

Thank you so much for the explanation. it was really helpful👍👍

[@ritvikmath]

Thanks!

[@yorker0507]

Super clear !

[@edalexander9649]

I have a question, if the log sign / ln or log2 whatever, are used to balance the same contribution from different P(X)/Q(X) ratio for random variable X across different value, i.e. X_i, why don't i just use the multiplication symbol (pie) 1/4*4 = 1? although i know that it may not be working well, like the number would be small due to X_i, but why? thanks

[@DataScienceAI-rf4kx]

Thank you :) for valuable content

[@sunset6109]

Bro is a legend

[@jayalekshmi936]

I am a masters student in data science and machine learning and I have to tell you that this is the best explanation one can get for concepts like this...Hope you make more videos on these types in concepts.

[@ritvikmath]

Wow, thanks!

[@chandrashekaravula1292]

Amazing explanation

[@ritvikmath]

Glad you liked it

[@orenkoriat]

great explanation!

[@ritvikmath]

Glad you think so!

[@shadabalam2122]

Great explanation 👏

[@ritvikmath]

Glad you think so!

[@TheFirebolt2010]

It would be interesting to have a video on how you study to understand a topic, what resources you use and the materials you look for

[@shuweiPeng-id4xv]

讲的真好，谢谢

[@user-xz9wf8ou5s]

Would you please have another video about Jenson-Shannon divergence?

[@anonymousiguana168]

You're just phenomenal

[@ritvikmath]

Thanks!

[@RAHUDAS]

That was really awesome

[@luisgoogle8098]

How can this guy only have 8,000 views on such a good video... Very nice way of explaining!

[@ritvikmath]

Wow, thank you!

[@physicsfaith]

Great job