Intuitively Understanding the Cross Entropy Loss

This video discusses the Cross Entropy Loss and provides an intuitive interpretation of the loss function through a simple classification set up. The video will draw the connections between the KL divergence and the cross entropy loss, and touch on some practical considerations.
Twitter: / adianliusie

[@leoxu9673]

This is the only video that's made the connection between KL Divergence and Cross Entropy Loss intuitive for me. Thank you so much!

[@jasonpittman7853]

This subject has confused me greatly for nearly a year now, your video and the kl-divergence video made it clear as day. You taught it so well I feel like a toddler could understand this subject.

[@nirmalyamisra4317]

Great video. It is always good to dive into the math to understand why we use what we use. Loved it!

[@ananthakrishnank3208]

Excellent expositions on KL divergence and Cross Entropy loss within 15 mins! Really intuitive. Thanks for sharing.

[@kvnptl4400]

This one I would say is a very nice explanation of Cross Entropy Loss.

[@bo3053]

Super useful and insightful video which easily connects KL-divergence and Cross Entropy Loss. Brilliant! Thank you!

[@alirezamogharabi8733]

The best explanation I have ever seen about Cross Entropy Loss. Thank you so much 💖

[@yingjiawan2514]

This is so well explained. thank you so much!!! Now I know how to understand KL divergence, cross entropy, logits, normalization, and softmax.

[@shubhamomprakashpatil1939]

This is an amazing explanatory video on Cross-Entropy loss. Thank you

[@viktorhansen3331]

I have no background in ML, and this plus your other video completely explained everything I needed to know. Thanks!

[@Micha-ku2hu]

What a great and simple explanation of the topic! Great work 👏

[@ssshukla26]

And no one told me that (minimizing KL is almost equivalent to minizing CLE) in 2 years studying in a University... Oh man... thank you so much...

[@DHAiRYA2801]

KL = Cross Entropy - Entropy.

[@TheVDicer]

Fantastic video and explanation. I just learned about the KL divergence and the cross entropy loss finally makes sense to me.

[@hansenmarc]

Great explanation! I’m enjoying all of your “intuitively understanding” videos.

[@matiassandacz9145]

This video was amazing. Very clear! Please post more on ML / Probability topics. :D Cheers from Argentina.

[@yfd487]

I love this video!! So clear and informative!

[@allanchan339]

It is a excellent explanation to make use of previous video of KL divergence in this video

[@hasankaynak2253]

The clearest explanation. Thank you.

[@whowto6136]

Thanks a lot! Really helps me understand Cross Entropy, Softmax and the relation between them.

[@franco-ruggeri-old]

Great and clear explanation!

[@newbie8051]

oh wow this was simple and amazing thanks !

[@chunheichau7947]

I wish more professors can hit all the insights that you mentioned in the video.

[@LiHongxuan-ee7qs]

So clear explanation! Thanks!

[@HaykTarkhanyan]

great video, thank you!

[@blakete]

Thank you. You should have more subscribers.

[@shchen16]

Thanks for this video

[@yassine20909]

Nice explanation, thank you.

[@kevon217]

Simple and helpful!

[@dirtyharry7280]

This is so good, thx so much

[@mikejason3822]

Great video!

[@lebronjames193]

really superb video, you should record more !

[@shahulrahman2516]

Thank you

[@yegounkim1840]

You the best!

[@sushilkhadka8069]

This is so neat.

[@Darkev77]

Brilliant and simple! Could you make a video about soft/smooth labels instead of hard ones and how that makes it better (math behind it)?

[@SA-by2xg]

Intuitively, information is lost whenever discretizing a continuous variable. Said another way, the difference between a class probability of 0.51 and 0.99 is very different. Downstream, soft targets allow for more precise gradient updates.

[@omkarghadge8432]

Great! keep it up.

[@mixuaquela123]

Might be a stupid question but where do we get the "true" class distribution?

[@patrickadu-amankwah1660]

Real world data bro, from annotated samples.

[@飛鴻-q1c]

Human is the criteria for everything，so called AI

[@AnonymousIguana]

In the classification task, the true distribution has the value of 1 for the correct class and value of 0 for the other classes. So that's it, that's the true distribution. And we know it, if the data is labelled correctly. The distribution in classification task is called probability mass function btw

[@kutilkol]

superb!

[@starriet]

essence, short, great.

[@jakobmiesner3995]

Thanks

[@MrPejotah]

Great video, but only really clear if you know what the KL divergence is. I'd hammer that point to the viewer.

[@jiwoni523]

make more videos please , you are awesome

[@PoojaKumawat-z7i]

How does the use of soft label distributions, instead of one-hot encoding hard labels, impact the choice of loss function in training models? Specifically, can cross-entropy loss still be effectively utilized, or should Kullback-Leibler (KL) divergence be preferred?

[@vandana2410]

Thanks for the great video. 1 question though. What happens if we swap the true and predicted probabilities in the formula?

[@thinkbigwithai]

At 3:25 why don't we model it as argmax Summ P* logP (without minus sign)?

[@fVNzO]

I skipped through the video but i don't think you managed to explain how the formula itself deals with the infinites that are created when inputting log(0). That's what i don't understand.

[@sukursukur3617]

Why dont we use just mean of (p-q)^2 instead of p*log(p/q) to understand dissimilarity of pdfs?

[@quantumjun]

will the thing in 4:12 be negative if you use information entropy or KL divergence? are they both > 0?

[@yassine20909]

As explained in the video the KL divergence is a measure of "distance", so it has to be >0. There other prerequisites for a function to be a measure of distance like symmetry, and couple other things i forget about.

[@madarahuchiha1133]

what is true class distribution?

[@elenagolovach384]

the frequency of occurrence of a particular class depends on the characteristics of the objects

[@genkidama7385]

distirbution

[@phafid]

3:24 .The aha moment when you realize whta's the purpose of the negative sign in cross entrophy

[@phafid]

4:24. do you know how golden the statement is

[@tanvirtanvir6435]

0:08 3:30 P* is true prob

[@zhaobryan4441]

hello, handsome could you share the clear slides?

[@zingg7203]

Volumn is low

[@ajitzote6103]

not really a great explaination, so many terms were thrown in. that's not a good way to explain something.

[@commonsense126]

Speak slower please

[@Oliver-2103]

Your name is commonsense and you still don't use your common sense lol In every KZbin application, there is the option to slow a video down to 75%, 50% or even 25% speed. If you have trouble with understanding his language, you should just select the 0.75 speed option.

[@commonsense126]

@@Oliver-2103 Visually Impaired people have problems seeing some of the adjustments one can make on a phone even when they know that they exist.