Cosine Similarity, Clearly Explained!!!

[@statquest]

To learn more about Lightning: lightning.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@marchanselthomas]

The explanation is so clean. I was clapping for him from my room. How can someone be so good at their job!

[@statquest]

Thank you! :)

[@nuridaw9586]

I clapped too, twice! :)

[@jwilliams8210]

You are EXCEPTIONALLY good at CLEARLY describing complex topics!!! Thank you!

[@statquest]

Thank you very much! :)

[@mattgenaro]

Such a simple, yet, a beautiful and powerful concept of similarity. Thanks, StatQuest!

[@statquest]

bam!

[@nossonweissman]

You literally make it so easy!! I can't help but smile 😊😊😊❤️❤️❤️ By far one of my favorite KZbin channels!

[@statquest]

Thank you so much! :)

[@torley]

QUADRUPLE BAM!!! Thanks for such fun yet pragmatic explainers.

[@statquest]

Thank you!

[@usamsersultanov689]

I think and hope that this video is a preamble for more comlex NLP topics such as Word Embeddings etc.. many thanks for all of your efforts!

[@statquest]

Yes it is! :)

[@xanderortega4359]

Cosine Similarity is used as an evaluation tool on word2vec

[@tysontakayushi8394]

I usually hate when people say that a video explains well, because usually this is not the case. But, haha, amazing job! Well done, really nice explained, it's a gamification, they way I understand!

[@statquest]

Thanks!

[@jasonlough6640]

Dude these are so good. I have to watch them several times, and then I try write some code to reinforce the concept. Your vides are absolutely amazing.

[@statquest]

Thank you!

[@DrKnowsMore]

Like most things, it is relatively straightforward when you remove the jargon

[@statquest]

bam! :)

[@RonnieDenzel]

Never understood something in such a slow but efficient pace,thanks💯

[@statquest]

Thanks!

[@virenpai9395]

My Love for learning Data Science and Statistics has increased multi-folds because of you. Thank you Josh!!🙂

[@statquest]

bam! :)

[@dukeduke1910]

This guy is seriously funny. I thought I was the only person who ever watched gymkata (like 50 times, especially the part in the town where everyone was crazy). This video def explains cosine sim clearly. Thk u!

[@statquest]

BAM! :)

[@MOROCCANFREEMIND]

The quality of your explanation is more than triple bam!!😂

[@statquest]

Thanks!

[@KarthikNaga329]

This is another great video, Josh! question: @3:51 you talk about having 3 Hellos and that still results in a 45 degree angle with Hello World. However, comparing Hello to Hello World seems to be a diff angle from comparing Hello to Hello World World. Is there an intuition as to why this is the case? That is adding as many Hellos to Hello keeps the angle the same, but adding more Worlds to Hello World seems to change the Cosine Similarity.

[@statquest]

Two answers: 1) Just plots the points on a 2-dimensional graph for the two pairs of phrases and you'll see that the angles are different. 2) The key difference is that "hello hello hello" only contains the word "hello". If we had included "world", then the angles would be different. Again, you can plot the points to see the differences.

[@insaiyancvk]

Wonderfully explained, Josh! You've earned a subscriber!

[@statquest]

Thank you!

[@gunnerstone120]

Great video. It would've been worth noting that magnitude of the feature space does matter in certain cases and doesn't in others. Your example of [Hello Hello Hello], caught my eye. In that example, the magnitude of that feature didn't matter because its direction didn't change. However the difference between [Hello World!] and [Hello Hello Hello World!] does have an impact on the angle.

[@statquest]

Good point!

[@nedlin7934]

Danke!

[@statquest]

TRIPLE BAM!!! Thank you for supporting StatQuest!

[@Neeeemg]

U nailed it enjoyed it hello ,BAM and best teacher ever 😂😂😂

[@statquest]

Thank you!

[@olucasharp]

It all seems so easy when you speak about such complicated things! Huge talent! And so funny ⚡⚡⚡

[@statquest]

Thank you!

[@abdulrafay2420]

What a great way of explaination !! Love it ❤

[@statquest]

Thanks!

[@lynnekilgore6011]

Terrific video, thanks Josh! I learned the basics of Linear Algebra before, but the explanations were never this clear (or fun).

[@statquest]

Thank you!

[@kforay42]

Your videos are such a lifesaver! Could you do one on the difference between PCA and ICA?

[@statquest]

I'll keep that in mind.

[@jitendrakumar-k4d7k]

you are the King Josh 👏👏👏👏 wonderful job!!!

[@statquest]

Thank you! 😃

[@magicfox94]

Excellent explaination! I hope it is the first of a NLP series of videos!

[@statquest]

I hope to do word embeddings soon.

[@AmineBELALIA]

this video needs more views it is awesome

[@statquest]

Thank you! :)

[@zeinebkefeya1946]

The Best!!! thank you for your efforts

[@statquest]

Glad you enjoyed it!

[@Artinm89.2]

you are insane at explaining clearly, btw you sing really well😂

[@statquest]

Thanks! 😃

[@davidmurphy563]

Could you cover discrete cosine/fourier transforms pretty please?* I've love to know how to break signals up into their component frequencies. If you haven't already!

[@statquest]

I'll keep that in mind.

[@Ankara_pharao]

Have you seen 3blue1brown video on this topic? Not sure if it about descreet FT.

[@azzahamed2063]

This is an AMAZING explanation !!

[@statquest]

Thank you!

[@bladongarland8635]

Hilarious, easy to understand, and entertaining. Bravo!

[@statquest]

Glad you enjoyed it!

[@bjornnorenjobb]

Awesome video! I had no idea what Cosine Similarity was, but you explained super clearly

[@statquest]

Thanks!

[@suzhenkang]

pretty good .

[@exoticcoder5365]

I must watch Gymkata ! Thanks for the recommendation ! And excellent explanation of the topic !

[@statquest]

bam! :)

[@chris-graham]

"in contrast, this last sentence is from someone who does not like troll 2" - I was expecting a BOOOO after that lol

[@statquest]

Ha! That would have been great.

[@RaynerGS]

I love you!!!! Salute from Brazil.

[@statquest]

Muito obrigado! :)

[@AreyHawUstad]

Holy shit did I land on a gold mine. Love the explanation (minus the intro, sorry Josh). Thanks a bunch!

[@statquest]

Thanks!

[@willw4096]

Great video! My notes: 3:52 4:23

[@statquest]

bam!

[@tarangrathod255]

You're a very good teacher!

[@statquest]

Thank you! 😃

[@anuj5576]

Super simplistic explanation! Thanks for your effort.

[@statquest]

Thanks!

[@infraia]

Excellent explanation!

[@statquest]

Thanks!

[@Ogunbiyi_Ibrahim]

I came here as I need to learn something in NLP. Thank you, I understood it clearly.

[@statquest]

BAM! :)

[@AU-hs6zw]

You deliver the moment I need it. Thanks

[@statquest]

BAM! :)

[@limebro8833]

This video saved me, I cannot thank you enough.

[@statquest]

Bam! :)

[@muhammadazeemmohsin5666]

what's an amazing explanation. Thanks for the video.

[@statquest]

Thanks!

[@abrahammahanaim3859]

Hey josh thanks for the video nice explanation.

[@statquest]

You bet!

[@artmiss-x8o]

You really are the best !

[@statquest]

Thank you!

[@suaridebbarma1255]

this video was absolutely a BAM!!

[@statquest]

Thanks!

[@kavita8925]

Your Explanation is great

[@statquest]

Thanks!

[@bachdx2812]

thanks a lot. this kind of videos are super helpful for me !!!

[@statquest]

Thanks! :)

[@성이름-g3q]

wow thankyou!!! i don't know how to calculate it , but after watching this, i become mathmatician!!

[@statquest]

bam!

[@theedspage]

Hello! Hello! Hello! Thank you for introducing me to this topic! Subscribed.

[@statquest]

Awesome! Thank you!

[@ymperformance]

Great video and great explanation! Thanks.

[@statquest]

Glad it was helpful!

[@murilopalomosebilla2999]

Hello!! Nice video!

[@statquest]

Thank you!

[@raphaelbonillo2192]

Você democratiza a matemática! Deveriam fazer assim nas escolas.

[@statquest]

Muito obrigado!

[@RichardGreco]

Great video. Very interesting. I hope to see you apply this to more examples.

[@statquest]

We'll see it used in CatBoost for sure.

[@yang9051]

super clear, thank you dude

[@statquest]

Thanks!

[@longgg972]

just beautiful.

[@statquest]

Thank you!

[@Ghulinzer]

Great video! I've seen though in many articles out there that people consider cosine similarity the same as Pearson's correlation since they produce the same outcome when E(X) = E(Y) = 0 and the means of X and Y = 0. This is not true since both measure different things. Cosine similarly measures the cosine of the angle between two vectors in a multi-dimensional space and returns a similarity score as explained in the video, while Pearson's correlation measure the linear relationship between 2 variables.

[@statquest]

Correct!

[@chrisguiney]

This video also does a good job highlighting how cosine and dot products are the same. Unless I'm mistaken, that equation can be written dot(a, b) / (magnitude(a) * magnitude(b)), where magnitude(x) = sqrt(dot(x, x))

[@statquest]

yep

[@dataanalyticswithmichael8931]

superb ! Thank you for the explanation

[@statquest]

Thanks!

[@millennialm1money500]

Great video 🎉

[@statquest]

Thank you 😁!

[@marceloamado6223]

This video was the goat!

[@statquest]

Thanks!

[@smegala3815]

Very useful 👍

[@statquest]

Thank you! :)

[@Francescoct]

Great video! Have you made one for the Word Embeddings?

[@statquest]

Coming soon!

[@AxDhan]

I'm a native spanish speaker, and it surprised me when it started speaking spanish, it will reach more people, but they will miss your motivating silly songs xD

[@statquest]

Thanks! Yeah - I'm not sure what to do about the silly songs. :)

[@pouryajafarzadeh5610]

Cosine similarity is a good method for comparing the embedding vectors, especially for face recognition.

[@statquest]

Nice!

[@gsp_admirador]

nice easy explanation

[@statquest]

Thanks!

[@mystmuffin3600]

Cool! (in StatQuest voice)

[@statquest]

bam! :)

[@ericvaish]

How am I able to understand this topic? Wasn't this supposed to be difficult? 😭 Seriously Great Explanation Josh.

[@statquest]

Thank you!

[@CristianoGarcia10]

Excellent and clear video! I wonder why NLP applications use more often cosine distance rather than other metrics, such as euclidean distance. Is there a clear reason for that? Thanks in advance

[@statquest]

I'm not certain, but one factor might be how easy it is to compute (people often omit the denominator making the calculation even easier) and it might be nice that the cosine similarity is always between 0 and 1 and doesn't need to be normalized.

[@fazelamirvahedi9911]

Thank you for making all of these informative, simple and precise videos. I wondered what happens if two phrases deliver the same meaning but have different orders of words, for instance: A) I like Gymkata. B) I really like Gymkata. In this case doesn't the extra adverb "really" in the second sentence disturb the phrase matrix? And one more question, if the three phrases have the same length and two of them have the same meaning but have used different words, like: A) I like Gymkata. B) I love Gymkata. C) I like volleyball. In this case, would the cosine similarity between A and B be more than A and C?

[@statquest]

In this video, we're simply counting the number of words that are the same in different phrases, however, you can use other metrics to calculate the cosine similarity, and that is often the case. For example, we could calculate "word embeddings" for each word in each phrase and calculate the cosine similarity using the word embedding values and that would allow phrases with similar meanings to have larger similarities. To learn more about word embeddings, see: kzbin.info/www/bejne/rJq9o4Kkf8ifj5I

[@edmiltonpeixeira3221]

Parabéns pelo conteúdo. Excelente explicação, como não encontrei em nenhum outro vídeo

[@statquest]

Muito obrigado! :)

[@sciab3674]

thanks a lot. easy to understand

[@statquest]

Thanks!

[@Sohy365b]

Perfection! BAM

[@statquest]

Thank you!

[@banibratamanna5446]

the generalized equation of cosine similarity comes from the dot product of 2 vectors in multidimension.....by the way big fan of yours❤

[@statquest]

scaled to be between -1 and 1. :)

[@jonathanramos6690]

Amazing!!

[@statquest]

Thanks!

[@gokulsubramanian3875]

BAM!!!!!! SUBSCRIBED FROM CHENNAI

[@statquest]

Thank you!

[@cartulinito]

Great video as we are used to.

[@statquest]

Thank you! :)

[@lifeisbeautifu1]

Thank you!

[@statquest]

Thanks!

[@nidhi_singh9494]

Hey...so cosine is only depends on angle not on lengths... When the case of three Hello were shown, how it can be distinguished between them as similarity is same for both sentence

[@statquest]

What time point, minutes and seconds, are you asking about?

[@debatradas1597]

Thank you so much

[@statquest]

You're most welcome!

[@SalahMusicOfficial]

Hi Josh, I’m trying to understand why cosine similarity may be the best metric to find semantically similar texts (using pertained embeddings). It sounds like the two vectors have to only directionally similar for cosine similarity to be high. What about using something like Euclidean or Manhattan distance. Would a distance metric be better to see if two texts are semantically similar?

[@statquest]

That's a good question and, to be honest, I don't know the answer. I do know, however, that most neural networks - when they use "attention" (like in transformers, which are used for ChatGPT) - just use the numerator of the cosine similarity as the "similarity metric". In other words, they just compute the dot-product. Maybe they do this because it's super fast, and the speed outweighs the benefits of using another, more sophisticated method. Also, it's worth noting that this is a similarity metric and not a distance. In other words, as the value goes up, things are "more similar" (the angle is smaller). In contrast, the Euclidean and Manhattan distances are...distances. That is, as the value goes up, the things are further away and considered "less similar" Lastly, cool music on your channel! You've got a dynamite voice.

[@SalahMusicOfficial]

@@statquest thank you! let me know if you need another voice in any of your intro jingles 😁

[@statquest]

@@SalahMusicOfficial bam!

[@ZOBAER496]

Can you please tell about some applications of cosine similarity like where is it used in which type of problems?

[@statquest]

I talk about that at the start of the video, but you can also use it whenever you want to compare two rows of data. For example, CatBoost uses it compare predicted values for a bunch of data to their actual values.

[@luizcarlosazevedo9558]

Hey, great video as always!! Is the cosine similarity good for regression problems in which the targets are pretty close to zero? Im trying to implement some accuracy metrics for a transformer model

[@statquest]

Hmm... I bet it would work (if you had a row of predictions and a row of known values).

[@jainanshu2000]

Great video ! One question - how is this diffrent from the regular string comparison we use various programming languages?

[@statquest]

I'm not sure I understand your question. My understanding of string comparison in programming languages is that it just compares the bits to make sure they are equal and the result is a boolean True/False type thing.

[@Worldwidenigespam]

The cosine formula shows Ai time Bi. When working through the example, you add them. I know you only get the correct answer when you add them so I thought it had been written incorrectly. However, searching the internet for the formula returns the same as that in the video. What am I missing?.

[@statquest]

What time point, minutes and seconds, in the video are you asking about? It is possible you are missing the "summation" symbol, represented by the upper case Sigma (a greek character).

[@MrJ17J]

Super interesting ! Do you have examples of how those are implemented in practice ?

[@statquest]

I talk about that at the start of the video, but it's also used by CatBoost to compare the predicted values for a bunch of samples to their actual values.

[@Levy957]

you are amazing

[@statquest]

Thanks!

[@Shehab-Codes]

Thank you so much I had no idea what cosine similarity is and you illustrated it easily, appreciate it Btw how cosine similarity can result in -ve number

[@statquest]

The cosine similarity can be calculated for any 2 sets of numbers, and that can result in a negative value.

[@Mrnafuturo]

Does cosine similarity equation ends up being a vector normalization of the projection of one vector over the other one?

[@statquest]

I believe that is correct.

[@samrasoli]

useful, thanks

[@statquest]

Thanks!

[@sushi666]

Can you please do Spherical K Means with Cosine Similarity as the distance metric?

[@statquest]

I'll keep that in mind.

[@notjustanyuser]

How can someone be so good at something! Thank you. I have bought a copy of your book "statquest_illustrated_guide_to_machine_learning" because I wanted to convey my gratitude. I am yet to go through the book (Just bought it!) but I am sure it would be awesome.

[@statquest]

Thank you very much!!! I really appreciate your support! :)

[@notjustanyuser]

@@statquest can you please do an episode on NMF

[@statquest]

@@notjustanyuser I'll keep that in mind, but it will probably be a long time before I can get to it.

[@001kebede]

how can we relate this with correlation between two continuous random variables?

[@statquest]

See: stats.stackexchange.com/questions/235673/is-there-any-relationship-among-cosine-similarity-pearson-correlation-and-z-sc#:~:text=TL%3BDR%20Cosine%20similarity%20is,a%20norm%20of%20%E2%88%9An.&text=To%20convert%20a%20z%2Dscore,function%20for%20a%20Gaussian%20distribution.

[@arvindmathur5556]

Somewhere it says Cosine Similarity is a number between -1 and +1 but in other places it is said to be between 0 & 1. What is the truth?

[@statquest]

The cosine similarity can be between -1 and 1. If all the input data are positive (like they are in a bunch of the examples in this video, since we are just using count data, and count data is positive) then you'll be restricted to values between 0 and 1, but the data don't always have to be positive.

[@shintaardani6332]

I am conducting sentiment analysis research and found that some data has a Cosine Similarity of 0. Are there any methods to make the Cosine Similarity not equal to 0?

[@statquest]

you could pad each phrase with something, so all phrases have at least one thing in common.

[@shintaardani6332]

@@statquest Thank you so much😁

[@eddiesec]

I still don't understand how that works for embeddings though. Each embedding dimension should represent loosely a grammatical property of the words, than how can one word that is farther than another in a single dimension (as in your Hello Hello Hello example) be considered identical?

[@statquest]

I'll do a video on embeddings soon.

[@miltonborges7356]

Amazing

[@statquest]

Thanks!

[@PromitiDasgupta-mz7uc]

can i use cosine similarity for building a similarity matrix between two different brain regions?

[@statquest]

Probably.

[@rajashreechakraborty747]

Can u please help me with this? This is my data: A: cosine: 0.58, z-score: 372 B: cosine: 0.63 , z-score: 370 How can I find the p-value/significance of the 0.5 change in the cosine similarities?

[@statquest]

We didn't cover p-values in the video.