Pearson's Correlation, Clearly Explained!!!

[@statquest]

NOTE: Although I do not mention it by name in the video, this StatQuest covers Pearson's Correlation Coefficient. Unfortunately, this did not occur to me until after I posted the video, otherwise I would have mentioned it at least 20 times...so maybe it's better the way it turned out. ;) Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@sunilkumarsamji8507]

Hi Josh Thanks a lot for the wonderful work. it helps learners a lot. My query: At 9 : 08, it is mentioned p = 2.2 * 10 ^ -16 means low probability that a randomly selected point has similarly strong relationship. Does it mean to say that the hypothesis or prediction (line through the data points) of the trend cannot generalize with respect new data point a randomly selected data point? Is that what a low p means to say? At the same time a low p means high confidence level in the trend which means that high confidence level implies that a randomly selected that will have similarly stronger relationship? Let me please know if I am missing some point.

[@statquest]

@@sunilkumarsamji8507 No. The p-value tells us that the probability that random noise could create the relationship we observed, or a stronger relationship. When you have small p-value, that means the probability that the relationship we observed is due to noise is small. This means we can have confidence that new observations will behave similarly to what we have seen before, rather than completely randomly. Does that make sense?

[@inderjeetsinghintech]

@@statquest yes, that is the reason we keep the threshold to only 5% or 0.05.

[@lorisbach9905]

@@statquest Thanks for your amazing videos. I am watchng them all to try to catch up in statistics for my master degree in geology. In this video, I am unsure on how you calculated the p-values. Can you please explain a little ?

[@statquest]

@@lorisbach9905 Unfortunately, I don't have a video that explains the p-values for Pearson's correlation coefficient in detail. However, I do have a video that explains the p-value for R-squared, which is very, very closely related (and is actually much more useful) here: kzbin.info/www/bejne/pJyVdIR_idKSm9E

[@SumitOli007]

I am crying rn, Statistics was the one thing that scared me in high school, never studied it in engineering & after watching tons of videos & losing hope. I finally found your channel. I am finally understanding bits and bytes of statistics & I owe everything to this beautiful pedagogy Infinite BAM

[@statquest]

Hooray! I'm glad my videos are helpful. :)

[@bigorange6328]

My days spent on statistics before knowing statquest were so wasted

[@statquest]

Wow! I'm glad you like my videos! :)

[@ThinkwithLex]

I can relate

[@kantaphongsaklo102]

Same

[@ahaha731]

Same

[@bartgacrama2295]

You are a genius in pedagogy.

[@statquest]

Thank you! :)

[@anitapallenberg80]

100 % agree! I love StatQuest with Josh Starmer!! ♥

[@vikranttyagiRN]

@@anitapallenberg80 Me too. Alot

[@Cat99]

Simple, easy to understand.

[@alberttamazyan]

I am so thankful to you!!! I tried learning statistics multiple times in my life and never succeded with any source. I discovered your stat quests about a week ago and I already feel so comfortable with many concepts in statistics! Huge thanks.

[@statquest]

That's awesome! I'm glad the videos are helpful. :)

[@mathavraj9662]

As soon as I started the video, the differences between r-square, covariance and correlation were lingering in my mind. Glad you cleared them all!!

[@statquest]

Glad it was helpful!

[@TheCJD89]

I just watched the Covariance and Correlation videos back to back. Very well put together and really easy to follow

[@statquest]

Thank you! :)

[@Alchemist10241]

I've added this channels videos to my Anki cards and every time I review them I get even deeper insights. well done statquest

[@statquest]

bam! :)

[@CamilaMachadodeAraújo]

Ohhh man!!! I'm instantly falling in love with this channel, definitly the best sense of humor to learn machine learning.

[@statquest]

Thanks! :)

[@allthingsconsdrble]

Very much appreciate the crawl, walk, run approach with emphasis on conceptual understanding

[@statquest]

Thank you!

[@bubbleworld4172]

Appreciative bäm from Germany.

[@statquest]

That's awesome! I'm glad Bam has an umlaut in German. ;) That makes it twice as cool. TWICE BÄM!

[@arnabchanda2609]

All My Life I have been looking out for you, glad that I found you... BAM!!!

[@statquest]

Hooray! :)

[@blakebodycote1024]

I have yet to get into most of these concepts in my statistics major, but I am so thankful to have these bite-sized informational videos with lots of visual explanations to explain each concept so I can start practicing and studying machine learning early. Thank you so much for every single video you put out. Truly a blessing.

[@statquest]

Thank you very much! :)

[@2002budokan]

I find the best and non-boring stats explanations in this channel.

[@statquest]

BAM! :)

[@BatkhuuByambajav]

Bam Bam BAM... Eventually, I've fallen in love with your BAMs :) Addictive BAMs and gorgeously simple videos! Thanks a lot!

[@statquest]

Thanks!

[@chocolatemodelsofficial5859]

This is why when drawing trend lines on stock charts they say you need at least 3 points/touches and not 2. Very helpful video!

[@statquest]

Thanks!

[@drditup]

you just explained this better than i ever heard. im a phd student (who for some reason wasn't given a decent statscourse through his master degree in robotics engineering. Needless to say, statistics are good for science)

[@statquest]

Thank you! :)

[@mostinho7]

Thanks! Great summary at 9:00 Correlation strength nothing to do with slope, but with how many points the line goes through. Can have correlation of 1 with large slope or small slope as long as the points lie on a line. 14:00 equation cov(x,y) in previous video

[@statquest]

bam!

[@terryliu3635]

The best intro on correlation, thank you!

[@statquest]

Thank you! :)

[@גיאחנן-ע4ט]

I just learn to my exam in two days with your videos ! You are awesome man keep going ! thank you !

[@statquest]

Best of luck!

[@zumiao234]

Thank you! You actually help me to understand many basic concepts in a clear and easy-acceptable way, you are so smart and kind-hearted.

[@statquest]

Thank you very much! :)

[@shubhamjain2423]

Dear Josh, This video made my endless nights trying to grasp on this topic.

[@statquest]

:)

[@amizan23]

Your videos are way better than most of the paid courses.

[@statquest]

:)

[@vamanieperumal5262]

I cant believe how all your videos are so perfect !

[@statquest]

Wow, thank you!

[@jairoalves8083]

how to obtain the p-value from this data?

[@amanmanveen]

@@minhtoto1542 Had the same question. Found this video helpful: kzbin.info/www/bejne/bnLaZWh-g5qfpKM

[@eugenefrancisco8279]

You might be referring to a t-test for slope. You would need to calculate a sample regression line using the data and then obtain a p value by performing a test on the data with some null hypothesis.

[@vedprakash-bw2ms]

Stat quest is the best ..

[@statquest]

Thanks! :)

[@jammabeergood]

Great rhyme!

[@수삼블-q9n]

Bam....I started to think that statistics can be fun....Huge thanks from Korea

[@statquest]

Hooray!!!

[@shreyastudies4693]

You have a knack for teaching... this was an amazing video, thank you!!

[@statquest]

Thank you! :)

[@pasqualegiorio3651]

Josh you are just a genius of Stat explanations, thank you.

[@statquest]

Thank you very much! :)

[@wernhervonbraun4222]

I'm very grateful to all of your videos. I want to support you but I am a student in 3rd world country. Even I get capable enough I'll surely contribute to this great project! Thank you

[@statquest]

Thank you very much! BAM! :)

[@LOVEONLYLOVEable]

If anyone finds a better teacher than this guy on you tube, do let me know 😎😎

[@taladiv3415]

Guys like this help make the study world a better place!

[@statquest]

Thank you!

[@sherrynsherryn5071]

Thank you from Indonesia, I love your videos!

[@statquest]

Thank you! :)

[@reach2puneeths]

These are the best videos which explains the concept in simple way. Thanks for making these videos. Please upload Al and deep learning videos.

[@Some_random_guy_16]

you are doing a great job enabling us to learn may super tough concepts relatively easy .. that too free of cost...thankss

[@statquest]

Thank you very much! :)

[@thesagecc]

As a graduate level I-O Psychology student.... thank you... I watched the summary first and then went back to watch the entire video

[@statquest]

bam!

[@kylebecker5083]

I've learned so much from this channel. Thanks, Josh.

[@statquest]

Awesome, thank you!

[@calciumfree9626]

Big thanks! I couldn't get any intuition from my school lecture, and it's lucky for me to find this video a day before my exam for this!

[@statquest]

Good luck on your exam! :)

[@kaigordon2900]

Extremely helpful and clear with good examples and explanation! Wonderful, thank you! BAM!!!

[@statquest]

Thanks!

[@OlegGolubev_yolo]

How good you r at this. I tried really hard to understand what it this when i've been in university. but failed. Because there was no explanation why we need this. Only the words that it is "how x related to y"... I figured out what is it actually only 7 years later... Thanks a lot man

[@statquest]

Happy to help! :)

[@YeekyYeeky]

your video makes it really easy to understand(even my english is not really strong , I can still understand almost all of them) , thank you from Thailand

[@statquest]

Hooray! I'm glad you like my videos. :)

[@mohammednadeem662]

StatQeust is really amazing to learn and understand things very easy

[@statquest]

Thanks!

[@hassanrevel]

Josh you're super great man. I really enjoy listening you.

[@statquest]

Thank you!

[@delliscool4924]

thank you , you are distinguished brilliant mind and great teacher for many

[@statquest]

Wow, thank you!

[@draviaartistwithbat5756]

Thanks for the video. And please make next video series on hypothesis testing (z test, t test, anova, chi square)

[@markobe08]

That is right!!!

[@statquest]

If you want to have a super deep understanding on t-tests and ANOVA, you should check out my StatQuest videos on Linear Models: kzbin.info/aero/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU

[@draviaartistwithbat5756]

Sure I will check it and let you know if anything else is needed. Thank you very much. You are doing great man keep up the good work.

[@gtrstreet]

Very well explained. I like that you give lots of examples and answer many of the possible questions in advance. Thanks a lot!

[@statquest]

Thank you very much! :)

[@李广鸣]

It solves my confusion. Thanks a lot.

[@statquest]

Bam! :)

[@sashacollum7583]

Soooo thankful to have found this video. Why did it seem so hard to understand before?!

[@statquest]

bam! :)

[@tuankietly6076]

your video is so great and easy to understand!

[@statquest]

Thanks! :)

[@joserobertopacheco298]

the ultimate clearly explanation

[@statquest]

BAM! :)

[@jihowoo8227]

Thank you very much. You saved my day with (silly) songs and also my day, even my course :))))

[@statquest]

Happy to help!

[@paveldvorak2014]

@Josh, it is great you actually put the text on the screen, I cannot play sound but I can still follow closely what you are saying. Great videos, I hope you will later dive into more advanced topics in time series analysis (unit roots, ARIMA, GARCH, etc). Pls keep it up!

[@statquest]

I'm glad you like my style. :)

[@Bharathkumar-gv4ft]

p-value superbly explained!

[@statquest]

Thanks!

[@TheEbbemonster]

Very good - I would have liked to see a p-value calculation also :)

[@SaintSaint]

kzbin.info/www/bejne/rJbQi6d7gptmfbs kzbin.info/www/bejne/a4ucgHyPdp17m5o Both answer this.... but I agree... a quick explanation of p values would be the only extra credit that I felt was missing from this video. Much the way he did variance recap at the beginning.

[@persephonez7067]

This is much better than the class in uni..

[@statquest]

Thank you! :)

[@caspase888]

Waiting for your videos is a cause worth waiting for 👍👍👍

[@statquest]

Thanks! :)

[@qiliu4100]

I am familiar with the concepts you talk about. But I am a fan of your songs, so I am here to listen to the music.

[@statquest]

BAM! :)

[@abhishek-shrm]

Best video ever seen on correlation👍😁

[@statquest]

Thank you very much! :)

[@abhishek-shrm]

@@statquest Welcome and thank you for making these videos😁

[@sammiechung2711]

Thanks for your detailed and clear explanation. Saving much of my time to read books which hard to understand.

[@statquest]

Thanks! I'm glad the video is helpful.

[@tatianajaramillo1148]

Thank you for your time to explain and make this video!!!

[@statquest]

Thank you very much! I really appreciate your feedback.

[@arismenachekanian1804]

Thank you for making this great video!

[@statquest]

My pleasure!

[@ckarcher4504]

Thank you for your amazing video! Could you explain how to calculate the p-value in this video (such as 12:30). I have watched your p-value, but still do not know how to use it in this video's examples' calculation. 🙏🙏🙏

[@statquest]

Unfortunately I can't explain it in a comment. Hopefully one day I'll make a video.

[@ckarcher4504]

@@statquest Great😊😇🤓 I look forward to it😍😍. thank you very much！🙏🙏

[@AnayFereshetyan]

This was so incredibly helpful, thank you!

[@statquest]

Thanks!

[@harithagayathri7185]

Josh, you explain in such a way that even layman can understand easily. A big shout out to all the hard work you put in for making these videos.👏👏

[@statquest]

Thank you very much!!! :)

[@石政泰]

Hi, Josh. Nice to meet you! I am Tai from Taipei, Taiwan. From the video you mentioned in @7:42, can we say that the probability of a random dot on a random line is equal to the proportion of a line to the 2-D plain, which is the area of a line/area of a plain = 0/1? As we are interested in the probability of a random dot on a random line, it's actually the same as asking the chance of the dot on the line/the chance of the dot on the whole plain. As a line is 1-D, and the plain is 2-D, the proportion is 0. Hence, the probability of a random dot on a random line is equal to 0.

[@statquest]

That might be a way to look at it.I've never thought of it that way.

[@石政泰]

@@statquest Thank you :)

[@taotaotan5671]

BAM.. Get addicted to your video

[@statquest]

Hooray! :)

[@Rustincohle88]

BAM !! You are legend 😭👏

[@statquest]

Thanks!

[@siddharthmishra8233]

always enjoys your song josh!

[@statquest]

Thanks!

[@PRODKAZ-fy8cx]

Thank you so much! This was so helpful.

[@statquest]

Thanks!

[@felipebraga7753]

Cara, seu vídeo é mega claro, sem deixar de ser rigoroso! Super obrigado pelo trabalho!

[@statquest]

Muito obrigado!!!

[@Neuroszima]

Hey nice video! In wikipedia there is also a "non-pearson" corelation, that aims to center data points around the origin, and calculate correlation with the use of covarianve in the form of the dot product with respect to vector norm of data points.

[@statquest]

Thanks for the info!

[@sumitkumar-el3kc]

I love how you teach us like we're bunch of 7-8 year's old kids.

[@statquest]

I just teach the way I teach myself.

[@robertdavis2855]

I didn't think that Machine Learning and humor were correlated but here we are...BAM!

[@statquest]

bam! :)

[@maheshsonawane8737]

Thanks Josh!!!!!!!!!!!!!! Helps lot.

[@statquest]

Thank you! :)

[@maheshsonawane8737]

@@statquest I can't believe u replied. I am pursuing MS Data Science. Your work really give me better understanding. I will pay ur tuition fee when I get job. ✌🤟👆👍😎

[@aapje180]

Bedankt

[@statquest]

TRIPLE BAM!!! Thank you so much for supporting StatQuest!!! :)

[@Clarkephix]

First time hearing a female voice on your channel, and it's hilarious. Anyway, thanks for all of your videos, it helps me survive throughout my statistic course

[@statquest]

Hooray! :)

[@Sanus-anime-world]

Amazing explanation

[@statquest]

Thanks!

[@aswathyskumar6622]

Thankyou for that intro song though🥺♥️

[@statquest]

Hooray! :)

[@vaibhavpandey7398]

Uncle josh, ur only one who answers my query of why can't squiggly line be made. Thanku

[@statquest]

It can be, but it's not as easy (however, modern neural networks can fit a squiggly line to just about anything. For details, see: kzbin.info/www/bejne/sKnEmHqqe56Am7c ). When we use squiggly lines, we use R^2 instead of Pearson's Correlation because Pearson's correlation is explicitly defined for straight lines.

[@vaibhavpandey7398]

@@statquest ok thanku.. It's entirely new for me

[@prateeknagaich13]

Hi, great video. Can you please provide additional guidance on the following: a. How do you quantitatively determine the P-value for a correlation? b. What's the difference, both formulaically and conceptually between R2, Correlation, and Beta/coefficient in a regression?

[@statquest]

For details on p-values and linear regression, see: kzbin.info/www/bejne/pJyVdIR_idKSm9E

[@qinqinkong8330]

Triple Bam!! Thanks for the great lecture, although I think the p-Value not only depend on the amount of data we have, but also depend on the strength of relationship. For example, given the same amount of data, the chance to generate stronger relationship from random points is smaller for higher correlation than lower correlation.

[@statquest]

Yes, that's sometimes true, but not always (for example, if your sample size = 2), so I decided to focus on the things that are always true in my video, and that is Correlation is determined by the strength of the relationship and p-values are determined by sample size. In other words, if the sample size is too small you will never have a small p-value, and if the sample size is huge, then it doesn't matter what the correlation is, the p-value will probably be significant. For example, if we have any 2 data points, we can draw a line through them, and correlation = 1, however, the p-value = 1. In contrast, if we have enough data, it doesn't matter how close the correlation is to 0, we can still have a significant p-value.

[@qinqinkong8330]

@@statquest You reply my comments! Bam!!!!

[@statquest]

@@yangyu5525 Corrected!

[@jaleshjalesh5712]

can u please tell how did you calculate p- value?

[@ALEX-mo1zo]

He did a video on P-Value

[@xruan6582]

Great course. May I point out that at (17:38) it is better to say "correlation quantifies the strength of linear relationships"

[@statquest]

True! :)

[@SevenRavens007]

Still getting this clear in my mind. ..At 13:11 you say that adding data (and a decreased p value) increases our confidence in our guess. I think this may be misleading because it suggests that b smaller p values mean more accurate guesses. I would rather say that smaller p value means more confidence that we are accurately seeing the QUALITY of the guesses we can make (not the guess itself, which is indicated by the correlation value). So with a weak correlation, smaller p value means I am more certain that there is a weak relationship and that my guess will be poor I hope that makes sense. Thanks for a great series

[@statquest]

What I was trying to say was in the picture on the left, we can't be sure if adding more data would give us a totally different correlation value, so we have low confidence in it. In the picture on the right, we have enough data to be confident that the correlation value will not change much with additional data.

[@yangyu5525]

Dear professor, at 12:57 in respect to the picture on the left, you said "increase the sample size ,don't increase the correlation". I have a different opinion about the statement. Because that at starting if I have two dots, so no doubt the correlation of the straight line is equal to 1,and P-value =1.then I add randomly some dots to the graph, well the correlation value will be changed , and so the P-value will do .thus, the P-value just tell us if there is a trend or not ,don't tell you how much the difference and how accurate the trend you find close to the actual of the stuff . Alternatively, the accurateness of trend or model you find depends on not only the amount of dots ,but also the development of technology, right?@@statquest

[@jingjingli101]

Great video! Can you also explain the difference between spearman and pearson corrlelation? Thanks a million!

[@statquest]

I'll keep that in mind.

[@yulinliu850]

Good job Josh! Thanks!

[@statquest]

Thanks! :)

[@usama57926]

amazing explanation....

[@statquest]

bam!

[@1miffy1]

When Phoebe decides to sing stats... xD Love the videos... lifesavers to sinking ships in the sea of numbers

[@statquest]

Check out: kzbin.info/www/bejne/emHIl3t7f9iZftE

[@1miffy1]

Omg xD best!

[@陈曦-h7z]

NICE VIDEO AND EASILY UNDERSHANDING

[@statquest]

Thanks! :)

[@arishali9248]

Love your videos mate

[@statquest]

Thanks! :)

[@kiranisrani7245]

Hi. Your explanation was perfectly fine. I have a doubt at 16:20, shouldn't it be "That means that there is 3% chance that random data could produce a weak relationship, or weaker". or "That means that there is 97% chance that random data could produce a strong relationship, or stronger". Because smaller the p value, stronger the correlation.

[@statquest]

The video is correct. p-values are kind of tricky, and to learn more about how to interpret them, you can check out this video: kzbin.info/www/bejne/rJbQi6d7gptmfbs Also, a small p-value doesn't mean a strong correlation. We could have a weak correlation, like 0.1, and still have a small p-value.

[@yagneshm.bhadiyadra4359]

You are producing vidoes like a mathematical web series

[@statquest]

Thanks!

[@difflyfe23]

Awesome lecture

[@statquest]

Thank you!

[@amanzholdaribay9871]

Thank you, man!

[@statquest]

Thanks! :)

[@TheCraZHI]

I lost it at "small bam" 😂

[@statquest]

:)

[@JupiterChamsae991102]

Awesome video again! But just a question about 15: 07 - 15:13, regarding "When the data all fall on a straight line with a positive or negative slope, then the covariance and the product of the square roots of the variance terms are the same and the division gives us 1 or -1, depending on the slope", I don't think I fully get it intuitively. So how could we know the absolute value of nominator and denominators are the same without calculation?

[@statquest]

Unfortunately the mathematics that show why correlation is limited to a maximum value of 1 and a minimum value of -1 are quite complicated, which is why I glossed over it in the video.

[@JupiterChamsae991102]

@@statquest Thank you so much for your instant reply! Then without calculation, is there a possible way to just understand it intuitively?

[@statquest]

@@JupiterChamsae991102 I did the best I could with this video.

[@JupiterChamsae991102]

@@statquest Ok~ Thank you so much as always ❤️

[@keylazy]

At 8:59, how is the p-value calculated for drawing random dots?

[@statquest]

Unfortunately the formula for pearson's correlation coefficient is pretty complicated. However, the p-value for r-squared, which is related is here: kzbin.info/www/bejne/pJyVdIR_idKSm9E

[@MahdiSafarpour]

Dear Josh Starmer , I am thankful to you for your wonderful videos. May I know why the numerator of correlation formula is always lower than denominator?

[@statquest]

That would take a whole StatQuest to explain. We'd have to go through the Cauchy-Schwarz inequality. However, it's on the to-do list. One day I will do it.

[@MahdiSafarpour]

@@statquest It was a great clue. I start reading about Cauchy-Schwarz inequality and look forward to watching your lecture in future.

[@pritikagowda5513]

CORRELATION ITS THE SENSATION, please make full version songs on stats haha!

[@statquest]

:)

[@evrenkutlu2339]

Thank you very much for every video, you are awesome. 16:26 I think there is a wording mistake; instead, that means that there is 3% chance that random data could "not" produce a smilarly strong relationship or stronger, am I right?

[@statquest]

The wording in the video is correct. For more details on p-values, check out the 'Quest: kzbin.info/www/bejne/a4ucgHyPdp17m5o

[@Fionelko]

you can find the answer for your question in @ali alqaraan comment below.