p-values: What they are and how to interpret them

This StatQuest is all about interpreting p-values. You've seen them online or in publications, or heard about them, whispered in dark, rave filled dance clubs, but you've never understood what they were all about. This 'Quest is here to explain everything you wanted to know about how to interpret p-values. Bam!!!
To learn how the p-values were calculated in this video, see...
• How to calculate p-values
...and...
• Fisher's Exact Test an...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZbin Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
0:17 Conceptual motivation for p-values
4:32 p-value defined
7:07 Different thresholds for a statistically significant p-value
9:16 p-values and hypothesis testing
9:47 p-values do not measure effect size
#statquest #pvalue

[@statquest]

Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@dezj9427]

After watching this video, I bought the book! Thanks!!

[@statquest]

@@dezj9427 Hooray! Thank you for your support!

[@hamman_samuel]

In an ideal world, all teachers should be this clear, intuitive, and fun!

[@statquest]

Thank you! :)

[@dengzhonghan5125]

I would say in an unreal word.

[@seriouscoder1727]

It,s null hypotesis

[@ck-jl6no]

in malaysia we dun understand what bam means... lol

[@chenzhiyao834]

I really like the vedio produced by Statquest. Much clearer explanationation than my lecturer

[@lindasegerious9248]

I'm sorry, but, for some of us, that the ice-cream truck arrives on time is extremely important.

[@statquest]

You made me laugh! :)

[@kanundash]

@@statquest BAM!!!

[@jtbauer3789]

@@statquest Hi Josh, great stuff, though I am still a bit confused. :) So that I can get the fact straight ... Question 1: "The threshold of 0.05 means if A and B are the same, there is only 5% of the tests will exhibit p-values that are less than 0.05". This statement is then equivalent to "If A and B are different, 95% of the tests will exhibit p-values that are great or equal than 0.05". If so, how can you come to a conclusion in the video at 8:48 that A and B are different just because they have ONE p-value which is less than 0.05? Question 2: When you say if some one wants to be very strict, he can set the threshold to be 0.0001. In this case, what will the corresponding p-value be, also 0.0001?

[@hemersontacon3168]

​@@jtbauer3789 Let me put this in a different way. Suppose that you are the guy trying to develop this new drug A. To be 100% sure that your drug really works and that isn't just luck or placebo that are curing people you would need to test this new drug in every person in the world, or even in the universe. For obvious reasons, this is not possible. To still be able to prove that your drug is curing people you can run this statistical test with a portion of people. We can say that this portion of people is a sample of your universe of people. For this statistical test, you need to establish a hypothesis that nullifies what you are actually trying to prove (this is the tricky part). In this case, this null hypothesis could be that your drug A is no different from drug B that we previously know that doesn't work. The origin of this knowledge about drug B it's not relevant for this test, you can just assume this previous knowledge. You also need to choose a threshold. Now comes the practical part of the test as explained in the video. You create two groups of people from your sample of people and each group is treated with only one of the drugs. You count, for each group, how many people got cured and how many were not cured. With these values, you calculate the p-value. The calculation of p-value needs an apart explanation. You just need to know that the p-value is the probability that the null hypothesis is true. Then if you get a p-value under the chosen threshold, this means a very low probability, it's very unlikely that the values that you observed in this one experiment happened only by luck (placebo or whatever). In conclusion, you can deny the null hypothesis and you can be pretty confident that your drug A does work.

[@littleKingSolomon]

@@jtbauer3789 the equivalent statement should be that: if A and B are different, 95% of the the tests will exhibit p-values that are less than 0.05 since it will be more likely that the test will report significant differences thus producing smaller p-values most of the time. or if A and B are different, 5% of the tests will exhibit p-values that are greater or equal to 0.05(only 5% of the test should suggest that A and B are the same)

[@jaggyjut]

It’s 12:25 am and finally understood what is p-value. BAM!!!

[@statquest]

Bam! :)

[@PritishMishra]

You can't imagine how much *EASY you are making* these confusing things or other people who are making this stuffs confusing !!! Love you man...

[@statquest]

Thank you!

[@emilylee2008]

I was struggling in my stats class lately but your videos SAVED me, thank you so much. I'll definitely be watching more of your videos. I wish my prof was as incredible as you!

[@statquest]

Wow! Thank you! :)

[@bhagyashreehote479]

Cannot imagine a better and simpler explanation to p-values. Amazing! Loving all the contents of StatQuest. Feels like finding the key to the Real Treasure of stats concepts!! Hurray!!

[@statquest]

Thank you very much!

[@firdawsnesrinemahboubi4519]

I'm astonished ... you explain the intuition behind this in 11 minutes while my teacher spent 3 hours trying to explain this ! Glad my sis recommended you ! ^^

[@statquest]

Glad it was helpful!

[@Moiez101]

my professors spent hours and multiple slides trying to explain P-values and still did like 20% as good of a job as statquest.

[@nicw1387]

omg I love how calm your voice is, I can actually feel relaxed trying to learn stats - thank you!

[@statquest]

Thanks!

[@phoenixfire9313]

You explain things so clearly, these videos are a gift to us all, especially to those like me studying for a statistics exam! Many thanks from Canada

[@statquest]

Good luck! Let me know how the exam goes.

[@EbonyPope]

@@statquest If the p-value doesn't decide effect size what does it then? And what does determine effect size?

[@fabiof.deaquino4731]

The StatQuest must go on! Thanks a lot, Josh. Stay strong and safe!

[@statquest]

Yes!!! I hope you stay strong and safe as well. :)

[@yashlakdawala7232]

Finally I don't dread this question in an interview. Godspeed to you!

[@statquest]

BAM! :)

[@mykhailozadorozhnyi4983]

Thank god, you saved me. It was explained so perfect! May my gratitudes reach you wherever you are

[@statquest]

Thank you very much! :)

[@ajaybhat1161]

These videos are concise & easier to grasp than the 2hr Stats lectures at college. Thanks, Josh & Stats Quest Team!!

[@statquest]

Thanks!

[@stephenpuryear]

Josh, this is really good! It covers that main points and helps us all focus on them. As a calibration person, I always find it interesting when I find situations in which we assume the perfection of the accuracy of our input data. This is not at all a criticism of your excellent effort, because if you tried to cover all considerations, you would never publish anything. This is what sank Charles Babbage. I appreciate how you pointed out the assumptions that are sometimes hidden in "hypothesis test". This is why juries find some people "not guilty" instead of "innocent". There is an important sense in which those two things are not equivalent to each other. Thank you again!

[@statquest]

Thanks! :)

[@SHASHANKKASHYAP08]

I really like this sentence, "This is why juries find some people "not guilty" instead of "innocent".

[@darceysinclair8929]

this channel saved me from joining the marine corps

[@statquest]

bam! :)

[@QuizzQuest08]

among all the videos I watched regarding pvalues, I'd say this is the most easy to understand. good job!

[@statquest]

Awesome, thank you!

[@meax4k216]

I would give this video 10/10 p values

[@statquest]

Thanks!

[@SHAMSHERSINGH-iu6pd]

I wish I had a teacher like you in my college days. You are out of the world. Really really love your work. Have a doubt in stats don’t worry!! We have Sir Josh Starmer.

[@statquest]

I'm really glad you like my videos! :)

[@drashtipatel8938]

You are absolutely awesome!! I watched a bunch of videos but yours was the only one where i got to understand the actual concept..Thanxxx a lot! You’re such a saviour.🤩

[@statquest]

I'm glad the videos are helpful! :)

[@giraldiego]

If my Null Hypothesis is I will understand the concepts explained for you, then I get a p-value of 1 every single time. Best teacher ever, period!

[@statquest]

BAM!!!

[@statquest]

@@JohnYoga The p-value is related to the assumption that the null hypothesis is correct. If it is correct, then our data should have a relatively high probability, and thus, we get a relatively high p-value.

[@sidagarwal43]

Amazing explanation Josh Sir, I have been watching the stats playlist during this quarantine. Thanks for the amazing content. Stay Safe

[@statquest]

Thank you very much! :)

[@awanishkumar6308]

Y

[@floriaamalan9029]

Best thing I've ever watched! Thanks for being so clear and giving examples.

[@statquest]

Thank you! :)

[@m.candace4967]

My brain has been so enlightened, thus I feel so much better about p-values as well :D How many times did I read about this trying to understand it?! But this video concisely elucidated the concept so well, I'll return here if I need a refresh. Thank you Josh and Statquest! Double Bam!!

[@statquest]

Thank you!

[@iamtrash288]

I'm watching this deep into the night and the consious part of my brain is half-foot in the dreamland, quarter-foot in the limbo and only about the rest in the real world. This kind of explanation managed to get me to get this faster than it lulled me into sleep, so have my like good sir

[@statquest]

Thanks!

[@alinyrodrigues97]

Thank youuu! You deserve a Nobel prize for providing such useful content for free. This helped me a lot. Now you have one more subscriber ❤

[@statquest]

Thank you very much! :)

[@caclso4909]

yooo wtf u look like alexandra botez

[@alinyrodrigues97]

@@caclso4909 oh, I wish! She's gorgeous

[@diencai1812]

You are a legend! Thank you for your clear explanation with some fun along the way.

[@statquest]

Awesome! :)

[@Dr.HusseinEid]

Omg. Thank you very much i read alot of articles talking about p value but i was confused but now after watching your video i mastered p value😊 i am abdy kany from somalia

[@statquest]

Awesome! And good luck with your studies! :)

[@beingnothing34]

Amazing explanation! The excitement your explanation generates, blurs the difference between an entertainment video and an education one!! :)

[@statquest]

Thanks! :)

[@davikaacademy4264]

This is the reason I have started my channel here to help others understand mathematics like you. This is very good approach in this field ,DAVIKA Academy which is my channel am working on both pure, applied and statistical mathematics. Once more keep it up let us transform the educational sector .

[@statquest]

Noted

[@sakurasuzuki9575]

I finally got the concept of p-value and I’ve heard and read so many explanations, still left confused. Thank you so much, you’re making the world a better place❤️

[@statquest]

Glad it was helpful!

[@EbonyPope]

If the p-value doesn't decide effect size what does it then? And what does determine effect size?

[@EbonyPope]

If the p-value doesn't decide effect size what does it then? And what does determine effect size?@@statquest

[@statquest]

@@EbonyPope It depends on what you are measuring. For example, if you are measuring the height between two groups of people, the effect size could be the difference in the average heights.

[@EbonyPope]

@@statquest Ok but what does the p value measure then? The likelihood of the null hypothesis being true? I understand that I reject it when it's equal or less my significance level. But apart from that? How do I interpret a very low or very high p-value? What does it tell me if for example p is 000000.1?

[@mohammadrezaghiasy6618]

Hey Mr Josh I can't Imagine the statistical World without you! THANK YOU ❤️

[@statquest]

Thank you!

[@rahulkumar-go7mz]

You're teaching is very unique and easy to understand. Keep up the good work.

[@statquest]

Thank you very much! :)

[@pavitrasubramaniyan7270]

thank you so much for making this A1 content to help us students out, we are so indebted and grateful to you.. :"(

[@statquest]

Thank you! :)

[@mardinmardin7108]

So intuitive and well explained!! Great job!

[@statquest]

Thank you!

[@SebastianCrain]

This is the 4th video about p values I've watched and you finally make it make sense, thank you

[@statquest]

Thanks!

[@raghavgaur8901]

Hi Josh,I really appreciate your efforts in making these videos

[@statquest]

Thank you! :)

[@rohanshirude5117]

now I understand the P value, don't know its just hyped terminology! Thanks man , the video is great.

[@statquest]

Happy to help!

[@pkmath12345]

Sweet. P-value is one of what students are confused about the most in the study of statistics. With normal distribution, p-value is both theoretical and practical. This would elaborate this point as well. Nicely done!

[@statquest]

Thanks! :)

[@robsonsilvadasilva]

This channel is being really useful for me. Learning a lot. Thanks from Brazil. Take Care!

[@statquest]

Thank you very much! And thank you for supporting StatQuest. :)

[@webtechbysuraj5942]

His "hello" is like waking me up from a deep sleep....that's awesome!!💌

[@statquest]

Bam! :)

[@aakashyadav2117]

Thanks a lot for this amazing video, I knew what p values meant but you gave me a whole knew way to think about it. A lot of statisticians are against using p values. I have tried to understand the reasons, but haven't made much progress. If possible please make a video of it or write it up on a blog.

[@nmertsch8725]

The problem with p values is that they are easy to manipulate and you need more information about an experiment than just the p value to assess if the results show anything. Let's say I have 100 drugs, test if they cure a diseases and say they are a good drug if the respective test resulted in a p value smaller than 0.01. Now even if no drug has any value we expect one p value smaller than 0.01, because we did so many tests. This happens a lot and instead a much smaller threshold should be used (e.g. by Bonferroni correction). What I read in a medical paper lately was that they tested I two groups of size 3 were different and got "p

[@statquest]

A week from today I'll release a StatQuest on something called "p-hacking" that explains how p-values are abused. Then I'll release a video on power calculations that shows how to fix the problem.

[@aakashyadav2117]

@@nmertsch8725 So here chances of getting p less than .01 is because we are comparing Large (100) no treatments. If we would be comparing just 2 treatments one control and new one than using p values would be fine?? Apologies for such a late reply.

[@intuitivecontrolsystem4436]

These series of lectures are a gift to humanity.

[@statquest]

Thank you!

[@aashishmalhotra]

Thanks for being such a nice teacher. You made teaching beautiful and fun!

[@statquest]

Wow! thank you!

[@ruxiz2007]

I love the songs you sing at the beginning of the lectures each time.

[@statquest]

Thank you very much! :)

[@Ron-yi3xu]

Josh Starmer's Stat Quest is excellent.

[@statquest]

Thank you very much! :)

[@manalin6917]

Way better than my university tutor explained, thanks you! Big help! Subscribed!

[@statquest]

Wow, thanks!

[@aaronrodriguez5454]

I really needed the, "if johnny has 3 apples" level of explanation

[@statquest]

bam! :)

[@DaisyDorothy]

this helped, good explanation!.

[@statquest]

Thanks! :)

[@mostinho7]

Thanks for great intuition video. Hope you do more mathematically rigorous videos also ❤️ 9:00 interpretation of large or tiny p value. A small p value doesn’t necessarily mean that there’s a big difference between the drugs, but it means that there is a statistically significant difference (even if that difference is small)

[@statquest]

Thanks!

[@nestorrabanal4451]

@@statquest Thanks for all your videos! I got one question, how is it possible that drug A and B are different if they have 35% and 34% cured people respectively?

[@statquest]

@@nestorrabanal4451 The larger the sample size, the more power we have to detect small differences between groups (or drugs, in this case). For more information about "power", see: kzbin.info/www/bejne/iKTGZq2krLdofKM

[@nestorrabanal4451]

@@statquest Thanks for your answer Josh!

[@staynomadic5828]

Loved it... was struggling to understand how the p value actually determine the differences in effect. This video explained it all. Thanks a lot StatQuest

[@statquest]

Glad it was helpful!

[@collingillespie2725]

I love the jingles. They are short and delightfully lacking in enthusiasm. Thanks for the great videos!

[@statquest]

:)

[@Sidbaazi1863]

Though I am not a biology guy, but by watching your videos, I guess I can learn statistics and biology at same time. Thanks a lot Josh.

[@statquest]

Thanks! :)

[@borisdjakovic]

Daddy, what are p-values? Sit down son, and let me tell you the story about drugs

[@statquest]

:)

[@brindhaganesan3580]

Thank you so much Sir for this crystal clear explanation. Fun and interesting watch. Huge respect.

[@statquest]

Thank you! :)

[@Veigaburame]

I spent like 2 days read about NULL HYPOTHESIS and when I came here you just said that to prove whether the drugs are the same or not.... Sir you are intelligent indeed!

[@statquest]

Thank you! :) I also have another video that is all about the null hypothesis: kzbin.info/www/bejne/ZqDGZWx6rqZmnrc

[@hamade7997]

Josh, I am not sure if I am going to pass my exam or not. But if I do, it is thanks to you. You are a king among kings when it comes to clear, meaningful instructions about statistics. Thank you so, so much for the time and effort you put into these videos. It makes studying bearable. Thank you, thank you, thank you.

[@statquest]

Good luck with your exam! :)

[@hamade7997]

@@statquest Thank you! I just finished. I'll let you know how it went (:

[@KyurinDiary]

BAMMMMM i got it now! thank you so much!

[@statquest]

Hooray! :)

[@bunkertons]

Dude, you're getting me through STATS right now! Thank you! 🥰

[@statquest]

Happy to help!

[@Alchemist_171]

Hurray! I'm on the right quest now. Thank you, sir!

[@statquest]

BAM! :)

[@pradeepkumar-ew1ze]

I reject the null hypothesis that "there is a statquest video that don't start with a awesome song" :)

[@statquest]

BAM!!! :)

[@ashutoshkumarjha41]

Thanks, for such creative interpretation of p value. Sometimes it appears to me that i have to cram a lot of formula in stats and that is quite irking for me.But your video is simple, adorable and short in which you have used set induction and then illustrated through various distinct examples. Lovely!

[@statquest]

Glad it was helpful!

[@chetankumar9463]

Bhai mujhe bhi samjha do. Mere kuch palle nhi pada

[@mehak8392]

I AM SO GRATEFUL I FOUND THIS!!!

[@statquest]

Hooray! :)

[@priyadharshinig4527]

The explanation was very easy! It really helped me a lot. Thanks a lott!

[@statquest]

Hooray! :)

[@jaskiratl.4434]

It's been over a year and there are still comments being hearted and getting replies lol, I love the commitment.

[@statquest]

BAM! :)

[@fengshandian]

Thank you so much! I'm a bit confused about why the closer a p-value is to 0, the more confidence we have that Drug A and B are different as if the threshold is 0.05 which means if A and B are same and if we did this exact same experiment a bunch of times, then only 5% of those experiments would result in the wrong decision. Then if the p-value is 0.03. It means only 3% of those experiments would result in the wrong decision. Why it doesn't mean it is more likely that A and B are same?

[@statquest]

The p-value tells us how different the observed data are from what we would expect if the data were the result of random chance. The smaller the p-value, the less likely that the data were the result of random chance.

[@statquest]

@OKUBO SELESTINE OPIYO It's not exactly "the the p-value, meaning there was less random chance influencing the results", it is more that "if it were random chance, then the event we saw would be very rare".

[@msaparajita849]

The last point about p values and the difference between the two samples is seriously 🤯 BAM! 👍🏽👍🏽

[@statquest]

Thank you! :)

[@digigoliath]

Hooray!!! I made it to the end! And now I get the idea behind p-values.

[@statquest]

BAM! :)

[@physics19941994]

Sir, I will do a case study on your teaching methodology

[@statquest]

BAM! :)

[@EkaluxUaareemitr]

Is it just me who laughs every time he says "small p"-value?

[@statquest]

:)

[@tayloranne6286]

StatQuest with Josh Starmer as I’m In

[@paulafromlodz]

Amazing explanation and great voice! It is a pleasure to listen to You and all examples are funny. Live the pictures and examples. Thank you!

[@statquest]

Thank you very much!

[@mehmetkazanc5855]

I graduated with your videos. I can say that the best and most clear explanation about my interested topics :)

[@statquest]

BAM! Thank you!

[@Leon_Binswanger]

Amazing Video, solved a lot of confusion for me! Thank you very much.

[@statquest]

Glad it helped!

[@guillermorosales3978]

Excellent explanation, simple, summarized and clear

[@statquest]

Glad you liked it!

[@Lehmann108]

One of the best explanations of p values!

[@statquest]

Glad you think so!

[@LucasSantos-yb2qo]

You made me sign on the first "hooray and bummer", but at the end of the video I realized u made a good material and worth the following, nice job XD

[@statquest]

Thanks!

[@rocky90091]

Amazing explanation! Please keep them coming 🙏

[@statquest]

Thank you! Will do!

[@Aya_riiia]

This is like dora the explorer for students like me 😭😭❤️ You saved me a lot of times bro!

[@statquest]

Thanks!

[@trishamaefrancisco6032]

Your "Horrayyy!" Makes me smile while learning from your videos.😆

[@statquest]

I'm so glad!

[@arvindgond4149]

Very clear cut and nice way u explained it 😍😍 Thank you sir 🙏🙏

[@statquest]

Thank you! :)

[@duygiangnguyen7942]

A really great explanation. Thanks so much!

[@statquest]

Thank you! :)

[@maksimgrinkov5857]

Man, you're awesome. I wish there were more content like this

[@statquest]

Thank you!

[@qualitytoolbox4872]

In simple terms. I would say that “p values has the ability to identify rare events”. But before that you have to decide what a rare event would be. However. Explanation provided by Sir is awesome.

[@mountainscott5274]

emphasis on the "deciding before" part, as this is often overlooked in understanding p-values. 👍🏾 repository.upenn.edu/statistics_papers/540/

[@statquest]

Thanks! :)

[@chuongnguyen6091]

thank you for the good work! you made things simple and fun!

[@statquest]

Thanks!

[@univbae]

Wow I heard exactly what I needed to know! Thank you man !! Horraaaay

[@statquest]

bam! :)

[@menoima9501]

bam bam bam aaannnd bam. This literally has revived my life. I was gonna be dead by imagining how difficult p-values were

[@statquest]

bam!

[@rafaelv.valadares2721]

Bam!!! Thanks! This video is going to help me in a presentation at work!

[@statquest]

Good luck! :)

[@zhanbolatmagzumov6409]

it is the best explanation I`ve ever seen. Thanks a lot!

[@statquest]

Glad you think so!

[@nick_g]

These videos are excellent. Thank you!

[@statquest]

You're very welcome!

[@rishabhsingh8416]

thank you very much Josh, I was very confused with this topic

[@statquest]

Glad it was helpful!

[@shekharsachdev7710]

Thank yu so much sir for such clear explanations. You rock. Hurraaayy 🎉

[@statquest]

Thank you very much! :)

[@reyhandwiprasetyo1449]

The best explaining video i ever watched.

[@statquest]

Thank you!

[@youwin6580]

want lo learn about data science and so many people suggest me to start from here, and finally i know why, absolutely great video

[@statquest]

Thank you!!!

[@neelshah8908]

This is really fun way to learn! BIG THANKS

[@statquest]

Thank you!

[@santuhazra1]

Hi josh big fan of your work.. All these videos helped me a lot in my career.. Just one thing which is not yet clear to me that is for a logistic regression classification or Regression problem we see p values for each independent variable.. Just want to learn exactly how it is calculated.. Hope someday you make video about this maybe with a small example.. All d best fr upcoming series.. Thanks a lot man..

[@statquest]

I explain this in my series on linear models. Specifically, these videos: kzbin.info/www/bejne/pJyVdIR_idKSm9E kzbin.info/www/bejne/sHq3enmKqM6phJo and kzbin.info/www/bejne/nqDOcn-aftinbs0 In short, we calculate the F-statistic for the model with and without each parameter.

[@alecvan7143]

Love those new slide transitions

[@statquest]

Thanks! ;)

[@smtxtv]

I should be studying for my upcoming statistics exam...but instead I'm up late at night watching another episode of StatQuest ! 😬😬🤔🤔😎😎😎

[@statquest]

bam!

[@giuliobrunetti8402]

That's actually a great video! Well done!!

[@statquest]

Thank you very much!

[@tymothylim6550]

Wow really interesting video! Did not know that p-value doesn't indicate the size of difference until you mentioned it.

[@statquest]

Glad it was helpful!

[@andresvodopivec5950]

Hi Josh. Your videos are the best. Really the best! Can you please do a video on partial least square and ordinary partial least square? Also, as second, a video on Hotelling T2? thanks