Statistical Power, Clearly Explained!!!

[@statquest]

NOTE: This StatQuest was brought to you, in part, by a generous donation from TRIPLE BAM!!! members: M. Scola, N. Thomson, X. Liu, J. Lombana, A. Doss, A. Takeh, J. Butt. Thank you!!!! Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@ShivamSaini-xt6rg]

What tools do you use to make your videos?

[@statquest]

@@ShivamSaini-xt6rg I use keynote and final cut pro.

[@ktburger659]

I’m watching this and tears are coming to my eyes. So many classes where I felt so dumb. But I just needed it explained a certain way. I can’t thank you enough for your videos, I’m going to share them with everyone I know doing stats

[@statquest]

Thank you very much! :)

[@appsoftomorrow]

I get it. So true!

[@1947umw]

Double BAM!!!

[@datajunkie3427]

I have just had a StatQuest Marathon today. You are one of my favourite teachers on KZbin for knowledge. Thank you sir!

[@statquest]

Wow, thank you!

[@socratic-programmer]

You are a truly gifted teacher. Thanks a lot for this video! I feel like statistics is that much more approachable thanks to this channel :)

[@statquest]

Wow, thank you!

[@skanvish]

You are an amazing teacher. Here I am preparing for interviews many years after I left school,.. instead of referring to my notes from school , i am watching your videos. A big thank you to you !!

[@marjiehoss1498]

Dang I cant reject the awesomeness of these videos

[@statquest]

BAM! :)

[@saneeshcs6604]

Last week my professor told me that we don't power to explain a result. I wondered the powerful one in the group says we don't have power. That powerlessness took me here. Thanks a lot for explaining power so clearly. She (professor) is powerful again, because she understood it. Thanks again.

[@statquest]

bam!

[@apoorvarawat2235]

I love the way you teach..... But honestly I love your opening! "There's clouds outside... But who cares.... It's time for Stat Quest... STATQUEST... "

[@statquest]

Ha! I'd forgotten about this tune. It's a good one. :)

[@apoorvarawat2235]

@@statquest thank you so much for your reply.... U r too good to be true♥️

[@katherineannnguyen]

bro i could literally watch these videos for fun they're so good

[@statquest]

bam! :)

[@harrymoore6140]

Showing the two distributions is very intuitive and really helped me visualise hypothesis testing. Thank you!

[@statquest]

Bam! :)

[@tomp9857]

Wow, this short video has explained to me what a 2 hour lecture failed to explain! Thankyou so much

[@statquest]

Glad it was helpful!

[@beigpodcast]

You explained the concept in a very simple, explicit, and fun way. Thank you.

[@statquest]

Thank you!

[@del7920]

honestly this is the only stat video i was entertained haha and i actually really enjoyed the video and understood everything!

[@statquest]

Bam!

[@RaeOfSunshine314]

best description of power that i’ve ever seen !

[@statquest]

Thank you!

[@arturleperoke3205]

Hi Josh! Please make a video on why caclulating post-hoc power is bad! Thank you!

[@statquest]

I'll keep that in mind.

[@arimorrison7816]

When the Null Hypothesis says "subscribe to my channel for more stat videos," my small p-value says "I will continue to watch your videos without subscribing!" MEDIUM BAM

[@statquest]

Noted

[@ShubhamSingh-ut3nh]

You are awesome in every awesome way possible The way you incorporate details of sample size in statistics is so great

[@statquest]

Thank you so much 😀

[@nielsthomsen2324]

🤡🤡

[@parttimemusic4629]

Omg ! I should have waited and take stats inference course this sem instead of last semester. This video is awesome ! I finally understand power now! Keep the hard work up Josh!

[@statquest]

Hooray! Thanks!

[@iqbahm]

Awsome video. No one else ever explained me so simply what "Power of a study" is.

[@statquest]

BAM! :)

[@karolineedrich8628]

thank you so much for existing you are literally saving my life

[@statquest]

Hooray! :)

[@maggiehuang5566]

When you are driving in the rain, you can kinda see where you are going(how I felt about statistics before your video), the wind shield wiper is still necessary because it makes everything clearer (that's how your videos are to me). Confession: I was rolling my eyes when the cheesy BAM!!!s came out.

[@statquest]

bam! :)

[@vanilla__latte]

This is the best stats channel on KZbin

[@statquest]

Thank you! :)

[@annawoodard6457]

Thank you for your great videos!

[@statquest]

Thank you so much for your support!!! :)

[@user-vc6wo7cq8v]

Your videos are the best I’ve seen. I’m working through a python for data science course…your explanations are fantastic and the animations make the concepts so easy to understand for applying with python. I can’t thank you enough for sharing!

[@statquest]

Wow, thanks!

[@avam.212]

Oh wow, I didn’t expect the example problem to be the exact one I was looking for!!!

[@statquest]

bam!

[@Justice_King]

Very comprehensive explanation to get the insight even for beginners. At the same time, I also had fun. My heartfelt thanks to you Sir!

[@statquest]

Thanks!

[@gowthammurali1224]

From this video i get to know that increase in power will decrease the type 1 error. May i right Mr.jos.. What a intutive video man...salute for your knowledge sharing....

[@statquest]

Thanks!

[@mostinho7]

3:15, 7:00 for great summary (doesn’t miss anything from the video)

[@statquest]

Yep!

[@farsikogama6114]

Dude, I really want to purchase your book after watching some of your videos!! Great job in explaining

[@statquest]

Awesome, thank you!

[@alecvan7143]

Love the sound effects

[@alecvan7143]

Yet another topic I just figured I'd never get so simply and logically explained. Thank you, Josh.

[@statquest]

Hooray!!! :)

[@suvarnapunalekar4727]

I never knew statistics can be so much fun!!! You are a star... thanks for doing this.. :)

[@statquest]

Thanks so much! :)

[@feidiqingfay]

"Shameless Self Promotion" ... hahahaha ... I can't stop laughing ... so cute ...

[@statquest]

:)

[@infinger2006]

I guess I'll have to watch the next video to see the difference between adding more samples to gat more power and the dreaded P-Hacking... on to the next video!! Thanks for the videos..

[@statquest]

bam! :)

[@dipeshbansal1396]

amazing...each lecture is a treat

[@statquest]

Than you!

[@zinnijha7556]

Thank you !! For simplifying for the rest of us.

[@statquest]

Thanks!

[@giraldiego]

1:33 On this slide, It seems like the reason we correctly reject the Null hypothesis it is because the p-value is less than 0.05, but based on what I have learnt from your previous videos, I think it is because the value was actually very small (0.0004). Am I wrong? Cuz, what we only can say if it is below 0.05, it is that you can reject the Null Hypothesis and you will do it 95% of the time, but you still can be wrong rejecting it. Maybe the word could be, reject with a lot of confident?

[@giraldiego]

Nevermind, I watched the rest of the video and now I understand better. It was just that at that point of the video I didn´t understand, without knowing what was coming next.

[@statquest]

BAM! :)

[@adriancruzat6994]

I hate that I'm finding out about this channel so late. My test is tomorrow and watching your videos would've helped so much. Nonetheless, I'll be binging these videos all night haha

[@statquest]

Good luck on your test! :)

[@adriancruzat6994]

@@statquest thank you!

[@MiguelDimase]

I never saw an author answer every question or acknowledge every comment; it's remarkable (and I'm not saying you HAVE TO do it; it's a huge job. I think it takes more time than making the videos themselves! -which, by the way, are excellent-)

[@statquest]

BAM! :)

[@MrMe77-u4y]

They should make a statue for you in each school to thank you for all the headaches saved to students!

[@statquest]

Ha! :)

[@Quijanos1]

Excellent explanation of Power; this was helpful. Thank you.

[@statquest]

Glad it was helpful!

[@Wagdimg12]

Thanks!

[@statquest]

Wow! Thank you so much for supporting StatQuest! BAM! :)

[@yangwang6805]

Finally understand what is power, thank you for your wonderful video!

[@statquest]

Hooray! :)

[@anna56887]

The varieties of BAMs got me 😂😂😂😂😂 Thanks for this video . Really helped me understand the concept

[@statquest]

Hooray! :)

[@SemicolonExpected]

I fully expected to hear "AND SUBSTITUTE MY OWN" after you said "I REJECT YOUR HYPOTHESIS"

[@statquest]

:)

[@mohammedelhamamsy3275]

Beautiful explanation, Thanks a lot! There's only one thing that I still find confusing that is: if the two distributions highly overlap with each other, what prevents us from thinking that both come from the same distribution? I mean we are looking at the weights of the exact same species (mice) why did we have a priori assumption that the weights of those on the special diet are different from those on a normal diet?

[@statquest]

If we have no reason to think that the mice come from two distributions, then we would not spend the time and money testing the hypothesis to begin with. So, in this case, we must know something about the diet - maybe one is very unhealthy, and the other one is very healthy - and this causes us to suspect that they might be from different distributions, and this then justifies spending the time and money doing the experiment and testing the hypothesis.

[@mohammedelhamamsy3275]

@@statquest Got it, Thanks again ❤

[@tina9753]

♥ what a wonderful video! not even comparable to my dry epidemiology classes...

[@statquest]

Wow, thank you!

[@marcelsa5191]

Thanks for the strong video! Cheers from Ox.

[@statquest]

Thank you!

[@RadicalVideos]

Very helpful and with comedic relief. Thank you so much

[@statquest]

Bam! :)

[@clairefontaa587]

Thank you for the great content again! Do you plan on making a video on Cohen's d, Cochrane's Q and all those meta-analysis merriments one day? :D (desperate PhD student asking)

[@casualcasual1234]

Thanks a lot and I would like to ask: at 4:23, if 2 distributions are the same then there is no need for power so no Power analysis. However, I think we do power analysis to find optimum sample size and then do tests to see whether special and normal diets have 2 different distributions or the same. So may I ask won't this be a contradiction?

[@statquest]

We simply assume that we have two different distributions.

[@casualcasual1234]

@@statquest Oh thanks.

[@noname-go2kt]

Hi,your videos are great,and the opening sequences as well,thanks a lot My question is, what do I do if I want the power but at the same time don't know if my null hypothesis is true or not

[@statquest]

You never know about the null in advance. You just calculate power based on the assumptions that the two things are different and by so much. So, given those assumptions, you can calculate power. We then do the test and, if there is a difference, we should be able to detect it and reject the null.. If not, then we'll just fail to reject the null.

[@noname-go2kt]

@statquest I am sorry I don't understand what do you mean by the two things.For example at 4:25 you have said that the concept of power doesn't apply here since we know beforehand if our null hypothesis is correct or not. However what if I don't know,in short,is there any situation ever where I should not calculate power when doing hypothesis testing? Thanks a lot in advance

[@statquest]

@@noname-go2kt Yes, when we know that there is no difference, then Power does not apply. But we never know - otherwise we wouldn't bother doing the test to begin with. So we assume that there is a difference and carry on from there. In theory you should always do a power analysis if you have reason to believe you need to do a statistical test.

[@aziausa]

I feel I am out of words that will fit in your appreciation. You are simply amazing !!!!! Thank you so much and keep these gems coming, please. Can you please explain what is Tukey HSD analysis and how do we perform that?

[@statquest]

Thanks, and I'll keep that topic in mind.

[@rimkenan4654]

This is so helpful thank you so much! 🙏🏽✨ Greetings from Belgium 🇧🇪☺️

[@statquest]

Thanks and greetings from Spain! (I'm in spain for the next week for work).

[@edwinadrianta9455]

Thank you so much for your amazing explanation. One of the best resources out there

[@statquest]

Thank you! :)

[@karducito22]

Man you are awesome! I hope some day ! will teach like you

[@statquest]

Thank you!

[@muallagun6765]

thisss was very fun and enjoyable to watch, thank you !!!!! :))

[@statquest]

Thanks!

[@jiangxu3895]

Hello there, I have been following your tutorial recently. Great job but far lower sub. for the work you guys have done. Knowledge spread in a intuitive way is invaluable. Thumb up.

[@statquest]

Thank you very much! :)

[@stephennguyen8052]

Hey Josh, thanks for the video! In the event that you truly have one distribution (4:17), what if you sampled data from the 2 tails and do hypothesis testing, isn't it possible to falsely reject the null hypothesis? Why is it that power doesn't apply in that scenario?

[@statquest]

Yes, it is entirely possible to falsely reject the null hypothesis. This is called a "false positive" and it is controlled by the threshold of significance you set for the p-value. So you control false positives with the p-value. In contrast, you control false negatives with power.

[@Happy.Traveller]

How do you calculate the P value of something that isn't due to chance? I watched your video on calculating P values, the example was on coins. Tossing coins is purely by chance if the coin was fair. Since food and drug are not fair, they are rigged, ie, they have a purposeful effect, how do you calculate P value? What is the value for each step? You "How to calculate p-values" video listed 3 steps. What would be the value of step 1 and 2? Would you say "What are the chances of the weight being specifically a value" (say, 80.19 grams) or "What are the chances of the weight being above or below a certain value"? Because if that is the case, it would be 50% since, when there is a clear difference between the 2, one entire group would be more than your set value, and the other will be less, as your graph demonstrates. Ie if 3 out of 6 are above a certain value (normal diet) and 3 out of 6 are under that value, it would be 3/6 which is 50%. Then since the value can be set at anything, if I set it super low or super high so it includes all or neither of the two groups, such as "chances of a mouse being less than 0 grams" or "more than 100kg" then you can say 0 out of 6. How do you calculate the p value then? As always: I hate statistics and statistics make no sense.

[@statquest]

Again, there is almost always variation in the data we collect. Some of that variation is due to things were are interested in, some of that variation is due to things were are not interested in. p-values help us filter out the variation we are not interested in.

[@pabloignaciomancilla1628]

This is fun and truly enjoyable, keep doing it!

[@statquest]

Thank you! :)

[@michellenamuyaba5558]

This is amazing! In just 8 minutes! thank you❤️

[@statquest]

Glad you liked it!!

[@varadgunjal]

Beautiful video! Any chance we can access the slides for quick revision @JoshStarmer ?

[@statquest]

I have PDFs of some of my videos here: statquest.org/studyguides/ and have a book coming out soon.

[@amritadayal1939]

Hi... Me and my husband really likes your videos... Your explaining skills are great and things become simple to understand... Please upload some videos related to Natural Language Programming

[@statquest]

Will try

[@vahidnajafzadeh4137]

dear Josh, do you have a video clip explaining what stat tests we should use for different types of data eg mean, median, portion, rate, etc? thanks

[@statquest]

Not yet.

[@vahidnajafzadeh4137]

@@statquest BAM it 😉

[@CandyLin0401]

Laugh upon hearing there's a shameless self promotion LOL Good job!

[@statquest]

Thank you! :)

[@rkperohindirich6824]

Amazing content! MANY MANY THANKS ❤❤❤❤❤

[@statquest]

Most welcome 😊

[@lizaminasyan9255]

thank you so much for this fun video

[@statquest]

Glad you enjoyed it!

[@giorgiosperandio8708]

Thumb up for the intro

[@statquest]

BAM! :)

[@chicagogirl9862]

I love the song in beginning of videos:)

[@statquest]

Hooray! :)

[@natalieolofssonno25]

How does this have to do with type I and II errors?

[@statquest]

Type 1 errors are false positives, type 2 errors are false negatives. Power and power analyses are important for reducing the number of false negatives.

[@natalieolofssonno25]

@@statquest this helps thank you!!

[@SoggyBagelz]

when you say "power is the probability that we will correctly reject the null hypothesis" - is the alternative of correctly rejecting the null hypothesis ONLY incorrectly rejecting the null hypothesis? or does it include incorrectly NOT rejecting the null hypothesis

[@statquest]

Power assumes that the null is not true and we should reject it, so the only alternative from correctly rejecting it is to not reject it.

[@dr.battulapradeep1183]

The P value by seeing....

[@statquest]

bam! :)

[@MM-qk8eg]

based on this video, I assumed that independent samples t-test should have more power than paired-samples t-test. But apparently, that is incorrect: "When the same participants are used across conditions, the unsystematic variance (often called the error variance) is reduced dramatically, making it easier to detect any systematic variance". could you help me with this? Thanks!

[@statquest]

If your samples are paired, then you should use a paired t-test (and it will have much more power than unpaired), this is because pair samples tend to have some correlation, and that information is helpful in understanding how different things are.

[@ekenechikwelu3790]

thank you so much stat quest, i am currently doing a master thesis on TOST and using Assurance in place of power to calculate sample size, my background is clinical medicine, have little knowledge on stat, but thanks to your video i am doing fine in stat. However pls, i have a request, can you make a video on the concept of Assurance, or refer me to resources to help me on this subject.. Thanks

[@statquest]

I'll keep that in mind, but I can not promise anything soon.

[@haykazmkrtchyan]

Hi Josh, thank you for posting such great content. Question...how does a measurement measurement repeatability error (from Gage R&R study) impact the power of the test as apposed to bias or reproducibility error? i.e. does measurement repeatability error reduce the power of the test?

[@statquest]

I'm not familiar with measurement measurement repeatability error

[@Kalmdownhema]

ok so i am not sure if i'll get an answer here, but if we a low power, does that mean we are more likely to do p-hacking? In the last example we knew that both the observations came from different distributions, but had we not known that, would rejecting the null hypothesis based on a few p values be p hacking

[@Kalmdownhema]

Ah nvm after going through a few of the comments apparently this gets answered in the next video.

[@statquest]

:)

[@powersteh2893]

Is the example here a "two tailed test"? or is this something totally different?

[@statquest]

I pretty much always use two-tailed tests.

[@benwilde1768]

Will you do a series on multilevel models (aka linear mixed models or hierarchical models)?

[@statquest]

It's on the to-do list.

[@TheVideosSalo]

Most fun statistics i've had!! BAMMM

[@statquest]

BAM! :)

[@neuroinformaticafbf5313]

Quick question: knowing that we have two different distributions, and repeating the test until we have a small p-value, isn't some kind of p-hacking?

[@statquest]

What time point, minutes and seconds, are you asking about?

[@neuroinformaticafbf5313]

@@statquest 5:39. I mean, even though you have two "different" distributions, if you run the experiment several times and you have almost always a p-value above the average, shouldn't you assume that the two distributions have no statistical differences?

[@statquest]

@@neuroinformaticafbf5313 The goal here is to illustrate the concept of power. How, when we have low power, only a small percentage of tests will correctly reject the null hypothesis when it is true. When we have a more power, then a larger percentage of tests will correctly reject the null hypothesis when it is true. Also, to be clear, p-hacking refers to a practice that results in incorrectly rejecting the null hypothesis, when the null is actually true. So it doesn't really apply in this context, where the null hypothesis is not true.

[@karthica5251]

Out of curiosity, at 2.13 even though we are knowingly calculating the p-values from samples from 2 distributions, do we still apply the BH method (FDR) discussed in the previous stat quest? When do we use FDR?

[@statquest]

When you know that you have two distributions to begin with, there's no point in even calculating the p-value to begin with. In other words, this is just an example. That being said, in practice, we never know, and if we do multiple tests, then we should correct for that with FDR or some other method.

[@muskanhassanandani1816]

By adding measurements, we increase power But if we are unsure if the measurements are from the same or different distribution, Won't that be considered as p-hacking?

[@statquest]

No, we don't have to be sure that they are from a different distribution, we just assume that they are and we assume that the difference is some value and the variation is some other value. Then we then find the sample size that would give us confidence that, if those assumptions are reasonable, we will reject the null hypothesis.

[@PhiNguyen-wm4kq]

So a low Statistical Power of a p0.05 tell that the result is not reliable too? Thank you.

[@statquest]

Low statistical power means that there is a low probability that you will correctly reject the null hypothesis (if it is true).

[@sayuriyamagata6853]

@@statquest Is it better to use a test with lower statistical power to test the positive effect of something that could be too dangerous if it wasn't that much effective? like a drug that may improve a health condition but has very harmful side effects...

[@JavierSegoviaHernaez-mn8uy]

So goood. Muchas gracias desde España !!

[@statquest]

De nada! :)

[@michaelnguyen7081]

Hi Josh, can you make a video about heteroscedasticity?

[@statquest]

I'll keep that in mind.

[@michaelnguyen7081]

@@statquest thank you!!!

[@pradeepkumar-ew1ze]

Going by these videos and your Linear Regression videos, I wonder how many mice you got as Pets :)

[@statquest]

I used to work in a laboratory that studied mouse genetics.

[@moart87]

How to determine that ‘same distribution’? I’ve always struggled with this concept in the context of A/B testing. Is the baseline distribution the treatment sample, the control sample, or the combination of both?

[@statquest]

Usually it is the control sample.

[@moart87]

StatQuest with Josh Starmer thanks for replying! I work in marketing and use a 10% control sample (to reduce opportunity costs). Should I use the treatment sample as a baseline in that case?

[@statquest]

@@moart87 It doesn't matter because either way you are trying to determine if one is different from the other.

[@dhritimandas1968]

How do we interpret if we do no see statistical significance even after using 9 samples ?

[@statquest]

Small effect size or lots of variation in the distributions. Either way, it's time for a power calculation.

[@sukieyakie]

could you help me with this? i am assigned an experiment with growing of mung beans in seasalt vs sodium chloride (table salt) and my null hypothesis for the experiment is that there is no difference in growth whether it is grown in seasalt or nacl. my alternate hypothesis is that there is a difference. however, before i start the experiment i need to find out the number of beans to use for each test i.e n=? and to find out the optimum sample size to use, my teacher asked me what is the statistical power for this experiment? please help ive been stuck on this for days and cant start my experiment because i still cant justify the amount of beans i should use to test on.

[@statquest]

Umm... Just follow the steps in this video to calculate the sample size.

[@Gengar99]

You're an angel.

[@statquest]

Thanks!

[@sridharbabu9287]

can you please make a vedio on statistical significance and relation with p value

[@statquest]

I've already got one: kzbin.info/www/bejne/rJbQi6d7gptmfbs and kzbin.info/www/bejne/ZqDGZWx6rqZmnrc

[@luisrodrigueziii7316]

YOU ARE THE GREATEST

[@statquest]

Thank you! :)

[@JavierBFV]

Wait wait, how can I calculate te P-value of the mean of examples of two different distributions? I watched the videos of P-value and didn't understand the how.

[@statquest]

You can do this with a t-test. I explain t-tests in a non-standard way, so you have to see two videos: kzbin.info/www/bejne/pJyVdIR_idKSm9E and kzbin.info/www/bejne/hHeYkJWqhMZ2n8k

[@hydrobell]

@@statquest I had trouble following this too, it is kind of glossed over. Thanks for the clarification.

[@haneulkim4902]

Thanks for an amazing video! One question, if I want to prove that data selected from two different group actually comes from the same distribution(opposite of your example) than do I need to set new null hypothesis("data come from different distribution") or does not rejecting null hypothesis multiple times prove that datas come from same distribution?

[@statquest]

Unfortunately you can't turn the hypothesis around. For details, see: kzbin.info/www/bejne/ZqDGZWx6rqZmnrc

[@timmel87]

Just one quick question for my last BAM today: Did I understand correctly that the p-value is the probability for a Type I error and (1 - Power) the probability for a Type II error? Thanks a thousand and good night from Berlin!

[@statquest]

A "Type I" error is a false positive, and a p-value is the probability of getting a false positive (incorrectly rejecting the null hypothesis when it is true). A "Type II" error is the probability of getting a false negative (incorrectly failing to reject the null hypothesis when it is false). Does that make sense?

[@constantinp6488]

@statquest man, you should create courses for Data Camp since they pretty much suck in explaining difficult statistical things :)

[@statquest]

I'll keep that in mind!

[@googledude5649]

Much power in the lol - so good man

[@statquest]

Thanks! :)

[@ShivamSaini-xt6rg]

What tools do you use to make your videos?

[@statquest]

keynote and finalcut pro.

[@stum5033]

Great video thank you!

[@statquest]

Thanks!

[@nancyanderson5413]

LOVE YOUR VIDEOS, BAM!!!

[@statquest]

Glad you like them!