Power Analysis, Clearly Explained!!!

  Рет қаралды 327,589

StatQuest with Josh Starmer

StatQuest with Josh Starmer

Күн бұрын

Пікірлер: 436
@statquest
@statquest 4 жыл бұрын
NOTE: This StatQuest was brought to you, in part, by a generous donation from TRIPLE BAM!!! members: M. Scola, N. Thomson, X. Liu, J. Lombana, A. Doss, A. Takeh, J. Butt. Thank you!!!! Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@JimtheEvo
@JimtheEvo 4 жыл бұрын
This video makes me feel so happy. I've had a paper in review for a little while, the editor and one reviewer noticed in one of my figures (in which I had shown the raw data and CIs) that a single datum point was not in line with the rest (n=6 not great but better than most molecular biology papers!). They asked me to add an extra replicate to the experiment. I refused on the grounds of p-hacking and showed them a power analysis (+93% power including the variation of the point) and showed the point is a Grubbs outlier. I'm still waiting for the editor to get back to me ;)
@statquest
@statquest 4 жыл бұрын
BAM! :)
@raycyst-k9v
@raycyst-k9v 4 жыл бұрын
What did the editor reply?
@sockaccount8116
@sockaccount8116 4 жыл бұрын
@@raycyst-k9v Editor said: BAAM!
@karolgilbertosolanosuarez9094
@karolgilbertosolanosuarez9094 2 жыл бұрын
So... What happened? Do you have your DOI for us to read? 🎉🥳🤩
@JimtheEvo
@JimtheEvo 2 жыл бұрын
@@karolgilbertosolanosuarez9094 I won ;) gurney et al 2020 microbiology Combinatorial quorum sensing in Pseudomonas aeruginosa allows for novel cheating strategies
@raycyst-k9v
@raycyst-k9v 4 жыл бұрын
Man josh i am trying to finish your playlist . Before this i had tried khan academy, brandon foltz, few books and i failed horribly. P value was something i could never grasp . I think the issue was that everyone talked about what is p value but never explained HOW it is calculated. You really go in the depth of how and why. And this one video right here has blown my mind. It just doesnt explain power analysis but also how you estimate population mean and how p values work . It's 5 am right now and i am still studying .Youve really blown my mind!
@statquest
@statquest 4 жыл бұрын
Hooray!!! I'm glad my videos are helpful. :)
@raycyst-k9v
@raycyst-k9v 4 жыл бұрын
@@statquest HUGE BAM!
@mathiasmadsen1992
@mathiasmadsen1992 3 жыл бұрын
You've freaking done it again. I can't belive how simple this explanation was compared to the lecture I got in school. Thank you very much!
@statquest
@statquest 3 жыл бұрын
bam!
@noedits5543
@noedits5543 3 жыл бұрын
the other teachers cannot explain the concepts clearly because their concepts are not clear in the first place LOL. thats where our Mr BAM Sir rocks :D
@tanishadorn6126
@tanishadorn6126 2 жыл бұрын
Thank you! For my paper my lecturer said we don’t have to include a power analysis, so it wasn’t taught… so glad I watched this because I was struggling to justify my sample size and now it all makes so much more sense.
@statquest
@statquest 2 жыл бұрын
Awesome!!! :)
@peterkilindberg675
@peterkilindberg675 2 жыл бұрын
Wow! I'm very impressed by your very clear explanations and calculations in this video. It was very easy to understand and follow. Big thanks from a PhD-student in Sweden!
@statquest
@statquest 2 жыл бұрын
Thank you very much! :)
@efrensuarez4950
@efrensuarez4950 3 жыл бұрын
One of the best channels out there to learn statistics.
@statquest
@statquest 3 жыл бұрын
Thank you!
@khoiavo
@khoiavo 2 жыл бұрын
I learned more (and laughed more) in these 16 minutes than a whole semester at med school. Thank you!
@statquest
@statquest 2 жыл бұрын
bam! :)
@sushanthraj3455
@sushanthraj3455 Жыл бұрын
I am so happy to see the growth rate of Josh Starmer's StatQuest. I still remember I was one of the few subscriber who joined your fan base when the subscribers where in thousands!. Great going happy learning
@statquest
@statquest Жыл бұрын
Thank you very much! :)
@JS-wv7vn
@JS-wv7vn 2 жыл бұрын
Thanks I haven’t slept this good in weeks
@statquest
@statquest 2 жыл бұрын
noted!
@vladfarias
@vladfarias Жыл бұрын
Your videos are absolutely amazing! It would be fantastic if you could create (if it doesn't already exist) a video covering the entire hypothesis testing process, including all the steps. That would involve determining the sample size and addressing the temptations encountered along the way until reaching the final result.
@statquest
@statquest Жыл бұрын
Great suggestion!
@uberdonkey9721
@uberdonkey9721 2 ай бұрын
I've done so much stats but akways had a difficulty understanding power analysis. This is very clear, and practical.
@statquest
@statquest 2 ай бұрын
Thanks!
@SubDonkess
@SubDonkess 2 жыл бұрын
Fantastic! I’m taking a statistical modeling class with machine learning and this particular topic just wasn’t sticking with me. As advertised, this was super clear and nailed home all the key points!
@statquest
@statquest 2 жыл бұрын
Glad it was helpful!
@MrDrache7
@MrDrache7 4 жыл бұрын
Josh in May 2020: "Imagine there is a virus" 2020: "Hold my beer"
@statquest
@statquest 4 жыл бұрын
:)
@d_b_
@d_b_ Жыл бұрын
At 14:09, several assumptions and parameters are presented on the screen, creating the impression of the existence of "power analysis hacking." Furthermore, the concept of double-layered probabilities, where there is an 80% likelihood of correctly rejecting the Null Hypothesis and a 5% chance of randomly obtaining results below the threshold, is enough to make one's head spin. Nevertheless, this video provides the most exceptional and clear explanation.
@statquest
@statquest Жыл бұрын
Thanks!
@pandapanda7966
@pandapanda7966 Жыл бұрын
i have a question. so knowing statistical power, significance level, effect size, and sample size are all related through power analysis, then wouldn't choosing a statistical power (or even significance level, as we know 0.05 threshold is just for teaching demonstration and in reality it can be any different amount) also be consider p-hacking by effectively getting a desire sample size in "remote" in order to have it prove (or disprove) hypothesis at the researcher's discretion? or is there an objective way to determine the statistical power (and significance level)? most of the materials i read often says "commonly use power = 0.8 or alpha = 0.05" or even heard "amount chosen at researcher's discretion" but without giving sufficient reason for picking those amount.
@statquest
@statquest Жыл бұрын
There are a couple of things to say. 1) Because we select all of the thresholds (power, significance etc.) before doing the experiment, we end up with a sample size that should not be biased in terms of having a higher probability of giving us a false positive of false negative. The key is that we set those parameters before we do the experiment and then we have to live with the result no matter what it says. If we didn't get the result we wanted, and then we did it again, then that would be p-hacking. 2) The thresholds for significance and power are often "field specific" and depend on how easy or difficult it is to control things. In physics labs, they can control things relatively well, so they tend require stricter thresholds. However, when working with human data, where it's hard to control anything, then they tend to have more relaxed criteria. The thresholds also change depending on how serious the consequences are of getting things wrong. For example, if we are testing for a Ebola, we might want to error on the side of caution and allow more false positives so that we can minimize the false negatives.
@nikhilgoyal204
@nikhilgoyal204 2 жыл бұрын
Josh, thanks so much for your videos - so clearly explained relative to other content I've seen on inference. I do have a few questions. 1. In an A/B testing scenario, we don't know the mean and the standard deviation of the distribution of the treatment group before hand. We do know the mean and the standard deviation of the prior distribution. If I want to estimate sample size required for different effect sizes, holding the chosen p-value threshold constant at 0.05 and power at 0.8, how would we do that given that we don't have s2 or standard deviation of the second group? 2. Also since we don't know the mean of the treatment group's distribution, how would we calculate the estimated difference in means to plug it into the formula for effect size? 3. Do you have a video on How to pick the right test?
@statquest
@statquest 2 жыл бұрын
1) Do a pilot study to get a sense of the mean and standard dev. 2) Same 3) Not yet.
@faye8212
@faye8212 3 жыл бұрын
Thanks Josh, you are the best at storytelling when explaining statistics.
@statquest
@statquest 3 жыл бұрын
Wow, thanks!
@taotaotan5671
@taotaotan5671 4 жыл бұрын
Simulation is really helpful to understand this.
@konstantinlevin8651
@konstantinlevin8651 Жыл бұрын
Hey! After chatting for an hour with chatgpt about p-values, p-hacking and power analysis, I wanted to test my knowledge with a simple case and chatgpt suggested me a coffee shop experiment. The experiment is going to be about a new bean and if it improved the customer satisfaction or not. I'll try to prepare a dataset for this. If I can, I'll try to share! Also, thanks a lot for the content. I want to be a machine learning engineer and decided to learn the fundementals of statistics for that, but statistics basically changed the way I see the world ahahahah.
@konstantinlevin8651
@konstantinlevin8651 Жыл бұрын
I have to learn how to do those statistical tests though :)) I'll watch the quest
@statquest
@statquest Жыл бұрын
bam!
@YASHKUMARJAIN
@YASHKUMARJAIN 4 жыл бұрын
Totally loving the series
@statquest
@statquest 4 жыл бұрын
Hooray! :)
@ZHL-n9l
@ZHL-n9l Жыл бұрын
Thank you Josh for being so careful and patient. I would like to ask you a question that I am not quite clear about. You used s when calculating pooled estimated standard deviations and represented s with a dotted line on the normal distribution. However, it seems that s is very wide in the figure, which looks like 2s. I want to make sure that this s is what we call s, right?
@statquest
@statquest Жыл бұрын
's' looks pretty accurate to me. About 65% of the area under each curve should be under the dotted line.
@PrabhakarKrishnamurthyprof
@PrabhakarKrishnamurthyprof 3 жыл бұрын
Thank you Professor. I will use it for my class. It is so well explained, I learnt how to explain complex concepts.
@statquest
@statquest 3 жыл бұрын
Glad it was helpful!
@abdelghanyaref6516
@abdelghanyaref6516 2 жыл бұрын
i swear to god that you are a life saver man
@statquest
@statquest 2 жыл бұрын
:)
@haneulkim4902
@haneulkim4902 3 жыл бұрын
Thanks Josh! Just want to make sure, the green and red normal distributions are created from 1. collecting data_size=n from one group, calculate mean 2. repeat 1 R times 3. use R means to create green/red normal distribution correct?
@statquest
@statquest 3 жыл бұрын
Unfortunately that's not correct. The green and red distributions are the "population" distributions that we are trying to estimate with relatively small sample sizes. For details on population distributions and how they are different from samples, see: kzbin.info/www/bejne/rJrOnJytn7aknLc and kzbin.info/www/bejne/iau9Z3qmmMuih7s
@zahrarezazadeh293
@zahrarezazadeh293 4 жыл бұрын
Hi Josh, thank you a lot for the awesome videos! I have two basic questions: 1. The denominator for the pooled estimated SD in a general condition is the number of the distributions (and not always 2), right? 2. What is the deal with statistics power calculator? isn't there simple formulas we can use ourselves? and does the googling we do to choose one, involve the nature of our experiment, or we just randomly choose one?
@statquest
@statquest 4 жыл бұрын
1) I'm not sure how you could have more than two distributions, but, presumably, if you did, then you would divide by that number. 2) Every single statistical test has a different formula for doing power calculations. And there are a lot of statistical tests, because there are a lot of different experiment types and data types. Going through every single statistical test would take forever and, to be honest, be pretty boring. So the best thing to do is to google "power calculator" and you'll find a page that has a pulldown menu where you select the test that you want to do (t-test? chi-square goodness of fit? K-S test? etc.) and then plug in your numbers. bam.
@TheodoraPapadopoulou-c3j
@TheodoraPapadopoulou-c3j 3 ай бұрын
Thank you, very helpful! I am a new bioinformatician and planning to do differential gene expression with DESeq2 comparing a human sample size=1000. How could we apply power analysis for RNA seq differential expression?
@statquest
@statquest 3 ай бұрын
I believe there are some standard RNS-seq power calculators out there that you can use. I remember using one, but that was 6 years ago and I bet there is something better now.
@rakeshdey3993
@rakeshdey3993 3 жыл бұрын
You are a great teacher, Josh ! Thank you so much. For a long time, I had difficulty in understanding this concept. Now, it is crystal clear.. :-)
@statquest
@statquest 3 жыл бұрын
Thanks!
@alexrinconp
@alexrinconp 2 жыл бұрын
Dude, you have awesome explanatory superPower!
@statquest
@statquest 2 жыл бұрын
Thanks! 😃
@SergeySenigov
@SergeySenigov 2 жыл бұрын
Josh, in terms of the statement 7:49 «because we don’t have a lot of confidence in the estimated means, we’ll end up with a relatively large p-value», can we conclude statement like this: we’ll end up with relatively more both large and smaller p-values, ie p-value more «unstable»?
@statquest
@statquest 2 жыл бұрын
That sounds like a reasonable statement. However, how "stable" or "unstable" the p-value is probably depends on the underlying distributions.
@janicealmojuela9068
@janicealmojuela9068 3 жыл бұрын
watching a night before my finals for STaT! thanks josh
@statquest
@statquest 3 жыл бұрын
Good luck!!
@Niilamaru
@Niilamaru 4 жыл бұрын
Nice way to start a remote workday.
@drpkmath12345
@drpkmath12345 4 жыл бұрын
Niila Saarinen yes right?
@statquest
@statquest 4 жыл бұрын
Hooray! :)
@pradeepkumar-ew1ze
@pradeepkumar-ew1ze 4 жыл бұрын
Hope your boss is not the subscriber :P
@apoorvasrini2196
@apoorvasrini2196 2 жыл бұрын
This explanation is amazing!!! Thank you so very much!!
@statquest
@statquest 2 жыл бұрын
Thank you!
@ericzheng4815
@ericzheng4815 2 жыл бұрын
The interesting examples really gave me a new perspective on the problem. Appreciate this! Thank you so much!
@statquest
@statquest 2 жыл бұрын
Glad it was helpful!
@lunapopo8415
@lunapopo8415 4 жыл бұрын
15:04 It seems tricky to find the right df for t score, for such small sample calculation, when n is unknown. Is it a trial and error process to find the right df?
@statquest
@statquest 4 жыл бұрын
df for a t-test is directly related to the sample size. So if you can solve for 'n', you can solve for the degrees of freedom.
@bowenzhang8714
@bowenzhang8714 2 жыл бұрын
This tutorial is great and make a lot of common sense.
@statquest
@statquest 2 жыл бұрын
Thank you!
@alexw31415
@alexw31415 Жыл бұрын
Been following the Statistics Fundamentals playlist quite well up until this point. Now things are getting a bit meta. P-values are the threshold for false positives, to be able to say "we have a 5% tolerance for getting the wrong result" - that's already pretty meta. Then FDR and the BH-method look at "distributions of p-values" to sub-divide the true positives from the false positives. Double-meta. Now on top of that we are doing a "power analysis" to get an 80% chance of trusting our p-values? Triple meta? We already set our tolerance of 0.05, saying "we know we'll get it wrong 5% of the time" so why do we then care about if those 5% are true/false or "close enough". I'll keep watching but FYI that's the thought process for a new viewer.
@statquest
@statquest Жыл бұрын
Power is sort of the opposite of a p-value. p-values tell us about the probabilities of false positives. Power tells us about the probabilities of false negatives.
@alexw31415
@alexw31415 Жыл бұрын
Very nice and simplified way to put it, thank you Josh! Keep up the good work
@Dragonite789
@Dragonite789 2 жыл бұрын
Thanks Josh. I wanted to ask what should you do if all your data comes from one giant experiment/repository. Your explanations usually assume we have done multiple experiments of small sample size and can generate pooled statistics. But if you did one big experiment with a very large sample size, should you artificially separate individual test into small groups so you can generate pooled statistics? Thanks again, I hope i make sense.
@statquest
@statquest 2 жыл бұрын
Most of the time people do one big experiment and collect all of the data at once. This is the standard procedure. However, in the video, I show what happens if you can repeat that same experiment a bunch of times to give you a sense of how whatever you are estimating (like the mean) has less variation when you have more samples. Thus, the repeated experiments in this video are just an illustration to give you confidence that a large sample size will be more accurate that a small one.
@Samurai_Jack__
@Samurai_Jack__ Жыл бұрын
I'm happy my fate made me study statistics after you made this playlist , Thank you so much for this amazing explanation
@statquest
@statquest Жыл бұрын
Happy to help!
@nishat_zaman
@nishat_zaman 3 жыл бұрын
This lecture was really excellent 😊
@statquest
@statquest 3 жыл бұрын
Glad you liked it!
@jiayinruan9282
@jiayinruan9282 2 жыл бұрын
so clear and understandable !!!! Thank you very much!!!
@statquest
@statquest 2 жыл бұрын
Glad it helped!
@suhacutis23
@suhacutis23 3 жыл бұрын
Thank you for making learning more interesting. Double bammm for the knowledge and funny remarks
@statquest
@statquest 3 жыл бұрын
Thank you! :)
@financequant
@financequant 4 жыл бұрын
Excellent lesson on power analysis. I am immensely impressed by StatQuest’s instructional videos. One criticism here, however, is that the final computation of the necessary sample size to achieve the desired power could have been covered rather than telling a student to find an app on some university’s web site. I think most students could probably work out the math but it seems to go against the overall grain of the talk which careful walks the student through a well chosen toy example that nevertheless fully elucidates the process.
@statquest
@statquest 4 жыл бұрын
Thanks! I decided to not go over the math for calculating power because I did not want to give the illusion that there was a single equation that fit all situations. In reality, for each test, there is a different equation that we need to use to calculate power and, rather than worry about all of that, I thought it would be easier (and much more practical) to show how power analyses are done.
@andrewl372
@andrewl372 5 ай бұрын
Thank you for the great video! I have one question though- 1:51 In the normal distribution graph, If the variable is day to recovery which can't be lower than 0, then would the estimated populatiln probability distribution be following poisson distribution..? Sorry if I may be confused with the concepts in advance!
@statquest
@statquest 5 ай бұрын
Believe it or not, the normal distribution is often used to approximate the poisson distribution. The idea is that if the means are far enough from 0, then it's a good fit. This is due to the central limit theorem. For details, see: online.stat.psu.edu/stat414/lesson/28/28.2
@andrewl372
@andrewl372 5 ай бұрын
@@statquest I understand the explanation, thank you!!
@MrCracou
@MrCracou 4 жыл бұрын
This is, as usual, excellent. I'm a little bit surprised that you do not show alpha and beta for a given test (as the surface below or above the critical value). It allows people to understand the main problem of the power analysis: you need to bet on the effect size as, by definition, it's unknown. It leads, as you know, to soo many problems as we alsways discover too late that the actual mean difference is smaller than expected (also known as the "curse of just a little bit too small sample". The most common solution being a ritual sacrifice of a random slave. Sorry, I want to mean a PhD student.
@josephgan1262
@josephgan1262 3 жыл бұрын
Hi Josh! Thanks for your amazing explanation! Appreciate greatly! I have a questions. I read from a book, it says that large sample size can have unwanted implication. QUOTE ....'By “too high” we mean that by increasing sample size, smaller and smaller effects (e.g., correlations) will be found to be statistically significant. The researcher must always be aware that sample size can affect the statistical test either by making it insensitive (at small sample sizes) or overly sensitive (at very large sample sizes). UNQUOTE So does it mean that even the test we are conducting is from the same distribution(or null hypothesis is true) . It will some how reject the null hypothesis given large enough sample size? Thanks!
@statquest
@statquest 3 жыл бұрын
Regardless of the sample size, you always need to make sure that the effect size is large enough to have "meaning". For example, if taking an expensive medication increases life span by 2 seconds, then who cares if it is statistically significant or not - 2 seconds is not worth taking an expensive medication. So when ever you do statistics, don't just look at the p-value, also look at the effect size and make sure it is meaningful.
@hussienali6561
@hussienali6561 2 ай бұрын
Thanks for your illustration. but to understand it more, you explained it on an example of two sample test. What if we run one sample test ?
@statquest
@statquest 2 ай бұрын
It's the same - even with just one sample, the sample has a distribution that may or may not be the same as an ideal distribution that we are comparing it to.
@jakubmazur1159
@jakubmazur1159 3 жыл бұрын
Love your videos, thanks! :) Quick question though - from this video it seems that we should google 'statistics power calculator' after we set up 3 things - alpha, power and effect size. We can easily choose alpha and power, but what about effect size? It seems to be the thing that you are able to calculate after the test finishes, you can't set it up beforehand. Or maybe you meant something like 'minimal effect size that is enough', e.g. from business perspective?
@statquest
@statquest 3 жыл бұрын
It depends on the area you are working in. For example, in biology, we often want a 2-fold effect size. In physics, a smaller effect size might be good enough. So you have to know what people want in the field to determine the value.
@jakubmazur1159
@jakubmazur1159 3 жыл бұрын
@@statquest thank you for the answer. And in case of AB testing? It would be nice to know beforehand what would be the sufficient sample size. Then what should we set up in place of effect size?
@statquest
@statquest 3 жыл бұрын
@@jakubmazur1159 Again, this is domain specific.
@drrajkumarsrihari
@drrajkumarsrihari 2 жыл бұрын
Thanks
@statquest
@statquest 2 жыл бұрын
Wow!!! THANK YOU VERY MUCH!!!!
@suk-yongjang8885
@suk-yongjang8885 4 жыл бұрын
Excellent explanation...
@statquest
@statquest 4 жыл бұрын
Thank you! :)
@ByeolKim-w8q
@ByeolKim-w8q 10 ай бұрын
I love it when you're saying "BAM!" lol
@statquest
@statquest 10 ай бұрын
BAM! :)
@Mostafaseyyedabadi
@Mostafaseyyedabadi 2 ай бұрын
Thanks for the great video. In your example you said 9 measurements per a group is enough. What does group mean in correlation studies? Do we have more than one group in these studies?
@statquest
@statquest 2 ай бұрын
What time point in the video, minutes and seconds, are you asking about?
@Mostafaseyyedabadi
@Mostafaseyyedabadi 2 ай бұрын
@@statquest thanks for your quick reply. I asked about minute: 14 and second: 59
@statquest
@statquest 2 ай бұрын
@@Mostafaseyyedabadi If you're just trying to establish a correlation, you just have one group. You can determine the sample size using a sample size calculator for regression. For example, you could try this one: www.statskingdom.com/sample_size_regression.html
@lupita3689
@lupita3689 4 жыл бұрын
Wait, but you assumed you already knew the two distributions? Since you'd need the population mean and s.d. to calculate the effect size?
@statquest
@statquest 4 жыл бұрын
The mean and standard deviations are estimated from the data.
@lupita3689
@lupita3689 4 жыл бұрын
StatQuest with Josh Starmer But if you already have the sample mean and data, wouldn’t that mean that you already finished your experiment and have the sample size fixed? To me this seems like a chicken and egg problem. Or are we talking about an iterative process where you finish the experiment, check if the sample size is sufficient for the Power you want, then do more experiment to get more sample?
@statquest
@statquest 4 жыл бұрын
You can estimate the mean and standard deviation from a preliminary experiment, or from a literature search, or just use an educated guess. I mention all of these options at 13:57
@lupita3689
@lupita3689 4 жыл бұрын
StatQuest with Josh Starmer Ah, sorry I missed that, and thanks!
@av3499
@av3499 4 жыл бұрын
@@lupita3689 that was exactly my thinking as well. 1. To check if samples belong to different groups, collect samples. 2. To know how many samples to collect, use power formula. 3. To get power formula, plug in mean & SD of the 2 different groups.
@KnowDAOself
@KnowDAOself Жыл бұрын
excellent video series in general, and particularly good video on Power Analysis! Thank you for making and sharing!
@statquest
@statquest Жыл бұрын
Thank you!
@nelhuens7590
@nelhuens7590 2 жыл бұрын
Great explanation! What do you do if you want to compare a normal distribution with a non-normal distribution?
@statquest
@statquest 2 жыл бұрын
If the sample size is large enough, the distribution doesn't matter (per the Central Limit Theorem: kzbin.info/www/bejne/j3LPe3Z7ea1lq7s ). Alternatively you can use non-parametric methods. These do not depend on the distribution.
@nelhuens7590
@nelhuens7590 2 жыл бұрын
@@statquest Thank you very much! We have a really small sample size, so I don't think that is an option. Do you maybe have a link to wich non-parametric methods we can use?
@statquest
@statquest 2 жыл бұрын
@@nelhuens7590 Unfortunately, I don't. But if your data is continuous, instead of using a t-test, you can use a Mann-Whitney U test: en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test
@sanaaharrass6239
@sanaaharrass6239 2 жыл бұрын
great explanation. Thanks!
@statquest
@statquest 2 жыл бұрын
Thanks!
@martinotanasini3716
@martinotanasini3716 7 ай бұрын
Thanks for the great video! Do I understand correctly that the sample size determined from the power analysis depends on the means and variances estimated from the first experiment? How do I deal with the fact that this first experiment could result in estimating means and variances far away from the population's?
@statquest
@statquest 7 ай бұрын
You have to start with some general sense of the variation in the data. It doesn't have to be perfect, but it's the best you can do.
@kairoselvarro6546
@kairoselvarro6546 2 ай бұрын
Thank you for the detailed explanation! But...I am having trouble finding a reliable power calculator online that allows me to input the Effect Size...perhaps you could recommend a few? What power calculator did you use to get the value of sample size 9? Regardless, I loved the first part of your video! Very detailed, very interactive, thank you for this!
@statquest
@statquest 2 ай бұрын
www.statskingdom.com/32test_power_t_z.html
@abolimalwadkar5216
@abolimalwadkar5216 3 ай бұрын
Thanks for this video I now get the 'effect size' true meaning I am unable to find the sample size calculator. I am learning stats to work on SPSS but was curious how you got the value 9 for the example
@statquest
@statquest 3 ай бұрын
Just google "sample size calculator" and you should be good to go.
@baksteen2420
@baksteen2420 3 жыл бұрын
Thanks for another great video Josh! There's one thing I still fail to understand, though: Isn't it true that a p-value and Power measure the same thing, namely the chance of (correctly) rejecting the H0? And if this is the case, is a chance of 80% of correctly rejecting the H0 not a downgrade from having a 95% chance when using an alpha of .05?
@statquest
@statquest 3 жыл бұрын
No, p-values and power are fundamentally different. p-values assume that the null hypothesis is correct. Power assumes that the null hypothesis is not correct. This makes pretty much everything about them different.
@baksteen2420
@baksteen2420 3 жыл бұрын
​@@statquest thanks a lot for replying! I clearly still have some work to do😁
@statquest
@statquest 3 жыл бұрын
@@baksteen2420 To learn more about p-values, try: kzbin.info/www/bejne/rJbQi6d7gptmfbs and kzbin.info/www/bejne/gILGZKyuZZKEb6c
@a45701
@a45701 Жыл бұрын
According to the video, calculating the effect size in the end using the difference of means, it is divided by the pooled estimated standard deviations. What standard deviation (std) is that? The std of samples, or the standard deviation of the estimated difference in means? (the std of the mean should be 1/sqrt(n) of the samples std) Does the effect size follow the same calculation as Cohen's d ?
@statquest
@statquest Жыл бұрын
It's the standard deviation of the samples.
@titustan2700
@titustan2700 9 ай бұрын
I have a question: if the original data that are used for the sd and mean of the two distributions are not representative due to small sample size, does all of this power analysis breaks down. That is to say, power analysis is only based on previous large sample size data?
@statquest
@statquest 9 ай бұрын
No, it doesn't break down because we give ourselves the potential to be wrong. When the "power" is set to 80%, that means we only have an 80% chance of correctly rejecting the null hypothesis. If we want a 90% chance of correctly rejecting the null, we need to increase the sample size.
@syhusada1130
@syhusada1130 2 жыл бұрын
So if power analysis told me to have 500 sample, and I have only 5, can I reach that 500 with bootstrap 100 times?
@statquest
@statquest 2 жыл бұрын
First of all, when we do bootstrapping, we usually do at least 1,000 and often 10,000 or more permutations. And, unfortunately, these permutations do not increase our power. All they do is give us a sense of the the distribution of whatever metric we are calculating (usually the mean) given the original sample size. Thus, if your sample size is 5 and we use that to calculate the mean, bootstrapping would then tell us how a mean, given n=5, is distributed. It doesn't increase the probability we will correctly reject the null hypothesis.
@datamanager-th
@datamanager-th 2 жыл бұрын
Thank you for the excellent video. If you please, I would like to ask you a simple question. In order to calculate the effect size, you need to know S which depends on the sample size (in the video it is 10). Then you use this to estimate the "sample size" which is 9. What happens if they are so much different? Can this happen in 2 cases 1. the start sample size >> the calculated one and vice versa.
@statquest
@statquest 2 жыл бұрын
In theory, 's' (the standard deviation) does not actually depend on the sample size. We can get it from any source or just guess based on previous experience. Thus, if the output sample size is very different, that's OK.
@datamanager-th
@datamanager-th 2 жыл бұрын
@@statquest Thank you very much for your answer.
@danlabrador
@danlabrador 4 жыл бұрын
14:55 If the calculator suggests a sample size of 9, is it safe to increase the sample size or should I just stick to the calculated sample size?
@statquest
@statquest 4 жыл бұрын
You can always collect and use more measurements than the minimum.
@ShuaiWang12
@ShuaiWang12 2 жыл бұрын
Thank for making these videos. One question on the beginning of this video. What is the actual method behind to "do a statistical test and compare the means and get a p-value=0.06"? From what I read, is T-test for two sample mean in this case? Thanks!
@statquest
@statquest 2 жыл бұрын
Yes, in this case I'm using a t-test.
@niazsayakhan1791
@niazsayakhan1791 2 жыл бұрын
Thank you Josh! I have a question. Could you collect a sample from Drug A and Drug B and use the two samples to calculate an Estimated Mean and Standard Deviation in order to calculate the Sample Size required for future research?
@statquest
@statquest 2 жыл бұрын
Yep!
@niazsayakhan1791
@niazsayakhan1791 2 жыл бұрын
@@statquest BAM!!! Thanks, Josh!
@harveyhensley875
@harveyhensley875 2 жыл бұрын
Wonderful video, very clear as always. Often the terms 'type 1 error' and 'type 2 error' are used in these discussions, or α and β. Was there a reason for avoiding this terminology, aside from just generally avoiding jargon?
@statquest
@statquest 2 жыл бұрын
I really don't like that terminology. I'd rather people say "false positive" or "false negative" instead of "type 1" or "type 2".
@SenthilKumarSubramanian-x6d
@SenthilKumarSubramanian-x6d Жыл бұрын
Hi, very useful material and well explained. Thanks. What should be the power if i want to accept null hypothesis? How should be the overlap and sample size, in this case?
@statquest
@statquest Жыл бұрын
Unfortunately, traditional statistics is not designed to accept the null. The best we can do is to "fail to reject the null". You can make this failure relatively convincing if you select a relatively high power (like 95%) for rejecting the null and still fail.
@SenthilKumarSubramanian-x6d
@SenthilKumarSubramanian-x6d Жыл бұрын
@@statquest I have a subject interviewed at time 0 and at the same time, the subject has given rating for those questions asked via an app. Now, such measurements are repeated at an interval. I want to show that there is no statistical difference between interview rating and self-rating via app. I need to find out the sample number for this analysis. What should be the power , alpha and effect size to find out the sample number? I appreciate your help. Thank you
@maximillianweil2672
@maximillianweil2672 Жыл бұрын
Is it possible to calculate the sample size required even if the distributions can't be assumed to be normal distribution? I would think it is possible by using the Mann-Whitney U test but can't seem to figure out how. Thanks for the great explanation anyways!
@statquest
@statquest Жыл бұрын
You can definitely calculate power for a mann-whitney U test. www.benchmarksixsigma.com/calculators/sample-size-calculator-for-mann-whitney-test/
@stonewatertx
@stonewatertx 2 жыл бұрын
Thanks for the video! One quick question: before we calculate the effect size and sample size, how do we get estimated µ and s for green and purple group in the first place?
@statquest
@statquest 2 жыл бұрын
I answer this question at 13:57
@stonewatertx
@stonewatertx 2 жыл бұрын
@@statquest BAM!
@statquest
@statquest 2 жыл бұрын
@@stonewatertx :)
@raffaelepiccini3405
@raffaelepiccini3405 2 жыл бұрын
Great video, I’m still a bit confused on how having different sample size for the two groups would affect the calculations
@statquest
@statquest 2 жыл бұрын
Here's something that might help: stats.stackexchange.com/questions/108079/can-i-do-a-t-test-power-analysis-for-unequal-size-groups-which-produces-2-differ
@raffaelepiccini3405
@raffaelepiccini3405 2 жыл бұрын
@@statquest thanks!! that was a useful link
@skytoin
@skytoin 5 ай бұрын
Hi, there are different power calculators online and they require different inputs. How do you call the one that needs: power, threshold, effect size?
@statquest
@statquest 5 ай бұрын
You have to know what kind of test you want to use - once you figure that out, the inputs will all make sense.
@simkort5799
@simkort5799 Жыл бұрын
Great series of video again Josh!! I have one question tho, and I have tried to ask Chatgpt, and it didnt help much... Given we rejected the null, if we say alpha is the possibilities of falsely reject the null, lets say it is 0.05. And then the opposite would be correctly reject the null which is 1-0.05 = 0.95. Isn't this exactly the definition of Power? So given alpha = 0.05, the chance of correctly reject the null is 95%, and that is our power? I know it is somehow false, but i cannot figure out why
@statquest
@statquest Жыл бұрын
I actually have an entire video that answers your question: kzbin.info/www/bejne/iKTGZq2krLdofKM (I'll give you a hint: 1-alpha is not what you think it is. 1-alpha is related to the probability that we will fail to reject the null).
@simkort5799
@simkort5799 Жыл бұрын
@@statquest Thanks a lot Josh! I realized I made that mistake haha! 1-alpha is consist of two parts, 1.correctly accept the true null 2. falsely accept the false null and Power is the chance of correctly reject the false null. So they are not completely the same
@jourdanbrune4856
@jourdanbrune4856 4 жыл бұрын
What if I am trying to compare three groups? How would I calculate the effect size? What other key words should I google in addition to "statistics power calculator"? Thanks in advance for answer my questions, your videos are awesome!
@statquest
@statquest 4 жыл бұрын
Add "ANOVA" to your search (which compares 3 or more groups). In general, if you have a specific test in mind, just add that to your search.
@Ginimo
@Ginimo Жыл бұрын
Hi Josh, for a trustworthy result I would combine Power Analysis to determine the sample size and after that calculate P-values with the FDR correction. Is this right? Or is the FDR correction not more needed after Power Analysis? Many thanks for your response!
@statquest
@statquest Жыл бұрын
If you do multiple tests, you should always use FDR.
@ArinaNotHarina
@ArinaNotHarina 3 жыл бұрын
Awesome channel. Thank you for what you do
@statquest
@statquest 3 жыл бұрын
Glad you enjoy it!
@amirmeysami7868
@amirmeysami7868 4 жыл бұрын
Hi Josh, thanks for the amazing videos! There is one part of this that confuses me, how do you get the population means and population standard deviations to put in the power analysis calculator? Do you simply run the experiment once with many replicates for your positive control and negative control? Then is it okay to apply the resulting sample size to your conditions?
@statquest
@statquest 4 жыл бұрын
You have to estimate the population parameters with preliminary data, or data from other sources (like publications) or just take an educated guess.
@amirmeysami7868
@amirmeysami7868 4 жыл бұрын
@@statquest Thanks very much for your reply!
@somenerdyblonde
@somenerdyblonde 2 жыл бұрын
@@statquest I'm also struggling with this. In previous publications, they don't show the actual numbers for mean and standard deviation; only a graph. I guess I have to just guess based on the graph?
@statquest
@statquest 2 жыл бұрын
@@somenerdyblonde Email the authors of the publication.
@arthurribeirodeabreuchaves1131
@arthurribeirodeabreuchaves1131 3 жыл бұрын
Great, easy, and simple video. Could you provide a reference for the formula you used to calculate Cohen's (d)? Thanks!
@statquest
@statquest 3 жыл бұрын
en.wikipedia.org/wiki/Effect_size NOTE: The pooled standard deviation is simplified to assume the same number of measurements from both categories.
@Zombiezzz101
@Zombiezzz101 2 жыл бұрын
So just to be clear. At the end of the video it's shown that we need a sample size of 9. If we go over the sample size of 9, we may run into p-hacking. If we go under 9, we will be less confident in our results thus our power decreases. So my question is what is worse, obtaining a lower sample size or obtaining a higher sample size than what is recommended? I've always assumed that obtaining the largest sample size possible is the way to go since it will give you a closer estimate of the true population mean. Is this not the case? Is there always some "goldilocks" sample size number that should always be computed before running a hypothesis test? Thanks
@statquest
@statquest 2 жыл бұрын
You can never have too much data. Collecting more than 9 samples doesn't amount to p-hacking. p-hacking occurs when you do the same test a bunch of different times and then cherry pick which one we want to believe. In contrast, if we have a large data set and we do a single test with all of the data at one time, then we can only have more power to detect differences if they exist.
@Quijanos1
@Quijanos1 3 жыл бұрын
Great explanation! Thank you.
@statquest
@statquest 3 жыл бұрын
Thanks!
@charlesgauthier4729
@charlesgauthier4729 2 жыл бұрын
I discovered these videos a while ago and they're really great for understanding various statistical concepts. I need to do some power analyses on non-parametric data, is that even a thing? Is there even a variability estimate for a Mann-Whitney two sample test? Thanks.
@statquest
@statquest 2 жыл бұрын
Power estimates for non-parametric models exist. Just google "mann whitney u test power calculation"
@ayesha-w3w4f
@ayesha-w3w4f 12 күн бұрын
Hello, thank you so much for this video. I have a question as I want to try do a power analysis for my data however it has a very small sample size so I could not assume that the data was normally distributed. If you do do the Shapiro Wilks test most of the data appears to be normally distributed. In the actual study I compared medians and IQRs rather than means and SDs. Does this mean I cannot do a power calculation because I decided to use non-parametric tests for my data because I couldn't assume normality?
@statquest
@statquest 12 күн бұрын
There are plenty of power calculators out there for non-parametric tests.
@vladfarias
@vladfarias Жыл бұрын
Man, I'm totally hooked on your playlist (it deserves to be on Netflix!). When you mentioned having a sample size of 9, I got a bit confused. How many samples, each with a size of 9, should I collect? I always struggle with this. For instance, would it be the same if I had 3 samples, each with a size of 3? Could you clarify that?"
@statquest
@statquest Жыл бұрын
What time point, minutes and seconds, in the video are you referring to?
@vladfarias
@vladfarias Жыл бұрын
@@statquest 14:56. Anyway, would it be the same? (One sample size 9 and 3 samples size 3)
@statquest
@statquest Жыл бұрын
@@vladfarias It depends. For example, 1 person getting 9 measurements could be different than 3 people getting 3 measurements each because there could be variation in how the 3 different people collected measurements. Or if we use the same person, but collected 3 measurements on 3 different days, because there could be extra variation due to the different days. So your estimate of variation needs to take how the data will be collected into account.
@gayanbandara1909
@gayanbandara1909 Жыл бұрын
Supereb sir... Thank you..
@statquest
@statquest Жыл бұрын
Thank you!
@ABeardedDad
@ABeardedDad 3 жыл бұрын
Awesome video! There's really nothing like being able to visualise a concept. Is there a reason you never mentioned the standard error when you were explaining how larger sample sizes increase the confidence that your sample mean is close to the population mean? I mean standard error is literally the measure of that confidence right?
@statquest
@statquest 3 жыл бұрын
Yes, the standard error is the measure of that confidence. However, I omitted it because I was simply trying to convey the concept of power, rather than dive into the technical terminology.
@casualcasual1234
@casualcasual1234 3 жыл бұрын
Thanks for the video :) and I got 2 questions: 1. If I cannot find any published data for the effect size, what should be the sample size to generate a preliminary data? 2. is the threshold of significance (alpha) = p value?
@statquest
@statquest 3 жыл бұрын
1) Even if there is no published data, there is probably a standard sample size used within your field. Or just take a guess. 2) Yes
@lifewithrahman7955
@lifewithrahman7955 Жыл бұрын
It was very very very easy explanation
@statquest
@statquest Жыл бұрын
Thanks!
@bristojoemon9423
@bristojoemon9423 3 жыл бұрын
I don't understand why we took the null hypothesis as drug a and drug b are different rather than they are same. Can someone explain on how we consider this null hypothesis and alternative hypothesis?
@statquest
@statquest 3 жыл бұрын
In this case, the null hypothesis is that the drugs are the same (not different). For details on the null hypothesis, see: kzbin.info/www/bejne/ZqDGZWx6rqZmnrc
@bristojoemon9423
@bristojoemon9423 3 жыл бұрын
@@statquest thank you.
@aashraysaini994
@aashraysaini994 3 жыл бұрын
Thank you Josh for all your videos! they help make statistics look less deadly. Would look forward to some content on bayesian statistics and then maybe solving problems using both bayesian and frequentist methods :)
@statquest
@statquest 3 жыл бұрын
Will do!
@danielsobczynski2107
@danielsobczynski2107 2 жыл бұрын
Really enjoying these videos - I was recommend by a colleague to check out your channel. One question I wanted to ask you - Is the idea that we would collect 10 samples several times, and then get the mean of the estimated means and compare that to the mean of the estimated means from the other distribution using a t-test? Just curious how it all comes together but maybe that comes in some later videos...thank you
@statquest
@statquest 2 жыл бұрын
The idea is to collect 10 measurements 1 time and then do your tests. However, this video illustrates what would happen if we did it a bunch of times to give you a sense of how much variability there would be in different sample sizes. The larger the sample size, the lower the variability in the mean, and thus, the more confidence we can have in its value.
@mikaelfiil3733
@mikaelfiil3733 3 жыл бұрын
Nice videos Josh. One thing concerning p-values that I think should be mentioned is, that they are a continues value, so 0.05 is somewhat arbitrary. In terms of p-hacking and the notion of correlation vs. causality, taking 0.05 too seriously as evidence can be troublesome. You may actually miss out on some good science, just because the p-value was > 0.05 and of course, you may be too optimistic in the other direction. Maybe you already mentioned this, and I just missed it :-)
@statquest
@statquest 3 жыл бұрын
I mention this in my video that explains p-values: kzbin.info/www/bejne/rJbQi6d7gptmfbs
@priyamvadabhardwaj6331
@priyamvadabhardwaj6331 3 жыл бұрын
Thanks for the amazing video! Could you please help understand how to know if the p value difference is significant when data is imbalanced and distributions severely overlap?
@statquest
@statquest 3 жыл бұрын
If the p-value is less than your threshold for significance, than it is significant.
@priyamvadabhardwaj6331
@priyamvadabhardwaj6331 3 жыл бұрын
@@statquest Wow, that was quick! Thanks! One last thing, umm.. is there any book you have written on applied statistics or one you recommend please? I mostly need it for machine learning applications. P.S. embedding you songs into your book would be something!!
@statquest
@statquest 3 жыл бұрын
@@priyamvadabhardwaj6331 I'm writing a book on machine learning that has some basic statistics, but it won't be out for another year. Unfortunately I don't know of any other good book.
@priyamvadabhardwaj6331
@priyamvadabhardwaj6331 3 жыл бұрын
@@statquest Looking forward to it!! Thanks and stay safe!
@uprmt
@uprmt 3 жыл бұрын
Would BH-corrected p-values still be questionable if a power analysis is not conducted beforehand?
@statquest
@statquest 3 жыл бұрын
I think they would be fine. As long as you don't add more data after seeing a p-value (or adjusted p-value) you don't like, you should be good. In that case, the worst thing that happens is that your study is underpowered.
@taemobang4489
@taemobang4489 3 жыл бұрын
@@statquest What should we do If the study is underpowered(e.g. power < 0.5)?..
@statquest
@statquest 3 жыл бұрын
@@taemobang4489 Use your data as preliminary data to conduct a power analysis.
@gabrielagalindo6339
@gabrielagalindo6339 Жыл бұрын
Thank you, great explanation
@statquest
@statquest Жыл бұрын
Thanks!
@Denise_lili
@Denise_lili 3 жыл бұрын
Hi Josh, thank you for the nice video! Quick question, in some online calculators, I find something called "minimum detectable effect", which seems I can pick ANY random number during the sample size calculation. Is it exactly same as the "effect size" you mentioned here in the definition perspective? I'm asking because it seems your effect size is a FIXED number.
@statquest
@statquest 3 жыл бұрын
I believe that the "minimum detectable effect" tells you how small an effect size your sample will be able to detect.
@shikoot
@shikoot 4 жыл бұрын
Dear Dr. Starmer. Imagine I run a pilot test with 5 subjects in each group to get an estimate of the estimated mean and standard deviation from each group, so that I can compute a power analysis. The result of the power analysis suggest that I should run the test with at least 15 subjects on each group. Question: Do I need to measure 15 new subjects for each group or can I reuse the data that I already have (5 listeners on each group) from the pilot test? Is this statistically fine or it is considered p-hacking? And it is an strict no-go or it is not recommended but it is ok? In some research fields, getting data from few subjects is very time consuming and expensive, therefore discarding it must be well justified. Could you share your thoughts about this? Thank you so much and thanks for a great stats course!
@statquest
@statquest 4 жыл бұрын
Ideally you would get new data. Imagine I collected 3 measurements per group, but those measurements were extreme compared to what I would normally get and that there is no real difference. The power analysis would tell me that I need to collect 3 samples. If I did not collect new data, I would get a false positive.
@miladvazirian251
@miladvazirian251 2 жыл бұрын
Great video! But how about dichotomous outcomes, which is very common in clinical trials?
@statquest
@statquest 2 жыл бұрын
The main ideas are the same for dichotomous outcomes, however, the details might be a little different. So I recommend you start by finding a power calculating program for the test you want to use.
@davidalfonsoriveraruiz9077
@davidalfonsoriveraruiz9077 2 жыл бұрын
Excelent video, thank you very much.
@statquest
@statquest 2 жыл бұрын
Glad you liked it!
@deepakmehta1813
@deepakmehta1813 3 жыл бұрын
Hi Josh, Thoroughly enjoyed the video. One quick question though: if the two means are d1 and d2, then how do you to take the difference between them, should it be |d1-d2| or one can used either d1-d2 or d2-d1. On a side note my kids have started complaining - They say that I am always watching stat quest :)
@statquest
@statquest 3 жыл бұрын
The order doesn't matter (and we don't use the absolute value either). If you want to learn a very cool way to compare two means, check out my playlist on linear regression. Believe it or not, a test to compare means is a type of linear regression: kzbin.info/aero/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU (Tell your kids that I apologize for making your watch more StatQuest! :)
The Central Limit Theorem, Clearly Explained!!!
7:35
StatQuest with Josh Starmer
Рет қаралды 840 М.
Covariance, Clearly Explained!!!
22:23
StatQuest with Josh Starmer
Рет қаралды 573 М.
Vampire SUCKS Human Energy 🧛🏻‍♂️🪫 (ft. @StevenHe )
0:34
Alan Chikin Chow
Рет қаралды 138 МЛН
Counter-Strike 2 - Новый кс. Cтарый я
13:10
Marmok
Рет қаралды 2,8 МЛН
Power Analysis
26:55
David Caughlin
Рет қаралды 15 М.
Statistical Power, Clearly Explained!!!
8:19
StatQuest with Josh Starmer
Рет қаралды 311 М.
Maximum Likelihood For the Normal Distribution, step-by-step!!!
19:50
StatQuest with Josh Starmer
Рет қаралды 560 М.
p-values: What they are and how to interpret them
11:21
StatQuest with Josh Starmer
Рет қаралды 1,2 МЛН
Teach me STATISTICS in half an hour! Seriously.
42:09
zedstatistics
Рет қаралды 2,9 МЛН
Hypothesis Testing and The Null Hypothesis, Clearly Explained!!!
14:41
StatQuest with Josh Starmer
Рет қаралды 611 М.
ROC and AUC, Clearly Explained!
16:17
StatQuest with Josh Starmer
Рет қаралды 1,5 МЛН
False Discovery Rates, FDR, clearly explained
18:27
StatQuest with Josh Starmer
Рет қаралды 222 М.
Lecture 1: Introduction to Superposition
1:16:07
MIT OpenCourseWare
Рет қаралды 8 МЛН
Why Your Brain Sabotages Your Goals (and How to Fix It)
11:56
Productive Peter
Рет қаралды 33 М.