Is the Bonferroni Correction Really Necessary?

[@Manny123-y3j]

I have an MS in statistics and broadly agree with you. There are times when it does make sense to apply some kind of correction, but in most cases, it probably makes better sense to publish the results and simply talk about it as a limitation or area of further exploration in the discussion, like you said, especially if the p value is just under 0.05. I would argue we need to collectively move away from viewing the 0.05 as a black or white judgement on whether research in meaningful or not. Even an p value of 0.10 *might* indicate there is a relationship there that would show up more significantly with a larger sample size or if some other factor of the experiment was controlled for. Also, thank you for the references. I will be using them.

[@vanessapoletto255]

I will be sending this link to my PhD committee member that keeps on pushing this analysis!

[@empaulstube6947]

My Two Cents on this issue: if you have small sample size (n =< 100), I recommend NOT USING Bonferroni. Small sample size have less power or less Type I error, or less chance of proving something is statistically significant. Based on this video, Bonferroni is very strict and controlled on the Alpha or Type I error. We wouldn't worry much of the Type I error with small sample size. But if we have large sample sizes, then I highly suggest using the Bonferonni. Large sample size has also lots of power or higher chance of proving something is statistically significant. Having a Bonferroni adjustment will somehow regulate such power. Another important thing to consider is, Type II error is less evil than Type I error. Because Type II error supports the NORMAL while Type I proves the ABNORMAL. To disturb something that is normal should have an OVERWHELMING evidence than just enough evidence. In Research, when we prove something out of the normal (alternative hypothesis), we should actually very and meticulously sure that it is actually true. There are more damaging effects of falsely proving something out of the ordinary.

[@silulekomkhize7839]

Nice, love it!!! To relax the apprehension pertaining family-wise error rate I believe that instead of looking at every single possible comparison, one needs to rather select only those that would be meaningful (that is, with regards to the outcome of interest). In this way the bonferroni wouldn't be so strict because every single comparison you make would be necessitated by sound reason and therefore justifiable. I am aware though, that reasons for selecting certain comparisons over others is a subjective matter. But, I believe this is where you're needed as a scientist, show us your justification for selecting those that you did, and show us what story you're selling. Let me know what you think about this.

[@TheHeadincharge]

While I agree with you, a small sample size has reduced power, but it does not have a reduced type 1 error rate. It instead has an increased type 2 error rate in order to keep the type 1 error rate at a constant value (usually of .05).

[@matthewmcmahon8980]

Video really helped me out, thanks a million. I think with exploratory studies there is real risk of underpowering and loss of potential rich data. A part of me suspects there is something very questionable about the fundamental premises Bonferroni adjustment logic is based on.

[@VenezSteward]

Great video! Thanks a lot! Wondering if it makes any sense to adjust for multiple comparisions when you are comparing means of a single variable between different groups e.g. pH values between 5 water treatments

[@gofelreylopez9413]

I have a problem of using the bonferroni adjustment.. I ran my data by Friedmans Test using spss software and the result is, it has significant difference.. And when I applied Post Hoc Test to determine the treatments with significant differences and using the boferroni adjustment it appeared that all the comparison of my data have no significant difference.. Is it okay not to use this adjustment in order to justify the significant difference result ran by the Friedman's Test?

[@josechvaicer7328]

My experience in the Medical Device industrial Process Validation is that there is very little place to apply the Bonferroni correction. However, for the bio-science research, where controlling variance is indeed a challenge for replications, it could be somewhat useful. So, in contrast with the shameful position of Harvard, MIT and Penn presidency the Bonferroni correction depends on the context.

[@random-i6l2e]

Thank you so much for these videos!

[@floofiq]

hi, my study is to look at pre and post treatment on control and experimental group. i did a 2x2 mixed anova so i had two main effects and an interaction effect. i ve been asked to do a further pre post t test for each group to interpret the interaction effect so i did a paired t test for each group. do i need to do a bonferroni correction? and if yes shall i divide the p value by 2?

[@718syee]

hi. it is a very good video. I would like to ask you a question. I have carried out a paired sample t test with alpha .05, research design of 1 group pretest - posttest experiment. I am measuring 7 subscales of a construct. where i compare the mean before test and mean after test. i would like to ask. is bonferroni correction applicable to paired sample t test? as a lot of information stated that anova etc. but there isnt informaton related to my situation. hope to get your reply. thank you so much in advance

[@miriza2]

What about Benjamini Horchberg method?

[@izzatsyahir9273]

Hi, how to report in writing if we use Bonferroni correction in t-test?

[@souhakabtni5192]

One question please, I am using SmartPLS 3 to do Multi-group analysis. I have three groups and I have to do the permutation-based confidence intervals for the mean values and the variances. In this case, how do I report the measurement invariance of the composite models (MICOM) and the permutation results (with or without the Bonferroni adjustment)? Thank you in advance for all the help :)

[@emanelkhateeb6889]

If I decided to go on with Bonferroni correction. I am quantifying the expression levels of proteins in control (A) and mild (B), moderate (C), and severe (D) diseased groups (4 groups overall). I want to detect any significance between all pairwise comparisons AB, AC, AD, BC, BD, and CD. In that case, should I divide the α by 6 (All the pairwise comparisons) or 3 (Knowing that each group will have 3 independent comparisons only)?

[@ChickennSoupp]

Could you by any chance share what you found out? I know it's been two years haha. Thanks.

[@TheHeadincharge]

The more I learn on FWE, the more errors I notice with a lot of the fundamental logic behind it and the less need I see for it in terms of the way it is normally corrected. Solutions that simply focus on expected significance based on p-value and number of tests performed are much more logically sound I find. Too many assumptions are made in FWER from the start as it assumes that every single test you perform will have a p-value of .05 and disregards the actual data collected. Besides, if you collect a mean that happens to be an extreme score, the number of tests you perform will not control for the errors you will find which is the core issue. Merely setting a more strict alpha is the best way to control for false positives if it is a large concern in a study. There’s also a huge theoretical issue that I often see not mentioned, which is that the original FWER calculation comes from probabilities of repeated sampling of the SAME exact population which is of course not true in any experiment. The FWER calculation is the exact same calculation used when asking the question, “what’s the percentage chance of me rolling a 1 on a 20 sided die over X rolls”. The idea that each comparison being made is the same as rolling the exact same die multiple times is problematic.

[@Isuppose12]

To support and appreciate how2stats, I suggest we at least don't skip the ads.

[@laboratoriodelconocimiento1449]

Thank you for this very clear and didactic video. It has been very helpful. Myself I am used to post-hoc analysis in one way ANOVA, but a reviewer "forced me" to do the Bonferroni corrections because I used multiple chi-square tests to a small sample. I answered with a column of uncorrected p values and a column of corrected p values in my table. Do you think that's a good idea?

[@how2stats]

I'm not sure what you mean. There are no chi-square tests in the ANOVA context. I think what you mean is post-hoc testing the contingency table analysis case. I discuss that procedure in this video (and, yes, correcting one column is appropriate): kzbin.info/www/bejne/mYDYaqWsbZh6Y6s

[@laboratoriodelconocimiento1449]

@@how2stats The problem is that the reviewer asked me to do Bonferroni corrections not in a ANOVA test but in a table where I was comparing nominal variables between two groups by means of chi-square tests. What do you think of that? Was he being reasonable by asking to do Bonferroni corrections in the context of chi-square tests? Post-hoc anaylsis in the context of ANOVA tests is a standard procedure, but what confused me was that he asked for Bonferroni corrections after I usted chi-square tests to compare nominal. dichotomic variables between two groups (in a small sample)

[@jamesjacobthomson6198]

Hi, Thank you for the video. If you have time: I am wondering whether the Bonferroni Correction test can be applied when conducting multiple nonparametric tests on one sample, e.g. the Krushkal H Wallis test or Kolomogorov Smirnof Z test? I am unsure since SPSS automatically produces an adjusted significance result.

[@RR-fg7nu]

Yes, the Bonferroni can be applied to non-parametric tests.

[@ninjanj6148]

It is also automatically applied when you use 'Analyse > Nonparametric tests > Independent samples'.

[@mariamfelici3731]

Excellent video👍🏽

[@andreasstylianou9383]

What about Bonferroni Correction in the framework of crosstabulations and difference in proportions? SPSS manual suggests using Bonferroni Correction. What is your opinion?

[@how2stats]

The Bonferroni correction is really your own option in the crosstabs case; or perhaps the sequential Bonferroni correction? You could possibly get away with the Fisher's protected LSD if it's a 2x3 crosstabs. You might want to check out this video: kzbin.info/www/bejne/mYDYaqWsbZh6Y6s

[@andreasstylianou9383]

how2stats www.ibm.com/support/knowledgecenter/en/SSLVMB_23.0.0/spss/tutorials/xtab_demo_howto_02.html

[@DrFurb]

I read that Bonferroni is too conservative. That we should consider False Discovery Rate instead. Thoughts?

[@miriza2]

Same question!

[@aroojeapp5078]

Hello I am comparing gene expression between two groups diseased and controls. I have to compare gene expression of 7 genes between two groups. For each gene the CT values are different for each group, therefore it means I am not using same data for multiple comparisons.. do I still need to perform the correction test.. secondly the sample size is very small 8 controls and 16 diseased.

[@how2stats]

If I understand the nature of your data/design correctly, I'd say you need to consider some level of familywise error rate inflation correction, yes. Whether the Bonferroni correction is your best/only option, I don't know.

[@Dr.Khalafzai]

How can I cite you in regards to the unnecessity of using Bonferroni correction? Thank you.

[@how2stats]

Check out: blog.apastyle.org/apastyle/2011/10/how-to-create-a-reference-for-a-youtube-video.html (I'm Gignac, G. E.)

[@souhakabtni5192]

Very informative :) Thank you

[@ninjanj6148]

How exactly familywise error rate increases? What's the reason behind?

[@how2stats]

The ultimate reason is 'chance'. You simply need to accept that when you conduct statistical analyses on a sample of data, apparently statistically significant results can arise simply by chance (no systematic effect in the population).

[@ninjanj6148]

@@how2stats Thank you very much!

[@lauraschuck6772]

Guys, what do you think about my data: I compared 14 morphological measures in lizards of two different environments with student t-test... Ex: head width of forest lizards x head width of dunes lizards... I have to use the Bonferroni correction?

[@how2stats]

I'm guessing your sample size isn't big enough to create body size factor scores. If so, you might have to rest on the argument that your 14 morphological measures are substantially positively inter-correlated, which would imply that a Bonferroni correction would be way too strict (as discussed in the video).

[@lauraschuck6772]

@@how2stats Thank you very much

[@lauraschuck6772]

@@how2stats I have made 14 t-tests (one per measure)... It's ok?

[@hugh2861]

💢💢💢

[@abdullaalsuman7891]

ok