Testing normality is pointless. Do this instead

Do you want more structured and personalized information? Come take a class with me! Visit simplistics.net and sign up for self-guided or live classes.
Video about diagnostics: • Diagnostics: What to l...
Video about robustness: • Robustness in Statistics
And here's the paper (and dataset) I referenced in the video: journals.plos....

[@Saynotoclipontiescch]

In twenty five years as a psychologist, I have never tested assumption of normality. Now I know I was right not to.

[@yulia6354]

as a russian person I think you nailed the russian accent! Well done :D and thanks for your videos! As a medical doctor and a big fan of statistics I really love your way of teaching people complicated stuff)

[@QuantPsych]

High praise from a native :)

[@billyboy1997]

I found a new hidden gem channel! Nice video.

[@killamaniac08]

Hey man, this was a really interesting video. In my master's forever ago they never explicitly mentioned this idea but rather implied it the language used to evaluate models. Namely, using tests at the introductory stages to later saying how robust a model is to deviations of the underlying assumptions. Also I'm a huge fan of you're emphasis on diagnostics. The first few times in industry I encountered some bespoke model my company had been paying for I was greeted with all shoulders from management and customer services for the model providers when I asked for model diagnostics to be included. Drove me nuts.

[@pipertripp]

And if Kolmogorov-Smirnov says your residuals are not normally distributed, it's big trouble for moose and squirrel!

[@QuantPsych]

?

[@pipertripp]

@@QuantPsych Borris and Natasha from Rocky and Bullwinkle. Good old fashioned cold war stuff.

[@Tascioni49]

Super useful, especially in ecology, because I rarely get normal data from my field experiments. And when I do, is usually because something went wrong 😆

[@RichmondDarko-qo2me]

Thank you very much for such informative videos. I spent several years in class and didn't understand all these concepts, but watching this video has made things easier for my comprehension. I have a few questions I would like to ask: When performing a statistical test, we use a parametric test if the data or variable in question is normally distributed, and a non-parametric alternative if the data or variable is not normally distributed. My question is: when does the central limit theorem come into play here? Also, a colleague of mine told me to always use parametric tests even if the data is not normally distributed. His explanation was that parametric tests are more powerful than non-parametric tests. So, should I straightforwardly use the non-parametric alternative when I observe that my data is not normally distributed, or should I take the CLT into consideration and use the parametric test?

[@QuantPsych]

Central limit theorem makes linear models very robust to violations of normality. That means your inferences will probably be sound (i.e., p-values and confidence intervals will be fairly accurate). But, inference is just *one* thing I'm trying to do with stats; I also want to accurately model the data. If the distribution isn't normal, I shouldn't assume a normal distribution. I instead use generalized linear models (not non-parametric tests). Your colleague is wrong. They're only more powerful if you meet the assumptions. But your colleague is right--use parametric models (but the parametric may be a negative binomial regression rather than a typical regression).

[@Eloss69]

Out of the topic but the video makes me think of it : why do we use Pearson correlation when modeling data ? Why not Kendall measure or even better, use Copulas ? Using Pearson looks to me like you know nothing about your variables interactions but you want to measure their linear interaction … you will obtain something but is it a useful information ?

[@galenseilis5971]

Whether some piece of math is useful depends on what you want/need to know combined with what constraints you are working under.

[@perfectmoments3876]

I went on to telling my students to run hist(rnorm(n, 0, 1)) a few times with n being their sample size, to get a feeling of what would all be totally fine samples of normal distributions. If their residuals (lm) or samples (t.test) look like they would fit in there, they're good. What do you think of this approach?

[@QuantPsych]

I'm actually developing a package to do basically what you're suggesting

[@TheHeadincharge]

I’ve always wondered why we don’t look at effect size when running these tests at least to make them slightly more useful. Although, I would argue that is true for all parametric tests. Turkey’s quote about parametric tests has always been my favorite to help me understand this interpretation properly. Great video though, normality testing is truly the most misunderstood concept by most psychologists in my experience.

[@galenseilis5971]

In principle you should be thinking about effective sample sizes if you are ever performing a null hypothesis significance test (NHST). In practice people doing NHST often don't know to do it or don't care to do it or don't know how to do it.

[@galenseilis5971]

I'm sure Tukey is rolling in his grave knowing he is now referred to as "Turkey". 😉

[@mohammedyounes98]

I read in a book that p-values such as 0.25, 0.5, 0.75, and 0.95 lack a rigorous scientific foundation and are largely arbitrary and we’re using it just because some guy said so over 50 years ago.

[@QuantPsych]

yup

[@igorbione4796]

Oh my, this video would save me a lot of work if I checked earlier! Thanks!

[@dimitrioskioroglou4316]

You're actually telling me to overlook the p-values and use my brain to... think? Come on!

[@QuantPsych]

Weird, eh?

[@dimitrioskioroglou4316]

@@QuantPsych Well, I cannot tell! The H0 says that it's not weird, so I need to test against it.

[@AC-go1tp]

I learned something today. Thank you. But too much comedyto the point that it is distracting.

[@ndrmkhn6559]

As Russian I may say your "Russian" pronunciation is adapted from the Snatch or similar quality spy series.

[@QuantPsych]

So you're saying it's perfect? ;)

[@idodlek]

Hello Mr. Fife 😀 Does, for example, running general linear model as t-test versus mann-whitney u test and comparing theirs results count as sensitivity analysis? Or only transformations, bootstraping and trimming would count as sensitivity analysis?

[@QuantPsych]

Yes, that could count a sensitivity analysis. I do wonder though if you might run into a situation where MW and t-tests agree, but modern robust methods would disagree.

[@galenseilis5971]

If you want your model to be as correct as possible, then you should aim for your model to do a good job of predicting the data distribution. Predicting the conditional expectation is a pretty rough approximation, especially with data sets like this where it is apparent that most of what is going on is not compressed well by a line.

[@deyvismejia7529]

Why do I feel personally attacked lol I like to test assumptions but great video!!

[@TheJucuska10]

Thanks for the video, it was great! You can also do one about the independence, because I had problems with it in my last rejected manuscript ;)

[@QuantPsych]

You can see my videos on mixed models. My introductory to mixed models video talks about it.

[@TheJucuska10]

@@QuantPsych thank you, I'll check it!

[@jishanzaman3421]

I've already imagined that one day you'll make a video on this topic...now I got that..thank u so much❤

[@QuantPsych]

😊

[@galenseilis5971]

I don't see a link in the description to the data set. 🐕

[@QuantPsych]

Ah! Thanks for the reminder. It's there now.

[@naftalibendavid]

The more power you have, the more power you have to show that your data aren't normal. GREAT! (But maybe a non-parametric...) What is a "meaningful" departure from normality? I don't know...is it big enough to make my real Type I error rate larger than my nominal alpha? Is it so far from normality that my power takes a beating?

[@QuantPsych]

Yes, these are great questions! None of them can be answered by a statistical test.

[@samj.vizcaino-vickers8512]

@Quant Psych Where's the paper? :c

[@QuantPsych]

Yes, thanks for the reminder. It's there now.

[@Nyonyokki]

I'd love to follow your steps in R but flexplot is not compatible with my R version 4.3.2. Which version do you use?

[@QuantPsych]

It should be compatible. Are you installing from github?

[@Nyonyokki]

@@QuantPsych Ahh, thanks for the hint! And also thanks for sharing your absolutely enjoyable humor!

[@hamidjess]

Dude, with all my respect to the depth of the content, could you please do accents more frequently?

[@QuantPsych]

Ha! I've had two entitled Karens tell me I need to change my approach to videos and stop doing accents. But, i side with your preferences :)

[@nizogos]

Doesn't normality imply t distribution of the models coefficients? Which in turn is necessary for their confidence intervals and all other things?If we're using the model merely for its output when we plug in the x values then yeah,normality doesn't play a huge role,but that's not how models are used to my understanding,at least that's not a scientific way to report on data.

[@QuantPsych]

Even if we're using confidence intervals, having non-normal data doesn't usually screw up our estimates of the confidence intervals.

[@nizogos]

@@QuantPsych is there a mathematical proof of that or is it heuristic from years of experience?

[@zimmejoc]

So is all this proving that our model is robust to violations of the normality assumption? That class was back in 1995 and my professor said we assume normality, independence, and one other thing, but that if we violated one of those assumptions it wasn’t a big deal because our test was robust to those violations.

[@QuantPsych]

There's two different issues: 1. robustness and 2. informativeness of tests. This video is about #2. Tests of assumptions are not informative. Robustness, on the other hand, is a different issue. Most models are very robust to normality violations, fairly robust to homoscedasticity violations, and not at all robust to independence or linearity.

[@idodlek]

​@@QuantPsych Could you please tell which models are most robust from normality and which are fairly robust to heteroskedasticity?

[@naftalibendavid]

@@idodlek It would depend upon which assumption you violate and in what direction and how severely, but robust alternatives (permutation, winsorized means, M-estimators, percentage-bend correlations) are your best friend. Check out Rand Wilcox's work. Like everything in Stats, it depends...

[@danhallatt4954]

Second (but, more accurately third ;) )

[@NikolaiKaverin]

I have to say that your Russian accent is pretty good

[@QuantPsych]

Many thanks, comrade.

[@djangoworldwide7925]

I want a version of your videos without the stupid comments. Instead of a 5 mins vid it became 20

[@QuantPsych]

I shall change my entire approach and structure to making videos to accommodate your preferences.

[@batesthommie2660]

Hahahahaha Good One

[@galenseilis5971]

If you want cut-and-dry technical descriptions, then I recommend you read mathematical stats papers. You will find the concision and lack of humour you are searching for there.

[@bmebri1]

First

[@QuantPsych]

Technically, I saw it before you did ;)

[@SkepsisUtrechtWG2]

Please, please don’t do silly voices, or other clown stuff. Its terribly annoying, and one reason I unsubscibed.

[@QuantPsych]

That's probably for the best. I am who I am, I do what I do.

[@galenseilis5971]

KZbin tries to match creators with audiences, but it doesn't always find good matches. I hope you find something else you enjoy watching. I'm partly here for the silly voices.

[@excelfanboy_]

This dude is just yapping, don't waste your time

[@QuantPsych]

Seriously. This guy's an idiot.

[@galenseilis5971]

I see it very differently. Fife has identified substantial problems with how statistical analysis is conducted and he has dedicated a lot of time, attention, and energy into helping address those problems. For all my commentary disagreeing with him on this channel, he and I are mostly on the same team: statistical practice must get better.