SEM Series Part 2: Data Screening

[@Gaskination]

Here's a fun pet project I've been working on: udreamed.com/. It is a dream analytics app. Here is the KZbin channel where we post a new video almost three times per week: kzbin.info/door/iujxblFduQz8V4xHjMzyzQ Also available on iOS: apps.apple.com/us/app/udreamed/id1054428074 And Android: play.google.com/store/apps/details?id=com.unconsciouscognitioninc.unconsciouscognition&hl=en Check it out! Thanks!

[@DarklordMan]

These are the best education videos I have ever seen.

[@mingminglein]

I really love your humor :D makes the whole SEM process less exhausting!

[@tongxin9923]

Dear Dr. James, I really appreciate your nice guidance, and I went to the web page that you suggested in the video, it is really helpful for me -- a Ph.D. student who is just getting started. Thanks a billion!

[@MrMusicalmateo]

These videos are ABSOLUTELY fabulous. Thank you for all this.

[@MMBariise]

WOW! Never learned so much in one clip than this video. Thank you very much.

[@DirtyPhlegm]

Wow this video is so entertaining. I'm fully engaged with this process

[@Gaskination]

If you mean deleting based on being an outlier, then you can cite any standard statistics textbook. They all say that if you can demonstrate that outliers are truly erroneous or anomalous, then they can be removed as "bad data".

[@Gaskination]

I'm not sure. Most of what I do is just what makes sense, not necessarily based directly on a journal article. I've never read an article that talked about unengaged responses (although I have one right now that is in second round of review).

[@jamie10157]

Thanks! Yes James, not the best situation! I think I will leave them to be on the safe side and keep my sample. This idea of 'unengaged respondents' hasn't been covered in the texts I use. Are there any good ones on this, or any journals? Perhaps 'unengaged responses' is not the only term...

[@jamie10157]

Hi James, in this video you seem to look at STDEV across ALL variables. Presumably the items composing these variables all have the same type of scale (e.g. 1-5) and thus you can compute STDEV for all these variables and look at unengaged responses? I have three different factors I intend to examine within the survey, each with different scales, so I guess I need to compute STDEV for each one and start removing individuals that unegaged for the different scales?

[@Gaskination]

certainly, if you have a lot of missing data. If not, just do a quick manual check like I do in this video.

[@Gaskination]

Not sure. Probably should do a find and replace for zeros to blanks.

[@Gaskination]

Hi Tianna, KZbin wouldn't let me reply directly, so I'll reply here. I have no reference for it. a 5-point scale can certainly be skewed, but the information skewness tells us is subsumed in the information Kurtosis tells us on a scale that small. With a 5-point scale, what we really want is sufficient variance to predict that variance. If it is manifesting high kurtosis, then we might not have enough variance. If there is high skewness, then it means we also have high kurtosis.

[@jamie10157]

Right okay thank again! I will have to make some decisions on this, and will see if there is any reference sources out there; if there are I will share!

[@jamie10157]

Hi James, great video thanks. Just wondering what basis you are saying 'justifying deletions', i.e. do you have a reference source for justifying the different deletions you did?

[@ChristianHigton]

Would you recommend doing a little's missing data analysis test to check your data is missing completely at random? A great series of videos by the way.

[@tiannal8132]

Hello! I was wondering why one doesn't need to look at skewness on a 5 point likert scale... Do you have a reference for that? Thank you :)

[@Gaskination]

correct. my scales are all on 5-point likert I think. you would need to do yours separately for each different size scale.

[@cenkbalkan8751]

Thanks for the videos, especially the new organized ones. I have a question about spss skewness and kurtosis values. For those values, (value)/(its std error) ratio is used in some references. And this ratio checked for +-2 or +-3 limits. Do you think Is it true? Thanks...

[@JEricB219]

James, thanks for all of your great videos. They are very helpful. I have a question about the way you test kurtosis in this video. You use the kurtosis stat, highlighting those with an absolute value greater than 2. Do you use the SPSS stat directly, or do you divide that stat by the kurtosis standard error? I thought you had to divide it by the SE.

[@mesfinmenza5244]

Thank you very much for sharing. Can I get the data that you used?

[@Gaskination]

Here is the updated playlist: kzbin.info/aero/PLnMJlbz3sefJaVv8rBL2_G85HoUko5I-- And here is the dataset: www.kolobkreations.com/KZbin%20SEM%20Series.sav

[@jamie10157]

I just wanted to add, regarding the issue of unengaged responses: I have a lot of people with STDEV around 0, or less then .5. But this may well be because of the likelihood to hit the ceiling of scoring. Not sure how best to deal with this, because I don't want to remove so many people (around 15%)...

[@Gaskination]

I'm not sure about a reference for +/-2 or 3. I just use that as a practical cutoff. I should probably publish something that uses all these practical cutoffs so that people can reference it... sorry.

[@supermarshmallow1996]

Question, whats the difference of checking missing variables and deleting blanks?

[@Gaskination]

You can check for missing values by column (variables) or by row (cases). It is usually better to impute missing values (replace them with known values or median/mean values) rather than deleting cases or variable that have missing values. Deleting cases reduces your sample size. Deleting variables limits your measurement ability and may undermine measurement validity for factors.

[@shueling]

Hi Dr. James, regarding handling missing data, there are recommendation for using ML method in which we check the "Estimate Means and Interecepts" box in Analysis Properties. However, by checking this, we are unable to obtain modification indices for Measurement Model later. Is there any way to obtain modification indices even when using ML method?

[@Gaskination]

***** Only when you don't have missing data. So, you can get around this by replacing missing values in SPSS with the median or mean.

[@muhammadzeshan2727]

Dear sir, whenever i encounter some problem in data analysis i start looking at your youtube page and i get a clue. so again i was checking the skewness and kurtosis of my data and found that one of the variable does have 3..359 kurtosis value. also share the paper that indicates that value of kurtosis differ from .2 to +2. one other question is in one of phd thesis i found that the author have showed the kurtosis values of latent variable instead of observable variable so what you suggest in this regard???

[@Gaskination]

The best reference for checking normality is just a textbook. One of my favorites is Hair et al 2010 Multivariate data analysis. As for latent variable normality, that can't be done in AMOS as far as I'm aware. You would have to first impute factor scores and then test the factor scores for normality.

[@nancyscott3008]

Hi Dr. Gaskin, This video was very helpful. Do you have a cite for handling unengaged responses? I've got the Lowry & Gaskin 2014 paper but don't see it in there. Thank you.

[@Gaskination]

+Nancy Scott No citation on that. It is just logic. Makes sense. No need for a citation when it just makes sense, or when it is simple math.

[@ahsanmba1]

Hi prof, Why should we check for outliers of variables measured on likert scale??Does it makes any sense??

[@Gaskination]

+Mohd Sadiq No, it doesn't make much sense. You can't really have an outlier on a Likert scale.

[@safalbatra4791]

Hi James..As per your demonstration, I have used SPSS to fill missing data, using median of nearby points. I have been asked to give the name of algorithm used and one citation why median filling method of imputing missing values is good. Can you plz tell me the algo (is it multiple regression approach?), and a citation, if you are aware. Will be great help. Thanks a lot

[@Gaskination]

The algorithm is just the median. Nothing fancy. You just took the median of the column and stuck it in the missing cells. That's the algorithm... As for a reference, just go to scholar.google. com and search for "median imputation" and you'll find plenty of good references.

[@jeromeflores6754]

Great video +James Gaskin . I saw the comments abot curtosis and I wonder if I have 12 items whit higth curtosis (+3000) I can use asymptotic distribution free (ADF) instead the usual ML. I have 800 respondent. Thanks a lot for your answer.

[@Gaskination]

+Jerome Flores I've never seen kurtosis values in the thousands before... My concern with that would be whether there is sufficient variance on the variable.

[@jeromeflores6754]

+James Gaskin sorry, I mean 3.0. the items are about 17%. Then I guess I can use ML, but I Wold like to know if the ADF can be better. Thanks again for your great videos.

[@Gaskination]

+Jerome Flores Oh. 3.0 isn't too bad. If it is more than 4.0, then I start getting worried.

[@jeromeflores6754]

+James Gaskin thanks! It's good to know.

[@nurulsharnizahusin34]

Hi Dr. James, What is the range accepted for stdev to check "Unengaged responses". If the value more than 1.000??? What shoud we do. Need your advise. Thank you in advance

[@Gaskination]

+Nurul Sharniza Husin There is not published range. The logic behind this test is to see if respondents answered with sufficient variance. If they answered with all 3, 3, 3, 3, 3, 3... then they were probably not engaged. I recommend looking over all of a participant's responses before deleting the row.

[@solanasalessi4059]

Hi James! I´ve got a doubt about data screenig in order to SEM analysis. I have to calculate skewness, and kurtosis for each item, for eache scale (and sub-scale) or for all of them? Thanks!

[@Gaskination]

+Solana Salessi For each item.

[@solanasalessi4059]

+James Gaskin Thanks James! Are this indication valid to data screening before an AFC and a structural model testing too? Or only for the first one? Regards!

[@Gaskination]

+Solana Salessi I'm not sure what an AFC is. Data screening should occur right at the beginning, before all other analyses.

[@arashhadadgar]

Hi James, very useful video! thanks. I just wonder, if I replace the missing values by mean/median, then it will cover the misfit items and it would be hard to distinguish bad questions? Am I right? Is removing missing values an assumption in SEM? Thanks

[@Gaskination]

If I have enough data, I usually just remove the cases with missing data. If I don't have enough data, I will impute/replace those missing values if there aren't too many of them. Replacing them does affect distribution of responses for the item, as well as correlations between that item and others.

[@caloycarlo9136]

Hi James, I am using a secondary data from U.S. with a total of 376,577 responses but 94, 948 did not complete the demographic section. What I am planning to do is that to drop all [94,948] then will do an EM algorithm method with the remaining samples who had missing data. Will this be acceptable? Thanks in advance.

[@Gaskination]

+Caloy Carlo Should be fine. With such a large sample size, make sure to focus on R-square and regression weights, instead of p-values.

[@caloycarlo9136]

Thanks a lot James.

[@sumitasarma6187]

Hi James, am unable to copy the SPSS data (opened in univ's remote labs) to an excel file on my laptop... Is there a way? Thanks

[@Gaskination]

Not that I know of. It must be an issue of using the remote lab. Sorry about that.

[@alexcoetzee3411]

Hi James. This video was very helpful thank you. My question is as follows: I have 39 variables which are part of their own respective constructs. Each variable has large numbers of missing values on their 7 point likerts, some up to 40 missing values out of a database of 706 respondents. Can I replace them with the medians? I think you mentioned that roughly 10% missing values is a bit worrying, therefore will I be ok to replace with the medians for such a large number of missing values?

[@Gaskination]

With such a large sample size, I would simply delete those records. You would be deleting roughly 5% of the dataset, which is not uncommon or unreasonable. Deleting much more than that would require you conduct some comparisons between those you deleted and those you kept. I don't think I would impute so much though.

[@大貓子]

Hi, Dr. Gaskin, the Kurtosis value of one of the items is 216.928. Can I consider this is an outlier? Should I delete it befor going to EFA?

[@Gaskination]

Wow! That's impressive. I've never heard of one that high! I would recommend doing a histogram of it to see how the distribution looks. But yes, this is probably a major issue with that variable, and so it will likely need to be dropped.

[@大貓子]

Dear Dr. Gaskin, well, I just did the histogram and it showed that about 75% data answered 3 to 5 (it is a five-point likert-type questionaire). This is the reason why the Kurtosis value so high? What does that mean if the value is so high? Thanks.

[@Gaskination]

Sabrina Chan Hmmm. that doesn't make sense. If they are distributed across 3/5 values, then you should not have such high kurtosis. Maybe it is being affected by an odd sample size...? Or maybe you are looking at the wrong number?

[@大貓子]

James Gaskin Dear Gaskin, I re-do the test again and still got the same value. And I checked all the data in this item again but I cant find any odd number. I also did a histogram and I cant really understand why this happened. The sample size is 421 and the Kurtosis value is good for other items. I find that after I delete this item, the pattern matrix looked a bit cleaner. So can I just delete this item?

[@Gaskination]

Sabrina Chan That is very bizarre. Yes, you can just delete the item. I really can't imagine why it is giving you this funny value unless 95% of all responses are the same (but you said they weren't).

[@ahsanmba1]

Hi, why we replace missing values with median for a variable measured on likert scale??Why not with mean??

[@Gaskination]

+Mohd Sadiq We use the median because the scale is ordinal instead of continuous. This means that only integer values have real meaning. We would use the mean on continuous scales.

[@ahsanmba1]

+James Gaskin :Thanks for the reply but then in many cases we treat it as continuous. we calculate SD and mean also. We apply many statistical techniques also by assuming it so.

[@adeliawahyu126]

Hello, i'm doing a research with the likert scale. and my question is do i have to tranform it to interval data first for doing sem analysis in amos? bcs i've read some papers that discussed about likert-scale as ordinal data and that it's not really suitable for sem analysis. Really appreciate your help. Thank you in advance!

[@Gaskination]

Likert is ordinal and is fine for SEM.

[@Gaskination]

how do we do what?

[@abbassyedgohar3824]

Hello James ... Your videos helped me a lot indeed ... I have some querries pertaining to conditions for linear regression 1- Why we take STANDARDIZED RESIDUALS instead of actual values while conducting Normality Tests, QQ plots, Histograms, Scatter Plots, Skewness and Kurtosis etc? 2- I have seen somewhere that the above mentioned tests e.g. Normality Tests, QQ plots, Histograms, Scatter Plots, Skewness and Kurtosis etc are conducted only for Dependent Variables .. is has created a confusion as why not for the Independent variables? .. In other words we check the normality of the data for the Dependent Variables only? is it true?

[@Gaskination]

1. The standardized residuals show the consistency of error, or the evenness of distribution of points around the line of optimal fit (the regression line). 2. I just submitted a manuscript about this very thing. We found that when normalizing the dependent variable, you increase predictiveness (R2), but if the predictors are not normalized as well (if there are distribution problems like skewness, etc.), then the model fit of the resulting model is hurt. By far, all our tests showed that having normally distributed variables in all parts of the model resulted in the strongest model. However, the 2nd strongest model was always the one with the IVs and Mediators normalized, but not the DV.

[@abbassyedgohar3824]

James Gaskin Many thanks :) .... Do you have any tutorials in which we can do Normality Tests, QQ plots, Histograms, Scatter Plots, Skewness and Kurtosis for Independent Variables and Moderators? I guess its not the same way as we usually do for Dependent Variables? 2ndly, the normality tests (e.g. Kolmogorove-Smi and Shapiro Wilk), Skewness, Kurtosis etc should be conducted on indvidual items (e.g. WorkOverload Q1, WorkOverload Q2, WorkOverload Q3) or their mean? I wonder why people conduct normality tests on dozens of their items rather than their means?

[@Gaskination]

Abbas SyedGohar 1. Here is a tranformation/normalization video (intended for continuous data): kzbin.info/www/bejne/qqjahWl8nNyfd9E. Likert-scale variables are not really in need of normalization, but can be problematic (and unusable) if highly kurtote (peaked). 2. I conduct these tests on individual items, not their means. The individual items must be normal, or else the latent factor is unstable.

[@monikager2966]

Hello, First of all let me start by saying that this video has helped me tremendously, however I was just wondering if maybe you had a reference for engaged responses part? My supervisor wants it and it is kind of awkward to say youtube... Thank you very much in advance!

[@Gaskination]

Determining if a participant was engaged is simply a matter of logic. If they answered the same value all the way across, then they weren't paying attention. Same if they answered reverse-coded questions the same way as positive-coded questions. Or if they failed attention traps. It is just logical. Not sure if there is a reference for it.

[@benali8952]

Please Dr . How to get data file used ? Because I need to follw you in your movies and analysis data with two softwares SPSS & AMOS .

[@Gaskination]

It's on the homepage of the StatWiki.

[@benali8952]

which kind of file , is it excel file ? please could you send to me file Dr ?my email is [e-mail]

[@Gaskination]

.sav (for SPSS). I'll email you.

[@benali8952]

thank you so much Dr , I got it

[@sarvnazbaradarani6665]

Dear James, the videos are fantastic, I learned a lot from them. I had a question about the data that I have, in my data I have 7-point Likert scales and 5-point Likert-scales. So, is it possible to do the exact same things that you mentioned about the skewness and kurtosis or I have to do them differently? Thanks :)

[@Gaskination]

Use Kurtosis for Likert Scales. Usually an absolute value for kurtosis above 2.2 is potentially problematic. I think it was Sposito et al. 1983 who says this: (Sposito, G., Holtzclaw, K. M., Charlet, L., Jouany, C., & Page, A. 1983. Sodium-calcium and sodium-magnesium exchange on Wyoming bentonite in perchlorate and chloride background ionic media. Soil Science Society of America Journal, 47(1): 51-56.)

[@sarvnazbaradarani6665]

James Gaskin Thank you

[@sailormark91]

James Gaskin For absolute values for kurtosis above 2.2, should we remove those items when moving to the next steps? One of my values is 4.4 and I'm thinking about removing it. Thank you for the videos!

[@Gaskination]

sailormark91 4.4 is high, but Kline says that you can get as high as 10... Personally I think anything greater than about 3.5 is too high. My rule is to examine whether their are multiple points of evidence to get rid of an item. If so, then you are justified in removing it. If not, then I would try with and without the item to see if results are sensitive to removing it. All of this is difficult if you only have 3-4 items on the factor. But if you have 5+ items on a factor, then perhaps just remove it.

[@sailormark91]

James Gaskin I decided to remove them.. my scale has 38 items (?!) so it is not a problem.. the only problem is that i realized it is a second-order variable. In my model,I have 5 variables and 2 of them are 2nd order. Should i do an EFA for the ones who aren't and move to a CFA then? If so, do i still copy the table from EFA containing all the variables in AMOS?

[@michaelaristian1368]

thaks for the video, it helps me alot. im new to statistic and have some question. 1. is measure ordinal or scale realy make a different? i see some tutorial for likert scale they use ordinal. 2. if i have some parameter which all same response for all response, or have pearson corelation 1. is it fine just to delete that variable? 3. for the missing variable when we use multiple imputation for them? or just all go with median for likert scale? Thankyouu!!

[@Gaskination]

+michael aristian 1. In SPSS, ordinal is for Likert scales. Scale is for continuous variables, like age, income, etc. 2. yes, delete the variable because it does not vary. 3. For likert scales, I would use median.

[@michaelaristian1368]

James Gaskin thankyou very much. It's verry helpfull.

[@riadincfcb]

Hello James, thank you very much for your videos, can I ask you 2 small questions: 1- can I use the median for missing data for controls variables (I have age, gender.. etc all in categories)? if not does it mean I will lose all cases that does have 1 or more missing data for control variable? 2- can you give me some references so I can back up using median for my analysis? again Thanks you

[@Gaskination]

Riad Cheikh 1. You can use Median. Hopefully you are not missing more than 5% of the data... 2. You don't need a reference for using median. Just use logic. Also, when data is skewed, median is more accurate.

[@riadincfcb]

James Gaskin is median good for categorical control variables such ages and religion?!

[@Gaskination]

Riad Cheikh Definitely not. You cannot impute religion. If age is collected in buckets (e.g., 20-25, 26-30), then it is also tricky to impute missing data.

[@riadincfcb]

James Gaskin So this will make ask the same question, if i can't impute missing data for a control variable, does it mean that i ll lose all cases that has at least one control variable (age, gender, religion..etc)? Thanks for your answer, really appreciate it

[@riadincfcb]

James Gaskin all my controls variables are categorical (age, 20-25, 26-30..etc)

[@Gaskination]

Some divide it by standard error. I just use the absolute stat when using Likert 5-point scales.

[@fundementalroast7002]

How can I run the first part in excel if an entire row is blank ?

[@Gaskination]

Delete the row. It is not adding any value.

[@danieldare25]

Dear James, your videos are so educative, please keep it up.I have a question that has to do with copying my data from spss data view to excel. Some of them show (0) Zeros. What can i do please.

[@Gaskination]

Daniel Dare If they weren't zeros before, then this is because SPSS didn't load those values before copying. SPSS runs a bit differently than Excel, and some slower computers (or Macs running Windows) take a lot of time to copy. So, I would copy and then wait for at least ten seconds, then paste in Excel.

[@danieldare25]

I will try it now. Thanks for your support. God will increase your wisdom IJN Amen.

[@danieldare25]

Can we be discussing by email?. I'm currently a PhD student.

[@Gaskination]

Daniel Dare You can email me, but I might be slow to respond, as my health is very poor this year. [e-mail]

[@danieldare25]

ouch! what happened? please stay strong. I commit you to God's healing hands. You are healed In Jesus Name!

[@shamsunnahar9590]

Excellent vedio. Thanks.

[@Gaskination]

Yikes! You'll just have to decide whether those represent true or unengaged responses.

[@farrukhiqbal7646]

Can i have this data file?

[@Gaskination]

+FARRUKH IQBAL this dataset is available on the homepage of my statwiki (statwiki.kolobkreations.com)

[@larryauyeung2179]

+James Gaskin I have searched the website, but cant find the data set anywhere.... may I know where the data set can be downloaded ?

[@larryauyeung2179]

+James Gaskin oh...... I found it ! Thanks! :)

[@piun6705]

hi james, I am somehow unable to view the file from statwiki, it says that side-by-side configuration is incorrect. i have SPSS17.0. Is it possible for you to email it to me if I write to you? many thanks..

[@Gaskination]

Which file? I have lots of files on StatWiki.

[@rahibali4377]

Excellent

[@raj-ms4qg]

Thanks a lot for the video series Dr Gaskin. I need some advice regarding the model and statistical process. Can , I get you email address?

[@Gaskination]

+raj Saha I would prefer you do your questions here if possible. If not possible, then please email me very concisely. I receive about two dozen requests each day, so it is very nice when the emails are short and to the point and ask specific questions, and provide a picture of a model (if relevant). [e-mail]

[@nikolajovanovic4259]

Why are u counting education only for college and above? Primary school and secondary is also education... So if someone is 30 yo and finished primary, secondary, high, college, etc.... it is like 23 years of education...

[@Gaskination]

In this dataset, my population is college students. So, including their primary and secondary school would be the same as adding 12 to every number. This would not change any of the analyses. If instead I was sampling a population outside of the university, then using all education would be more relevant.

[@aaryaaaaaaaaa]

Dear sir, I love your presentation and request you to share beginning to end analysis on the basis of likert scale . This will be your great support and also share your mail i.d

[@aaryaaaaaaaaa]

Very nice and easy to understand.

[@Gaskination]

Everything I learn I put in these many videos. So far there are 165 videos on the channel about SEM. I don't have any specifically about Likert Scales, although I use them in most of my work. Best of luck to you.

[@josephinelawal2722]

I just wish you took your time to explain, everything was rushed

[@dejal.3606]

One cannot measure Playfulness by Scale, it is nonsense.