Multigroup CFA in Mplus: Automated Invariance Testing

[@IMold363]

Fantastic video! Is there a step you can take after this to test for strict/residual invariance?

[@QuantFish]

Thank you! Yes, you can strict for strict MI (equal residual/error variances) but this has to be done manually in the current Mplus version. See my video here: kzbin.info/www/bejne/enekomiNiZ6cb80 Best, Christian Geiser

[@chriskowalski4376]

Hi Dr. Geiser, This is a great video! Thank you for everything you do! My indicators have 4 response categories, so I'd like to use the WLSMV estimator. Is there a way to do it using this method of Equivalence testing?

[@QuantFish]

Yes, you can use ANALYSIS: MODEL = CONFIGURAL SCALAR; with categorical (ordinal) indicator variables. Best, Christian Geiser

[@CoryCobbthaProf]

Thanks for the video. What about when the first part of the output for the configural, metric, and scalar models have significant chi square values but the comparison models (e.g., configural vs metric, metric vs scalar, etc.) do not?

[@QuantFish]

When the less restricted model does not fit according to the chi-square model fit test, strictly speaking, chi-square difference tests relative to other models should not be conducted. For example, when the configural model is rejected according to the chi-square test, it doesn't make a lot of sense to compare it to the metric model (which is even more restricted) using a chi-square difference test . Best, Christian Geiser

[@hsienfushih]

Hi Dr. Geiser! Thank you for all these useful videos on SEM! My advisor and I would like to take a closer look at the Mplus defaults of this Multigroup CFA Automated Invariance Testing. Where can I find more information about this? Thank you!

[@QuantFish]

See the Mplus User's Guide (statmodel.com/download/usersguide/MplusUserGuideVer_8.pdf), section starting on p. 540. Best, Christian Geiser

[@hsienfushih]

@@QuantFish Thank you very much!

[@KaitlynStormes]

Dr. Geiser, thank you for sharing this very informative video. I have a few quick questions for you after reviewing your materials: 1. Do you need a minimum number of cases per group to do the testing? I read the minimum is 200 cases per group, but I see you have fewer (i.e., 179 in your control group). 2. I want to see if my measure holds across (a) gender, (b) race, (c) intersections of gender and race, and (d) across three survey time points. None of these variables are dichotomous-- as even gender has three levels. Would I need to dichotomize these variables and test one at a time, or could I use a grouping variable with more than two categories? 3. If one's chi-square p-values are significant, I assume that means you do have measure variance. Would you then need to make separate factors for each grouping variable OR try again to make a factor that does not vary by group?

[@QuantFish]

Hi Kaitlyn, Thank you for watching my video! Below are my answers to your questions: 1. The sample size question is difficult to answer in general because it depends on many factors including model structure, effect sizes, and number and quality of the indicators. Simulation is a good way to examine sample size requirements for a specific application. 2. Multigroup analysis is not limited to the case of only two groups. You can use grouping variables with > 2 categories. 3. If you find measurement non-equivalence across groups, that could be an interesting finding in and of itself but it is difficult to say in general. One would have to see your specific case/results. Best, Christian Geiser

[@KaitlynStormes]

Thank you. Your responses were very helpful!@@QuantFish

[@suleymandemirgul5975]

Hi Dr Geiser i have a question when I compare three model to each other configure metric and scalar they are all significant but I have 3000 sample size which means I can ignore this statistics and go on to the CFI TLI and RMSEA so when I go there they are mostly 0.80 in this case as far as I know I do not have to focus on whether they fit percent or not I need to focus on the CFI and Tli differences between metrics if they are below 0.010 and RMSEA 0.015 i hope you understand. Or as Chi square is significant and chi tli are all below 0.90 should I claim that I can not do comparison

[@QuantFish]

CFI and TLI values below 0.90 point to poor absolute fit. Below 0.80 would be really bad. I would address that first to make sure you have a well-fitting comparison (configural) model. Otherwise, the comparisons wouldn't be very meaningful. Best, Christian Geiser

[@suleymandemirgul5975]

@@QuantFish CFI 850 tTLI 874 in you case I can not make group comparison right now?

[@QuantFish]

@@suleymandemirgul5975 CFI = 0.85 and TLI = 0.874 would indicate extremely poor fit. I would not use the model for anything. Best, Christian Geiser

[@gertlang4316]

Thanks, I was not aware of this very useful analysis option in Mplus. I still work with version 7 were this is probably not available but I know that this kind of analysis options are given in JASP software on the basis of the r-package lavaan.

[@QuantFish]

Hi Gert, Good to know! Best, Christian Geiser

[@fmunozt]

Dear Christian, (Sorry for my English) I have seen your video about the function MODEL = CONFIGURAL METRIC SCALAR. It works perfect, thanks. However, it does not work for models with second order factors. Do you recommend any video, book, from where I can see the syntax, to perform a factor invariance analysis, but in the "old" way? That allows to measure the invariance (configural, metric and scalar). Thank you very much for your videos. I have learned a lot. Best regards. Flavio.

[@QuantFish]

Hi Flavio, Thank you for watching! Have you checked out the following document? It appears to be pretty comprehensive with regard to the Mplus specification of multigroup analysis/manual invariance testing: blogs.lse.ac.uk/lcat/files/2015/08/LCAT_Mplus_and_R.pdf Best, Christian Geiser

[@KaitlynStormes]

Hi Dr. Gesier or QuantFish reps, What does it mean if the output says I have configural invariance (p < .05), but do not have metric or scalar (each p>.05), or that I just do not have metric (p>.05)?

[@QuantFish]

Hi Kaitlyn, A small p value for a difference test (e.g., p < .05) means the more restrictive (more constrained) model fits worse than the less restrictive model. For example, metric invariance is more constrained (and may fit worse) than configural invariance. When the metric invariance model fits worse than the configural model, this may indicate that some or all of the factor loadings differ across some or all of the groups. Some or all of the items/measures may function differently across groups (indicating at least partial measurement non-equivalence). Best, Christian Geiser

[@KaitlynStormes]

Thank you, Christian. Is it possible I don't have configural because my latent facor only has 3 items? I'm wondering if Mplus can assess configural on such a small latent factor. @@QuantFish

[@QuantFish]

@@KaitlynStormes Hi Kaitlyn, I'm not sure I understand the issue(s) that you are facing. If you don't have configural invariance, then you would not even test metric or scalar invariance. In case you are interested, I do offer personal consulting packages (christiangeiser.com/consulting). Best, Christian Geiser

[@KaitlynStormes]

Hi again, Christian; my apologies for the confusion. I'm new to this, but what I meant to say is that my output suggests I have configural invariance (p < .05) but do not have metric (p > .05) and, in some cases, do not have metric or scalar invariance (both p > .05). I heard that having configural invariance but not the later more strict options might be because of my use of a small, 3-item factor.@@QuantFish

[@QuantFish]

@@KaitlynStormes Hi Kaitlyn, If your model is a single factor model with just three indicators, then the configural model would be saturated (zero degrees of freedom) and would trivially fit the data perfectly. The metric and scalar models would be overidentified (non-saturated) even with just 3 indicators as they include equality constraints on factor loadings and/or intercepts across groups. Hence, only the metric and scalar models could actually be tested against the data in this case. I'm not sure about your p-value interpretation. A small p value (e.g., p < .05) for a chi-square difference test suggests that the more constrained model DOES NOT fit. Best, Christian Geiser

[@healthenvy3490]

Thank you Christian. Is the function of 'Analysis: Model=configural metric scalar' from recent Mplus versions?

[@QuantFish]

Yes, I believe it has been added recently.

[@healthenvy3490]

@@QuantFish thank you so much! That’s good to know ☺️

[@gabgabgab07]

Thank you so much for this great video. How would you report the comparison results between the models? Would this be acceptable Δ χ² = 1.215, p = .545 (for metric against configural) ? Thanks ! :)

[@QuantFish]

I would include the degrees of freedom (df) in parentheses behind Δ χ² before the = sign. For example, if the df for the difference test were 4, I would write: Δ χ²(4) = 1.215, p = .545. Best, Christian Geiser

[@gabgabgab07]

@@QuantFish Thank you Christian ! You're the best !

[@gabgabgab07]

Dear Christian, do you know if automated invariance testing can be done with other indices p.ex CFI ? From the output, only the Chi-Square is available. To your knowledge is there a way to add other indices ? Thanks !

[@QuantFish]

@@gabgabgab07 You would probably have to calculate the differences in fit statistics by hand. Best, Christian Geiser

[@fmunozt]

Lo siento. Thank you Christian. This is valid for ordinal variables and grouping with more of 2 category? For example: GROUPING = G (1 = Chile 2 = Argentina 3 = Alemania);

[@QuantFish]

Hi Flavio, You can definitely use more than two groups in multigroup analysis in Mplus. It appears that you can use MODEL = CONFIGURAL METRIC SCALAR also with ordinal outcome variables, except that METRIC is not available for binary (dichotomous) outcomes. Best, Christian Geiser

[@rtyqwe3812]

Thank you Christian! It is very helpful! But when I used second-order factors, it happened:*** ERROR in ANALYSIS command Measurement invariance testing with the MODEL option of the ANALYSIS command is not allowed when the model contains second-order factors. MODEL=CONFIGURAL is not allowed for this analysis.

[@rtyqwe3812]

How to use the command in second-order factors😭?

[@QuantFish]

Thanks for watching! In a second-order CFA, you may have to test MI by specifying the relevant equality constraints manually. Best, Christian Geiser