SmartPLS 4: MICOM Measurement invariance and MGA Multigroup Analysis

[@ahmadonly]

Hi James, you mentioned about making another video for achieving partial invariance by removing the items of the constructs. Have you made such video ?

[@Gaskination]

I did not end up making that video. Sorry about that.

[@AyaKasber]

Thank for the video! do we consider the confidence intervals as measure of significance in this case, or is it only the p value that we look at? if the confidence interval includes 0 what does this indicate?

[@Gaskination]

They can tell the same story. If the confidence intervals include zero, then it is considered "not significant" at that confidence level.

[@AyaKasber]

Thank you! 🙏🏻i would like to ask you about something regarding the MGA, If I run the bootstrap MGA, do I need to check MICOM first? And what is the difference between these two methods when comparing only two groups? Also if MICOM output is not good, can i rely on the bootstrap MGA significance or is it a must to improve the MICOM? Appreciate your response

[@AyaKasber]

In my permutation output, there are some significant Ps, with confidence intervals including 0.. in this case the reviewer commented that they are insignificant because of the confidence intervals including 0

[@Gaskination]

@@AyaKasber 1. MICOM is for measurement invariance, and we want to see no difference (i.e., invariance). MGA is for structural invariance, and usually we theorize that there is a difference (i.e., non-invariance). 2. If MICOM shows a difference (i.e., significant p-values), then we must observe at least "partial invariance", which is that each factor has at least one non-constrained indicator path that is invariant across groups. If this is not the case, then we cannot rely on the MGA results related to that factor. 3. Correct, if the confidence interval includes zero, then it is not significant at that level of confidence. A significant p value with a zero in the confidence interval can happen under two circumstances: the alpha threshold (acceptable p-value) does not match the confidence level (span of confidence intervals) - i.e., accepting p-value of up to 0.100 while looking at 95% confidence intervals. OR, the p-value is examined for the original sample, while the confidence intervals are examined for the bootstrap sample mean.

[@AyaKasber]

@@Gaskination thank you so much

[@maralguero2504]

Hi James, is there any kind of article where this method is applied that you can point to?

[@Gaskination]

www.smartpls.com/documentation/algorithms-and-techniques/micom

[@lauraromero8115]

Hi professor, greetings from the Canary Islands! First, thank you very much for your videos, they are really helpful. I have an inquiry, hopefully you can help. I have a PLS-SEM model with three independent variables (higher-order, reflective formative) and one dependent variable (reflective reflective). I ran both permutation test and Henseler's MGA (the grouping variable was a 1-0 dummy), and some differences between groups were found. Nice. Then, a reviewer asked me to additionally control for another variable (this was a continuous variable -age-, so I created five dummy variables from it, one per age interval). But SmartPLS4 doesn't allow me to run the mulitgroup analysis when connecting the control variables to the dependent variables. I get the singular matrix error. Do you have any suggestions on how to solve this issue? I'll be very grateful for your feedback. Best regards.

[@Gaskination]

Either drop a reference category (just the youngest age group) or include the age variable as it was measured, rather than as dummy variables (since it is probably close to ordinal).

[@anisalshargabi1244]

Thanks James! What if I have more than two groups and I want to see the differences between them? Additionally, what if I have several sets of grouping, e.g. "Gender groups", "Job level groups", "Organizational unit group", etc.? thanks for your kind help.

[@Gaskination]

SmartPLS let's you compare more than two groups at a time. Just check more than two groups when selecting comparisons in the analysis options. As for different types of groups, just select the groups you want to compare each time, and deselect the irrelevant groups.

[@anisalshargabi1244]

Hi @@Gaskination under "Permutation Multigroup Analysis" in SmartPLS, there are only two options "Group A" and "Group B". So, two groups only to be compared at a time. However, under the "bootstrap MGA", it is possible to compare more than two groups at a time. Is my understanding correct? Thanks

[@Gaskination]

@@anisalshargabi1244 Correct.

[@SAIDETTIS]

Thanks Dr ... is it better to use consistent permutation rather than normal... seeing the model is reflective?

[@SAIDETTIS]

In a newer version of SmartPLS 4, there is consistent permutation

[@Gaskination]

@@SAIDETTIS Yes, if the consistent algorithms work for you, then they are better when all factors are reflective. Sometimes these algorithms fail in SmartPLS, so I just used the regular algorithms in this video.

[@구태봉]

Hello, I enjoyed the video I have a few questions. It's an emergency.. I'd appreciate it if you responded. What does it mean that the f2 value of the independent variable is extremely low, most of which are .002, and the f2 value of the dependent variable is 1.133 and more than 1?? Is it generally possible that the f2 value exceeds 1?? In addition, is it reasonable to remove and analyze the potential variable that has a problem with permutation mean=.976, original c=.867 in step 2? Additionally, permission mean=1.000, original c=.1000 and p=.For variables that are significantly shown as 036.. How should we deal with this? It's an important and urgent situation... I look forward to hearing from you Thank you.

[@Gaskination]

f2 greater than one is not impossible, though it is uncommon. It just means there is a very large effect. Low effect-size may be due to one predictor dominating the model. It is always permissible to explore your model by removing or adding variables to see the robustness of the model.

[@chefberrypassionateresearcher]

Sir, if i want to do one way ANOVA, shall i first do measurement invariance test, so that i am able to get a model with valid indicators for my contructs (as i will remove those with inadequate loadings in second step of MICOM) & then do the descriptives and one way ANOVA?

[@Gaskination]

I have never heard of running an invariance test prior to ANOVA, but you could run a Levene's homogeneity of variance test prior to (during) ANOVA.

[@chefberrypassionateresearcher]

@@Gaskination prof. my question is that if during MICOM, i have to remove certain items due to their poor loadings, then all the models of my different groups will not have those items. Rite? So, how can i analyze the same constructs in ANOVA with all the items? Shouldnt I do MICOM first and then do ANOVA or any other anlysis with finally accepted items for the model?

[@Gaskination]

@@chefberrypassionateresearcher You would remove them from all groups, not just one group. I would only do that though if there is a strong difference.

[@chefberrypassionateresearcher]

@@Gaskination Yes Prof. I know that i would remove them from all groups, but i want to know whether i should take this respecified model / data of the constructs for doing other multivariate tests like ANOVA or even the descriptive statistics of my study variables?

[@Gaskination]

@@chefberrypassionateresearcher No. Measurement invariance is an assumption of SEM, not of other analyses not using latent measurement through factors.

[@lottiegriffin96]

Hi, thanks so much for all the videos! Do you have any resources on/could you make a video on how to examine the loadings of factors that are significant/variant in the MICOM analysis, and how to correct this? I'm running MGA for one of my PhD projects and in some MGA comparisons, 3/4 of my variables are showing variance and I don't know how to fix it!

[@Gaskination]

I don't know of a way to check for partial invariance in SmartPLS 4 (e.g., by constraining paths across groups) since PLS does not rely on the chi-square from the covariance matrix. A very coarse approach would be to check the differences in loadings between the segments and then omit indicators (one at a time) that have really high differences. However, this is also changing the measurement of the construct, so should only be done with measurement quality/validity in mind.

[@Gug_family]

Thanks! I would like to know what's the recommended permutations in numbers. When I run it with 1000 permutations, permutation mean difference for one of the relationships shows as statistically insignificant as p-value is 0.052 but if I run it with 2000 permutations, this relationship becomes statistically significant as 0.044 (p-value). Your help will be appreciated.

[@Gaskination]

As with bootstrapping, the number of permutations will have an effect on the estimates, as well as error reduction. Less error with higher permutations. Less error means lower p-values. Thus, it makes good sense that it is lower with 2000 permutations.

[@Gug_family]

@@Gaskination Greatly appreciated! Larger the better so I ran it with 5000 permutations then the p-value went up to 0.052 (2-tailed) again. I initially ran the permutation test for the invariance test (results are complete invariance) but since the results at structural level slightly change based on numbers in permutation, I also ran the bootstrap MGA and it give me p-value as 0.046 (2-tailed) for this specific relationship. I am measuring this for gender difference and now wonder whether to go with the permutation test or bootstrap MGA. Your advice will be helpful.

[@Gaskination]

@@Gug_family haha. In this borderline case, I would recommend just accepting a higher level of tolerance (e.g., 90% confidence, p

[@nomanrasheed9644]

i am using smart pls with a license provided by you during your conference of CB-SEM on 30 December. the license is going to expire on 31st march. kindly guide me how can i get full version of smart pls.

[@Gaskination]

The full version is available for purchase on the smartpls.com website.

[@bernardomarta9296]

Hello! First of all, thank you fir the amazing video. What if my original correlation is higher or equal than the correlation permutation mean and the value of the 5% but there is one (or more) permutation p value(s) lower than 0.05?

[@Gaskination]

The permutation p-value is the final determinant. So, if it is less than 0.05, then you would infer that the groups are not the same. However, taking a more liberal approach by using a p-value threshold of 0.10 is usually fine in invariance, particularly if most factors do achieve invariance at the 0.05 threshold.

[@farahfarhana4014]

do u mean 0.1 or 0.01?@@Gaskination

[@shameemashaban8257]

James sir i have gender in my model as categorical variable sir should I include this gender in my structural model and perform moderation analysis or should I not include it in my model and i should perform mga

[@Gaskination]

If it is a moderator, then use it in MGA (and exclude it from the model). If it is just a control variable, then include it as a binary variable directly in the model and do not use it in MGA.

[@sochtosach6861]

WHEN I GENERATE GROUPS FOR AGE IT ONLY CRATES 2 CATEGORIES WHEREAS I HAVE CODING AS 1,2 AND 3. WHAT MIGHT BE THE ISSUE?

[@Gaskination]

It only creates groups that have at least ten members. If one of your age groups does not meet this criteria, then it will not create that group.

[@sochtosach6861]

@@Gaskination Thank you so much. your guidance means so much in my journey. god bless.

[@farahfarhana4014]

Hi Sir, do u have paper saying that if compositional invariance is not established with 1 or 2 variables, we can proceed to step 3?

[@Gaskination]

Search for "invariance" on this page: statwiki.gaskination.com/index.php?title=References and you'll find the references you're looking for.

[@farahfarhana4014]

@@Gaskination would it be possible the variance is due to only having a single item for the involved variables?

[@Gaskination]

@@farahfarhana4014 Having only one item for a factor isn't the cause of the non-invariance. But the non-invariance could be focused there. This is particularly likely if the single-item factor is a demographic/grouping variable that has a strong relationship with other variables in the model.

[@farahfarhana4014]

Sorry sir I didn't find sources saying that we can proceed to step 3 if we are having partial invariance in step 2. How do I support my thesis?@@Gaskination

[@Gaskination]

@@farahfarhana4014 Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456-466. doi.org/10.1037/0033-2909.105.3.456

[@hameemislam753]

Hiii james gaskin if i have gender moderator in my study is it necessary to create it dummy first i have create it as binary in smartpls4 1 and 2 i e 1 for male and 2 for female is it necessary to make it dummy first i mean 1 - 0 and 2- 1

[@Gaskination]

If your gender variable was measured as binary values, then it is already a dummy variable. In this case, if 2 is female and 1 is male, then this is essentially a dummy variable for female, with male values as the reference category.

[@guangmingcao7846]

Thanks for the concise introduction. Always enjoyed watching your videos. One question: is it still okay to do MGA if STEP 2 is only partially established?

[@Gaskination]

You can move forward. You just have to report in your manuscript that any inferences or implications should be made cautiously if based on variables that did not show invariance.

[@guangmingcao7846]

@@Gaskination Much appreciated

[@CC-op3ez]

@@Gaskination Thank you for your interesting video. For example, in step 2 of MICOM, I have a total of 13 latent variables. six p-values were larger than 0.005, and others were smaller than 0.05. Would you please explain to me whether I can move forward to MGA?

[@Gaskination]

@@CC-op3ez You can just say that it is partially invariant, then move on.

[@lenifarokaofficial]

Hi, i 've a question about MICOM Analysis. what if in step 3 (scalar invariance) permutation p-values of mean it doesn't shown number. Do you know how to solve it? I'm running MGA for my thesis and now i'm stuck... hopefully you will answer ASAP :) Thanks!

[@Gaskination]

Yikes! I'm not sure why the p-values would be missing... This is something you might need to email SmartPLS support about. Good luck!

[@sarashahnawaz2727]

Hi! Thanks for the helpful video. Please let me know how to solve singularity matrix problem in MGA. I am unable to run MGA because of singularity matrix problem. Please tell me how to resolve this issue.

[@Gaskination]

Singularity matrix occurs when there is zero variance on a variable in the model. This can occur in an MGA when a variable in the model is redundant with the grouping variable. For example, if we were grouping by age and asking about drivers' licenses, we might have zero variance for the younger group because none would have a drivers' license.