Econometrics - Marginal Effects for Probit and Logit (and Marginal Effects in R)

[@tyskarmada]

Wish you were my professor at my University! Great content and easily explained. I came for the method (margins package), but I subscribed due to your easy and great short explanations of the models.

[@rizkydarmawan6540]

Thank you for this. I needed a refresher on this particular subject and this video is one of the best there is. Simple and intuitive with good practical examples 👍

[@samirhuseyn]

I like it when the video has some messages to future listeners :) great!

[@zhengzhang127]

The clearest demonstration that I have ever learned, thanks!

[@yujiangsun5428]

Soooo informative!!!! Million thanks! After watching so many videos about econometrics, how come youtube never recommended you on my home page... looking forward to your upcoming content! Thank you.

[@Grimscribe732]

Fantastic explanation, thank you!

[@henningsouz2833]

Thank you for your videos. Not only this video. This is so helpful towards thesis work. It would be greatly appreciated if you would do a video on interaction terms in binary logistic regression ( contionous*dummy, dummy*dummy) in binary logistic models. The Ai Norton article was interesting, however, not that "helpful" for the average student like myself. Getting the marginal effects for the interactions feels so difficult. Both in R, and Stata. Thank you again :)

[@NickHuntingtonKlein]

Thank you! On that topic, in Stata there is the inteff command, see stats.idre.ucla.edu/stata/seminars/stata-logistic/ And in R there is this guide stats.stackexchange.com/questions/47020/plotting-logistic-regression-interaction-categorical-in-r#47025

[@henningsouz2833]

@@NickHuntingtonKlein Thank you for you reply. Very much appreciate it. Will check both links (Stata, R). Hopefully it helps. Thanks again :)

[@arquero143]

Thank you so much, you make it so easy to understand it

[@KaptenIglo]

Thank you for your video, it helped me a lot!

[@nhungluong9311]

Hi Nick, thanks for your video. I am wondering the difference between type=“response” and type=“link”? Here is the explanation I found in R documentation for the margins package: In a generalized linear model (e.g., logit), however, it is possible to examine true “marginal effects” (i.e., the marginal contribution of each variable on the scale of the linear predictor) or “partial effects” (i.e., the contribution of each variable on the outcome scale, conditional on the other variables involved in the link function transformation of the linear predictor). The latter are the default in margins(), which implicitly sets the argument margins(x, type = "response") and passes that through to prediction() methods. To obtain the former, simply set margins(x, type = "link"). There’s some debate about which of these is preferred and even what to call the two different quantities of interest. However, it is not clear for me how to interpret the marginal effects in the two cases. Thanks!

[@NickHuntingtonKlein]

The link margins are basically what they mean - they're the relationship between a one unit increase in the predictor and the index, before it gets passed through the link function. The response margins is the relationship between a one unit increase in the predictor and the increase in the expectation of the response variable. For example with a binary outcome, the increase in the probability of a 1 rather than a 0

[@Tech_home_ify]

Hi Thanks very much for this video. I would love to know the package you installed before library(margins). Thank you. I am using version 4.3.1

[@NickHuntingtonKlein]

The other two packages I loaded before margins were "wooldridge" (which I just used to get data) and "jtools" (which I used for regression tables, although these days I'd more likely use modelsummary)

[@FrancisAAfful]

Thanks very helpful, but what is difference between using invlogit of your estimates to get the probabilities and using marginal effects

[@NickHuntingtonKlein]

If you've done it properly you should get the same estimates either way, but marginaleffects is easier and faster to do and provides stuff like standard errors.

[@FrancisAAfful]

@@NickHuntingtonKlein thank you very much for the quick response. really appreciate your efforts to make these videos

[@r3lativ]

Fantastic videos, Nick!

[@bennyke1979]

This video's helped a lot, thank you!

[@liss_eq9164]

Thank you for this, very helpful!

[@arasafaryan73]

for marginal effect at the mean; lets say you have a dummy variable as dependent outcome and a regressor in your model for which you take the mean; how do interpret the marginal effect at the mean? Like what does the number calculated say?

[@NickHuntingtonKlein]

This would say that "at the mean value of all the regressors in the model, a one-unit increase in the regressor of interest increases the probability that the dependent variable is 1 by (marginal effect)"

[@CanDoSo_org]

Thanks, Nick. I got a question. Do you know any package which can calculate marginal effects for sampleSelection model? "margins" you introduced here does not work.

[@NickHuntingtonKlein]

Hmm I'm not sure off the top of my head; maybe try the marginaleffects package

[@CanDoSo_org]

@@NickHuntingtonKlein the marginaleffects package does not work. I tried several package, no luck so far.

[@debashisbanerjee260]

Thanks ! You are a rockstar

[@Allu-oe6ih]

Thank you for very interesting video. I have a few question that i'm a bit confused. Firstly when i use margins(glm_model) the p-values differ a bit from the ones from logistics regression. Which one i should report if i'm going to report marginal effect in my paper. Secondly, should interpetate coefficient of categorical variable as in normal regression agains the "zero class"? So probability for x is higher/lower by % point than the zero class for a given variable. And lastly do you possibe know we AME for margins and logmfx yield a bit different results? And which one one should use? Thank you for answering in overhead. Have a nice weekend :)

[@NickHuntingtonKlein]

To your second question, yes it's all relative to the reference group. For first and third it's because margins gives average marginal effects (AME). Marginal effects at the mean (MEM) report the same p values as the base model, but AME estimates it at a different level of the predictors, so it doesn't. Logitmfx does MEM.

[@Allu-oe6ih]

@@NickHuntingtonKlein Thank you for a great answer! I’m going to go with margins because I’m after AME. Do you have any advice of what kind of standard errors to use with margins. As an economist I would like to choose “robust” as in (robust to heteroskedasticity), but I’m not very familiar with delta method which is a default option. I understand that these are case specific, but if you have any advice or paper/article where to read about usage of margins standard errors choice I would appreciate it. Than you for you videos by the way. They are great!

[@NickHuntingtonKlein]

@@Allu-oe6ih If you think you have heteroskedasticity then you should definitely use heteroskedasticity-robust standard errors. I'd recommend the website marginaleffects.com for more information (and they do have a page on uncertainty and standard errors)

[@CamilaSanchez-ex2jn]

Hi nick, I would like to know how to calculate marginal effects for a binomial logistic regression with interaction term. something like when D is high and Z is high, when D is High and Z is low. I mean the possible combinations for my interaction term. I don't know if in R there is any command for this. Note. D would be the independent variable and Z the interaction term. Thank you very much. Camila.

[@NickHuntingtonKlein]

Interpreting interaction terms in logit can get a little tricky, as described by Ai and Norton (2003), since the interaction effect varies based on all the values of all the covariates. I believe the intEff function in the DAMisc package helps map out the interaction effects. Or just read hte paper and use predict() to map it out yourself.

[@andressatb3908]

great video, thanks!

[@kwizeralambert1316]

Hello, I love your content. Would you find the time to create content on The generalized method of moments (GMM) and its application especially with dynamic panel data, and how to conduct analysis with GMM in R? Thank you

[@NickHuntingtonKlein]

Thank you! Honestly I'm probably not the best person for that topic as I've barely worked with dynamic panel gmm

[@kwizeralambert1316]

I understand, I am reading about that topic since I am forced to apply it in the project research I have. Thank you. By the way, I was even checking on your book "Effect" if you wrote about it@@NickHuntingtonKlein

[@NickHuntingtonKlein]

@@kwizeralambert1316 it's more common in macro, which I don't really do. It's not in the book (or if it is, it's a brief mention)

[@kwizeralambert1316]

I understand, thank you@@NickHuntingtonKlein

[@haraldurkarlsson1147]

Very interesting. Now there are some missings in the card data. Fathers' ed is missing about 23% and IQ about 32%. Is that of concern in the modelling?

[@NickHuntingtonKlein]

Yes that can be a concern and may be enough to warrant an approach like multiple imputation

[@haraldurkarlsson1147]

@@NickHuntingtonKlein What is considered an "acceptable" loss percentage wise? This is tricky stuff. I know that major issues have arisen due to improper imputation (e.g. Rogoff at Harvard if I recall correctly).

[@NickHuntingtonKlein]

@@haraldurkarlsson1147 was Rogoff a multiple imputation issue? I thought it was something else. There's not really a specific cutoff (cutoffs that guide your inference or analysis in statistics are almost always a bad idea or at least subpar). But if there's a small amount of missing data (say in the like 5% range), then it likely won't cause a huge issue. More and at the very least you need to start thinking about why it's missing

[@haraldurkarlsson1147]

@@NickHuntingtonKlein I think you are right in regards to RR (Reinhart and Rogoff). I may have mistaken omission of countries in the study by RR as the result of imputation. In the paper criticizing the results (Herndon, Ash and Pollin) it is stated that "The omitted countries are selected alphabetically. It is clear from the spreadsheet itself that these are random exclusions." (section 3.2 Spreadingsheet coding error). That is what caught my eye. However, it does show the effect of selective use of data and its dangers. Thanks for your reply.

[@dehiole6463]

0:45❤❤

[@akoredeadebayo4269]

Thank you... Really helpful

[@nishchaymehrotra1342]

Hi Nick, thanks a lot for explaining! I wanted to know how would someone interpret the average marginal effect results when the variable data has been rescaled to a range between 0 & 1 in a logistic regression model? I mean can we still talk in percentage change or do we have to rescale the marginal effects in order to interpret them? Thanks again!

[@NickHuntingtonKlein]

Meaning the dependent variable has been rescaled to a continuous variable between 0 and 1,and you're running fractional logit? In that case the only thing that should change for the interpretation is that you're back to effects being in terms of a change in the conditional expectation of the dependent variable, like in OLS, rather than terms of the probability that the dependent variable is 1, as in regular logit.

[@NickHuntingtonKlein]

@Greg Wann The interpretation remains the same in one sense - the marginal effect coefficient still gives you the relationship between a one-unit change in the predictor and the probability of the outcome. The only difference is that now a "one unit change" means "a one standard-deviation change" in the original variable.

[@vojtechkolar5897]

Hello, thanks for the video. Did I understand it right? I have a binary logit and my predictor variables are factors such as: gender(male = 1, female = 0), income = split into brackets a, b c, d,, owning a car or not, etc..) R works with these factors like dummy variables, so i always have to set one refference level which than disapears from the results. In the results I understand that for example when a male is 1 and a reference variable is 0 = woman, the marginal effect i get with using the same approach as you in thhis video can be interpteted like percentage difference in reaction between men and women? Like if I get 0,16 marginal effect that means that men react in 16 % percentage points more? My dependent variable, is 1= reaction, 0= no reaction.

[@NickHuntingtonKlein]

Correct. As long as they're binary, character, or factor variables (or numeric variables that only take the values 0 and 1) it will treat them as dummies, and automatically drop a reference level for you (you can reorder the factor to pick it yourself, or if using feglm in fixest, use i() with the ref option). Then, yes, the marginal effect will represent the percentage point difference between the marginal effect of the coefficient for that group and the reference group.

[@vojtechkolar5897]

@@NickHuntingtonKlein Thanks !

[@elliotthardy3371]

Great video- when discussing marginal effects of logit model is it better to focus on the statistical signficance of the logit coefficients or those in the AME?

[@NickHuntingtonKlein]

Thanks! The significance should be the same for both.

[@elliotthardy3371]

@@NickHuntingtonKlein In my results I have a variable whereby it is insignificant on the logit yet sig at 5% level for AME & MEM. The only reason I can think of is that they're two different hypotheses- in that when looking at marginal effects you're testing a function of all the coefficients not just the one of interest?

[@NickHuntingtonKlein]

@@elliotthardy3371 ah, yeah, it can happen sometimes (very model dependent). It does have to do with the marginal effects incorporating all variables. In these cases you generally want to focus on the original model significance, as the AME significance has a lot more moving parts and choices in it

[@lukaverstockt7577]

Dear Nick, I'm having issues in interpreting the marginal effects of a discrete variable in my model. The dependent variable is binary and represents if someone owns financial assets. The variable 'risk preference' is a score that was given on the basis of a few questions and can take values between 0 and 10. - Do the marginal effects in this case represent the percentage change when the 'risk preference variable' increases by 1% or by 1 unit? - For the interpretation of marginal effects at mean for dummy variables: I assume that the dummy variable is set at the mean, so does the logitmfx output represent the change in probabilty of the event occuring when the dummy variable is increased by one unit from the mean? (Or does it represent the dummy variable changing from 0 -> 1? - A side question: does it matter for R whenever dummy variables are pre-coded in an excel file? I'm pretty new to R so I didn't knew about the factor notation of categorical variables before cleaning and preparing my data for analysis. Any help woud be greatly appreciated

[@NickHuntingtonKlein]

1. Marginal effects give the impact of a one-unit change in the predictor 2. You're talking about a dummy predictor here, instead of your 0-to-10 risk preference variable? It represents a unit-scaled version of the slope of the logit function at the mean. That's pretty confusing, but if you use average marginal effects instead (which I recommend anyway) then it's just a dummy variable change from 0 to 1. This is a case where AME is easier to interpret than MEM. 3. Depends what you mean by pre-coded I guess. Like a column of 0s and 1s? Sure, it can handle that.

[@lukaverstockt7577]

@@NickHuntingtonKlein Thanks for the swift response! 1. Okay, because I was a bit confused as the logitmfx indicates that for certain variables (dummies in my case) the dF/dx is for discrete change. This made me think that the other variables could be interpreted as non discrete changes. 2. Yes, I'm talking about a dummy variable in this case. I'm reporting both marginal effects in my master dissertation, but the discrepancy between both led me to conclude that the discrete change for the MAM was from the mean instead of from 0. 3. Yup, I pre-coded it in columns of 0s and 1s. Thanks a lot for the clarifying input!

[@21LeonidasZ]

I would like to ask if there is a difference between average partial and marginal effects because I came across with the former term, yet I cannot see any difference. Thank you for the informative video.

[@NickHuntingtonKlein]

No difference

[@21LeonidasZ]

@@NickHuntingtonKlein Many thanks for the fast reply.

[@sarahnunu]

Just found this video and it's extremely helpful. You make it so simple and easy 😄. I have a question though (in R) perhaps you can help me out. I have 2 datasets (10 years apart with different N observation). The dependent var: success/not. Goals: to observe from the 2 time periods what are the attributes an individual has to be successful and if certain level of education plays a huge role into it. Do you have any video / thread suggestion / reference for this matter or could you do a video for this by chance 😅

[@NickHuntingtonKlein]

Thanks! And for that question it sounds largely like a probit or logit modeling question. Success is the dependent variable and then you can see which predictors predict success

[@melissahondeveld571]

Hi Nick, I am currently writing my thesis and this video helped me a lot! But I do have a question. My dependent variable is a dummy variable, so I want to use logit regressions for it. However, I am trying to get the mfx table to Word to show the coefficients, but this does not work. If I'm trying to export the table using asdoc or estout, both export the logit table instead of the mfx one. Do you know a solotion for this? Or should I just take the logit table and make the calculations myself to get such mfx table? Hope you could help me out, it would mean a lot to me.

[@NickHuntingtonKlein]

This is in Stata? You need to add post to your margins command. See www.statalist.org/forums/forum/general-stata-discussion/general/1481777-exporting-marginal-effects-using-esttab

[@melissahondeveld571]

@@NickHuntingtonKlein yes, it's Stata, I'm sorry. Thanks a lot!

[@namelessbecky]

Thank you

[@shwetaagarwal6240]

Hey Nick, great video! Can you also explain how to interpret a logit or probit regression model with year dummies or industry dummies. Please

[@NickHuntingtonKlein]

Same way you'd interpret any other logit or probit model - in terms of the index function, or take the average marginal effects and then in terms of the probability of the outcome

[@shwetaagarwal6240]

@@NickHuntingtonKlein I meant how do we compute the probability with a year dummy or industry dummy. Sorry for posting the question imprecisely

[@NickHuntingtonKlein]

@@shwetaagarwal6240 Plug in all the appropriate values for the dummies and other variables and get the predicted probability? I'm not sure I understand what you're asking. There's nothing special about dummies in probit/logit aside from how you'd normally treat dummies in regression (other than you don't want to have too many of them, probit/logit can't handle it)

[@shwetaagarwal6240]

@@NickHuntingtonKlein I will email you explaing in detail my concern. Thank you for prompt reply. You are awesome!

[@thuydungnguyen2983]

Dear Nick, I am just wondering how can we estimate probit/logit model with (multiple) fixed effects and cluster standard errors by firm. Thank you so much. I check the "bife" package but they only work for fixed effects.

[@NickHuntingtonKlein]

The feglm function in fixest can do this.

[@thuydungnguyen2983]

@@NickHuntingtonKlein Thanks Nick. I know this function, but they only has logit model.

[@NickHuntingtonKlein]

@@thuydungnguyen2983 does it? I thought you could set it to family =binomial(link='probit'). In any case, fixed effects probit is less well-defined than fixed effects logit, and isn't always advised. Notice how, for example, Stata's xtprobit doesn't have a fixed effects option, while xtlogit does. In any case I'm pretty sure you can do something they call fixed effects probit in the alpaca package. I've never used it myself though

[@thuydungnguyen2983]

@@NickHuntingtonKlein many thanks Nick. I will try with the package you suggested.

[@kriswright3355]

When interpreting the coefficients for probit and logit using "export_summs" from the margins package, is R assuming homoskedasticity? How would I account for heteroskedasticity? I get an error when running coeftest(probitmargins, vcov. = vcovHC). Thanks!! Great videos btw

[@NickHuntingtonKlein]

Try robust=TRUE in the export_summs function

[@ssuyingchen7412]

Hi, I ran the logitmfx command with my logit model and got the results, but I am wondering what value are the binary variables set to? is it 0? or the mean? Please help!

[@NickHuntingtonKlein]

It is evaluated at the mean of all the predictors; by default logitmfx gets marginal effects at the mean. You might also consider using margins() from the margins package, which by default does average marginal effects instead.

[@ssuyingchen7412]

@@NickHuntingtonKlein Thanks for the reply. I tried running margins() but got an error saying "variables were specified with different types from the fit". All the variables listed in the error message were binary. Is there any way to solve this? I also tried running logitmfx command with atmean = FALSE, but my laptop couldn't run it... (the error says "vector memory exhausted (limit reached?)")

[@NickHuntingtonKlein]

@@ssuyingchen7412 I've never seen either of those errors. This might be more a stackexchange question

[@saramagalhaes7279]

Hey Nick! First of all thank you for your work! I have a question: is it possible to use the margins package, just as you did in the video, with ordered probit models?

[@NickHuntingtonKlein]

I'm not sure. If not, try the marginaleffects package, or the erer package

[@eyusal8023]

Dear Nick, I am using Multivariate Probit model for my research project and find its coefficient estimates of Multivariate Probit regression results through using five dependent variables but I could not able to find Marginal effect for each dependent variables. Therefore, please could you help me how or the steps that I should follow to calculate the Marginal effects of explanatory variables on dependent variables in stata if it is possible? Finally, I am expecting your lucid responses.

[@NickHuntingtonKlein]

I believe the margins command with the dydx option should do it

[@eyusal8023]

@@NickHuntingtonKlein will u send me z syntax if it is possible ?? Or will send me script of mvprobit in R

[@NickHuntingtonKlein]

After you do your probit model, do margins, dydx(X) To get the marginal effect of x That's generally how the margins function works

[@eyusal8023]

@@NickHuntingtonKlein I did it as u said but the mfx results are the same with that of coefficients.

[@eyusal8023]

Will u send me ur whatsup or facebook address and then let me discuss about z problem if it is possible ??

[@TashaRoose]

What should I do, if my logit is multinominal?

[@NickHuntingtonKlein]

Try the mlogit package

[@anchaleeaommy536]

@@NickHuntingtonKlein ​ Hello Nick, can you explain more for the marginal effects in multinomial function, is it correct for my case ? 1. Marginal

[@NickHuntingtonKlein]

@@anchaleeaommy536 That code won't estimate a multinomial logit. glm() will only do binomial logit. As I mention in the comment, try the mlogit package. Once you do that, if you calculate marginal effects, they can be estimated in a similar way to a binomial logit, in that the marginal effects for a particular option (you'll get one set of marginal effects for each option except the reference) are the relationship between a one-unit increase in the predictor and the probability of choosing that option relative to the reference.

[@Probusto]

Hey Nick, thanks a lot for the video! I have a question: is there a way to export the export_summs output to Overleaf/LaTeX?

[@NickHuntingtonKlein]

Yes, you can pipe it to the huxtable quick_latex function

[@Probusto]

@@NickHuntingtonKlein thanks a lot :)

[@abdullahalruhaymi660]

Hi Nick, Could you help me with missing data, Code for MAR is MCAR, I have the MCAR code.in R.

[@NickHuntingtonKlein]

If you think your data is MAR, I'd recommend the mice package. See here for a guide data.library.virginia.edu/getting-started-with-multiple-imputation-in-r/

[@abdullahalruhaymi660]

@@NickHuntingtonKlein Thank you, it was very helpful, I appreciate it, Ok sir, I want your specific answer: Q1- I generated 10% from my data MCAR type, now I want to generate MAR mechanism again, I did the codes both R. MAR followed Schouten and Vink paper(2018) without using ampute function and without MICE library. Now I want to generate MAR in a different approach [Psycologists approach ( Graham paper)] that is MAR is conditional MCAR, I do not know how to set conditions as said to let the mech instead of MCAR is into MAR. please help me, and if can give sessions I will pay for it, thanks

[@NickHuntingtonKlein]

@@abdullahalruhaymi660 hi again, I'm afraid I don't know those papers and haven't done all that much multiple imputation myself. I'm also not offering paid sessions right now. Your best bet might be looking for someone else on upWork or something

[@stoychorusinov5519]

Can you show us how to do the graph?

[@NickHuntingtonKlein]

Which graph? This one? lost-stats.github.io/Presentation/Figures/marginal_effects_plots_for_interactions_with_categorical_variables.html

[@stoychorusinov5519]

@@NickHuntingtonKlein yes please

[@NickHuntingtonKlein]

@@stoychorusinov5519 that link shows how

[@stoychorusinov5519]

@@NickHuntingtonKlein seems complicated tbh and some explanation of the syntax will be lovely

[@lezlhynevlogs7259]

Can u help me how Will i explain the concept of marginal effect?

[@NickHuntingtonKlein]

In this context, the marginal effect is the effect of a one unit change in the predictor on the probability that the outcome is 1.

[@MegaMattia77]

Hello i got a question: how to calculate the marginal effect at the mean with library(margins)? thank you

[@MegaMattia77]

and in linear regression?

[@NickHuntingtonKlein]

Use the at option of the margins function to evaluate at a list of values, and make those values the means.

[@meddykacy8819]

Thanks man!

[@koketsomalesolo7728]

how do we calculate partial effect without using R?

[@NickHuntingtonKlein]

well, you could do it by hand, as mentioned it's beta*(predicted probability)*(1-predicted probability). Or did you have a different language in mind

[@koketsomalesolo7728]

@@NickHuntingtonKlein yes I meant by hand. Thanks

[@dialloibrahima5628]

how to getting marginal effects for ordered model by use fonction clm? please i need your help

[@NickHuntingtonKlein]

I'm not familiar with the clm function, but check the erer package for ordered model marginal effects

[@dialloibrahima5628]

Nick Huntington-Klein when I load package erer with my R 4.0.2 it send me “ installation of package erer had non-zero exit status”! Can you help me?

[@NickHuntingtonKlein]

@@dialloibrahima5628 see the other error messages it lists first. It should give more detail. This is more a question for something like stackexchange

[@dialloibrahima5628]

Nick Huntington-Klein thank you

[@MegaMatzzz]

i love u bro

[@Gustavo-ui2bi]

Legend