Logistic Regression with R: Categorical Response Variable at Two Levels (2018)

[@earlymorningcodes6100]

2:46 two way table of factor variables 3:23 Data Partition 5:21 logistic regression Model,8:36 prediction 10:05 probability calculation 17:32 error test data 14:24 Interpretation of coefficinet,18:28 goodness of fit 15:03 error training data 16:04 confusion matrix

[@bkrai]

Thanks!

[@Lilian.Chidinma.Nwafor]

Dr., I don't know how to tell God to bless you for me. your MLR video saved me during my research presentation, see my explaining like a pro!! all thanks to you. This also is very helpful, especially how you gave detailed explanations.

[@bkrai]

Glad it was helpful!

[@diliniherath1299]

No words to express my gratitude for you. Found your channel days before submitting the project and you saved me !

[@bkrai]

Great to hear!

[@wardhereadan1187]

DR. Bharatendra Rai that video was amazing!! I hope you continue to post more videos like this! seriously amazing!!!!!!!!!!!!!!!

[@bkrai]

Thanks for comments!

[@crossray974]

Thank you Mr. Dr. Bharatendra - your stuff and method are on top of youtube, greets from Europe!

[@bkrai]

Most welcome!

[@youroldmangaming8150]

Thanks mate. Been struggling to find a practical person to show how to do this. Very clear and well thought out. Thank you.

[@bkrai]

You're very welcome!

[@shaikhalishams4065]

Finally I've got a perfect video on this topic.

[@bkrai]

Thanks for comments!

[@kir66846037]

the best teaching of logistic regression!!!! Thanks a lot

[@bkrai]

Most welcome!

[@mnorberta24]

Thank you for helping save my grades for this module!!!!! I might just watch all your videos because they're so helpful!!!!

[@bkrai]

Glad to hear it!

[@saipri]

Extremely crisp and accurate! Hope you get many more views! By far the best on this topic...

[@bkrai]

Thanks for the comments!

[@philippdegens6776]

Thanks. You gave a clear and concise explanation and a bonus was that it was in R which I am learning.

[@bkrai]

You're very welcome!

[@bkrai]

You may also find this useful: kzbin.info/aero/PL34t5iLfZddvv-L5iFFpd_P1jy_7ElWMG

[@allabtlyf]

Wonderful video.. I was struggling to calculate the probablity from estimate in notebook but you made it quite simple. Thanks a lot

[@bkrai]

Thanks for comments!

[@jbei9981]

Thank you so much. Excellent video. I was really thinking I would fail my assignment until I found this.

[@bkrai]

You're very welcome!

[@viigeminalegio]

Thanks you very much Dr. Bharatendra. I was looking to solve some of my doubts and I finally solved them. Thanks for sharing your knowledge. I wish I could have the opportunity to help you in some occasion. Thanks for all, great job.

[@bkrai]

Thanks for your comments and feedback!

[@pranatim]

Best tutorial on logistic regression. Thank you so much for sharing.

[@bkrai]

You're very welcome!

[@soumikchatterjee3996]

Excellent video. Just few things to mention. In glm result, residual deviance is greater than residual degree of freedom that means the data has overdispersion. Better to use quasibinomial function rather than binomial. Other wise p value would show false significance level. Second thing to mention backward variable selection without montecarlo permutation has type2 error therefore better to use it cautiously or use Information theoretic approach proposed by Burnham etal with model weight as a criterion. Thanks for this beautiful video sir

[@soumikchatterjee3996]

Although you created seed and resample which can reduce the error but it is extremely difficult to find proper seed size without understanding model weight (wi). Thanks

[@bkrai]

Thanks for the feedback and comments!

[@laxmanbisht2638]

Sir, thanks a lot!

[@bkrai]

Most welcome!

[@flamboyantperson5936]

Excellent video Sir. You are a great statistician and expert in R. Thank you for the video Sir

[@bkrai]

Thanks!

[@jared1122]

Thank you Dr Rai for the wonderful explanation👍 God bless you 🙏

[@bkrai]

Welcome!

[@PA_hunter]

Thank you Dr. Bharatendra Rai. Can you explain more why Rank 1 is not included in the model, please?

[@bkrai]

For factor independent variables, we covert them to dummy variables. For more detailed coverage see: kzbin.info/www/bejne/qWOWdIB_n8uqfs0

[@williamstan1780]

Very informative video and explain it in a manner that easy to understand I have a question though , what is the difference between logistic regression and multinomial logistic regression ?

[@bkrai]

response variable has more than 2 levels in multinomial. See this for details: kzbin.info/www/bejne/nKXNf6iGoLWEeJY

[@nth.education]

Amazing explanation, loved the way you went through with the code and how to proceed step by step. I have a doubt with the pvalue calculation at the end. Can you explain a bit more the "with" command you used ? i couldn't understand the parameters used in that, interpretation of p-value is fine, but would like to know the use of the command so i can employ that in some places as well. Thanks

[@bkrai]

you can run ?with in the console, it will give you all details and also examples.

[@narasimhapuvalla3211]

1.) Suppose we have categorical fields in our data. Is it mandatory to always change to numeric factors ? 2.) If the answer for question 1 is correct, then what if we have too many unique values in each category columns? Let us take for example : I have a dataset of 100,000 records. There are a few columns with categorical data in it. Each of these categorical columns may have 1000 or more unique values. So if I convert them into factors, then "labels = c(1:1000 or more)". Is this ok to do it this way? 3.) Is there a way to not convert categorical data into numeric values and still use them in the machine learning model? 4.) How do we deal with Date fields? 5.) The conversion of categorical variables into dummy variables --> should we do this in all cases or is this something we need to consider only if the unique values in the categorical fields are limited to a lesser number?

[@dhavalpatel1843]

1. No , it is not mandotary to change. You can set family parameter as “binomial”. 2.Answered in no.1 3.Answered in no.1 4.Convert it into factor variables 5.Try to consider it in all cases.

[@dipeshpatel3106]

Sir plz make on Monte Carlo simulation R

[@bkrai]

Thanks for suggestion!

[@rajeshtukdeo]

Amazing video to understand the logistic regression concepts thoroughly !!!

[@bkrai]

Thanks for comments!

[@rohitkamble1737]

Very clear explanation. Understand all things

[@bkrai]

Thanks for comments!

[@rohitkamble1737]

@@bkrai sir, I am working on project on Real estate and banking model to predict prizes of house, could you plz help me on that?

[@dmukherjee4049]

Sir can you explain "goodness of fit test". What is df.null-df.residual, lower tail & why it is 'F'? Thank You

[@bkrai]

When in RStudio, you can run ?glm. This will provide you with more details.

[@genevieveemefaasare8352]

thanks so much. very precise and concise explanations. Thank you Sir.

[@bkrai]

You are very welcome!

[@mohamedbousarout6515]

Thank you sir keep up the good work ;)

[@bkrai]

You are welcome!

[@yogeshdhar5825]

Very well explained!

[@bkrai]

Thanks for comments!

[@harishnagpal21]

Hi Bharatendra, I saw your linear regression video also. The explanation on results was fantastic. I got to learn new things. One query - when to use linear and when to use logistic regression? Thanks

[@bkrai]

When y variable is factor, logistic is used. For numeric y linear regression is used.

[@harishnagpal21]

thanks :)

[@jarrelldunson]

Thank you for sharing, very helpful

[@bkrai]

You are so welcome!

[@lindanidube5714]

This was amazing... you explain everything step by step nicely :-)

[@bkrai]

Thanks for your feedback!

[@femiakinmade4077]

I enjoyed your video, thank you! Can I get some clarity on why you used the "train" dataset in your prediction instead of "test"? dataset: ## p1

[@bkrai]

After 'train', I also use 'test'. Note that if you get good results with 'train' but not with 'test', it will suggest over-fitting problem.

[@femiakinmade4077]

@@bkrai Thanks for your response. Appreciated

[@bkrai]

Welcome!

[@fernandoflores3161]

Excellent explanation! How do you deal with ordinal and nominal categorical variables?

[@bkrai]

If response variable is ordinal, refer to this: kzbin.info/www/bejne/p5zMp32wn957paM

[@kavyayd3577]

Great explaination, sir, can you upload a video of logistic regression with more than 10 varaibles. it would be great help.

[@bkrai]

The process will work same with any number of variables.

[@anuchowdarybds]

Very clear explanation . Thank you . Do you have any more videos on logit regression ?

[@bkrai]

Thanks for comments! The link below also has multinomial logistic regression and other regression based methods. kzbin.info/aero/PL34t5iLfZddtKNwFNic3HWNV2qMsQ9AjD

[@ezechielamoussou7409]

Thank you for the video Sir. If I were running a logistic regression with categorical predictor variables, should I change them to factors?

[@bkrai]

Yes.

[@alphar85]

You are just amazing 👏. You made my life easier with the codes.

[@bkrai]

Happy to hear that!

[@AnaPTedim]

Great video it really helped a lot. I have a question though can I use the same model if one of my categorical variables has in the two-way table that equal zero? If not is there any alternative? How can I solve this?

[@bkrai]

Let's say your categorical variable has 10 levels and the last one has frequency below 5. You can combine last two levels into one and then do the analysis.

[@AnaPTedim]

Thank you very much! That might work :)

[@victorhenostroza1871]

Thanks man, again other amazing job, u r the teacher we all want at univ.

[@bkrai]

Thanks for comments!

[@Debashish_Chatterjee]

I love your videos .... concise and to the point. Superb .... keep it up

[@bkrai]

Thanks for comments!

[@shyamchaurasiya1069]

Love You Sir Very Useful videos

[@bkrai]

Thanks for comments! For recent Python video, see this link: kzbin.info/www/bejne/o3zFZpuIf9Knebc

[@hanadmohamud1881]

Thank sir

[@bkrai]

You are welcome!

[@yousif_alyousifi]

Hi, Dear Dr.Bharatendra Rai What are the best models for fitting the binary data? I know that the logistic regression model is one of the models. What is the other model to make a comparison with the logistic model to find the best model? I would be grateful if you could assist me with this. I look forward to hearing from you soon Best regards,

[@bkrai]

You can use tree based methods for comparison, especially random forest and extreme gradient boosting. See this link for details: kzbin.info/www/bejne/nnSvfICfj6eHqLc

[@yousif_alyousifi]

@@bkrai Thank you, Prof. Are these methods (tree-based methods) can be used for regression or classification? Since my concern is to do regression ( predict disease status). As I think that these methods are used only for classification. Kindly confirm. Best regards

[@bkrai]

It does both regression or classification. I have included examples for both regression and classification.

[@yousif_alyousifi]

@@bkrai Thank you, Prof.

[@guruji885]

Sir Outstanding 👍✍️

[@bkrai]

Thanks!

[@sagaranvekar856]

Great Explanation.Thank you Sir!

[@bkrai]

Thanks for comments!

[@abdulazeez9863]

Excellent explanation... please make a video of Boosted Regression Tree model with R. Thank you sir.

[@bkrai]

Thanks for comments and suggestion! I've added it to my list.

[@reubenmarfo9855]

Professor, can you please comment on why in your previous video on logistic regression, you trained the model and predicted on the same data without splitting.

[@bkrai]

Just wanted to show mainly how to run logistic regression. But after getting feedback created this on which is more complete.

[@valeriasanchez4910]

Excellent video Dr.!, I just have one question: Why it is necessary to do the data partition for the estimation?

[@bkrai]

It can help to avoid over fitting which happens when results are good with training data, but not so good on test data.

[@mueezwaq]

Hi there, thanks for this. I don't like how R displays the results for factors with more than 2 level - is there any way to get output like SPSS (which supplies a single odds ratio, 95% CI and p-value for each variable in the model). I have tried both the logistic.display and exp() commands but they do not provide an overall value like this. Any ideas?

[@bkrai]

You can use the output and customize it.

[@maheswarivemula141]

Thank you sir for the wonderful video. Sir, I have a doubt that I'm not getting value while running on the test dataset. Could you please help me out of this error. It is showing ' all arguments must have same length '

[@bkrai]

Check your data again.

[@InfinitesimallyInfinite]

Brilliant video professor. I have question... so it is always that you convert categorical integer variables into factor variables before performing logistic regression? At the other places, like the algorithm XGB, I haven't seen you convert 'Admit' variable into a factor variable, why is it so? Thanks.

[@bkrai]

Different methods require data to be prepared in certain way. For example, XGB and neural networks require response to have numeric format.

[@InfinitesimallyInfinite]

Thanks professor for the quick response. Really appreciate. 😀

[@bkrai]

Thanks!

[@Drgautham]

Thank you so much Professor🙏🙏

[@bkrai]

You are very welcome!

[@rizwanghulamhussain7309]

Excellent Video! Could you please guide how to fit panel logistic regression in R. I want to make confusion matrix / ROC curve using pglm library but could not find fitting probabilities in pglm library

[@dipanjanroy589]

sir can you please provide the code for testing accuracy of this example. I'm a new learner & i find it pretty interesting & simple by the way you teach.

[@bkrai]

It's in the description.

[@JamTik734]

Perfect Sir

[@bkrai]

Thanks!

[@rohinipatil2929]

Very well explained

[@bkrai]

Glad it was helpful!

[@kartt100]

Excellent explanation sr

[@bkrai]

Thanks!

[@ramp2011]

Great video. rank is a factor variable and looks like logistic regression has auto converted that in to dummy variables internally (from the summary model). Is there a way to find which algorithms auto converts categorical variable to dummy variables automatically and the ones one has to convert manually? Thank you for your help

[@bkrai]

Many algorithms do not need conversion of categorical variables to dummy variables. However, when using regression-based methods, R does so automatically.

[@MemphianSounds]

Great as always! What do you do when you have so many rows and variables that your computer can't compute the vector in R?

[@bkrai]

You can take a sample.

[@rithishvikram1759]

wow wow wow wow wow wow wow wow thank you so much sir

[@bkrai]

Thanks for comments!

[@adtx11]

One quick question: model 1 : a is the output variable, b and c are covariates, and both have significant p value. Model 2 : same output variable, b and d are covariates, and both have significant p values after we run the summary command. Finally Model 3: same output variable and all three b, c, d are covariates. Here if we see that only b and c are significant, but d doesn't have a significant p value , - then how do you interpret the result ? Can we say that adding covariate d doesn't add value to the model , even though it was significant in the previous bivariate scenario? Thank you.

[@bkrai]

Check relationship between c and d, that may help clarify.

[@SachinSingh-wr5yv]

sir, please make a video on K- Fold cross validation.

[@bkrai]

Thanks for the suggestion, I've added it to my list.

[@evansumido6191]

hi sir. do you have a code for cross validation? thank you.

[@bkrai]

refer to this for various ways to use CV: kzbin.info/www/bejne/fZ7Oeal6qJV6fZo

[@priyadarsinisamal1779]

sir how can i use one data set for training and another different dataset(having similar variables like training set) for testing?

[@shajibkumarguha234]

Hello Sir! Why did you choose rank as factor and not as ordered?

[@bkrai]

You are right, ordinal will be more correct.

[@akashprabhakar6353]

Thanks for this video sir... Kindly tell how can we increase the accuracy of this model...as error rate is quite high..

[@bkrai]

You can try other methods to improve accuracy: kzbin.info/aero/PL34t5iLfZddsQ0NzMFszGduj3jE8UFm4O

[@mujeebrahman5282]

Absolutely brilliant thank you

[@bkrai]

Thanks for comments!

[@mujeebrahman5282]

Dr. Bharatendra Rai could you please make videos on Machine learning using python

[@bkrai]

Thanks for the suggestion, I'll work on this.

[@zelennefontainerivero5091]

Thanks a lot for the video, it helped me a lot. Would be great if you could plot these results from your video or at least write me please how would you do it.

[@bkrai]

Let me know what exactly you are looking to plot. Here results are simply summarized in the form of a confusion matrix.

[@YatiChoudhary]

Sir, do we change intergers into factors if the variables are categorical even in Multinomial Logistic Regression or it is done only in Logistic Regression?

[@bkrai]

For Multinomial Logistic Regression you can refer to this: kzbin.info/www/bejne/iWPVi6NqgcShjtE

[@YatiChoudhary]

@@bkrai Sir actually I came to this video after watching your video on Multinomial Logistic Regression. But now I am confused if we should always change all categorical variables into factors or it just happens in logistic regression. Because in Multinomial Regression you changed only response variable into a factor.

[@bkrai]

For response variable I would say yes. But for others you can go case by case.

[@YatiChoudhary]

@@bkrai Thank you Sir

[@bkrai]

You are welcome!

[@amritthapa9315]

Could you please explain the importance of "xtabs" command in logistic regression? You said we should not get zero. Could you explain more on this.

[@bkrai]

Key idea is to have sufficient number of samples in each cell. If there are too few or zero samples, then the prediction model may not be stable or consistent.

[@JidduVillarin]

@@bkrai Thank you for this video. It is very concise and understandable. I'd like expand on this question slightly. If you did have a zero value in the xtab, what would have been the appropriate course of action?

[@yaweli2968]

If you have two or more categorical variables which are strings, how do you decide which one to make a factor of 0 or 1. Like how do you assign them specific factors ?

[@sk93359]

Hi Sir, i like your video and very useful for learning thanks for making video, i have a question can you make a video on credit card data set i want to analyze customer who will better spend on category like online, dinning ,etc, can you predict dinning spend , thanks i hope you will post video soon i am waiting your video

[@bkrai]

Thanks for the suggestion! Send me a link to this dataset.

[@sk93359]

@@bkrai data.world/ash018/credit-card-transaction. This is data set I want to analysis customer who will spend more on dining

[@BBkhadka]

thank you for great lecture

[@bkrai]

Most welcome!

[@uhsay1986]

Hi Sir , i have a retail train data set where i need to predict if a store should be opened or not in a respective location. I removed NAs from the train set , trying to apply glm function ( store~. , data=train, family='binomial' ) .. even after waiting 5-10 min i dont get any output .. the data set consist of character , int columns.

[@bkrai]

You will have to look at the structure of your data and make sure response variable is of factor type.

[@rohankulkarni8613]

Please let me know if we have data visualization on this data ? like in tableau or any other software ?

[@bkrai]

For data visualization, you can try this link: kzbin.info/www/bejne/pJqlZnRupsZrbqs

[@sayamnandy5855]

I have a question sir..Should i check multicollinearity (Vif) while performing the logistic regression? If any of the variable's vif value is greater than 2 then i will remove this variable from my model. Can i do that?

[@bkrai]

Yes, you should be able to do it.

[@shivamparashar...9536]

Sir when I uploaded a data set then it doesn't take all data ..it is leaving a few rows. Please tell me how I can upload a dataset. Thank you

[@bkrai]

How many rows your original data has?

[@s9438679525]

Hi sir, Please explain the use of type='response' in line number 23 Thanks

[@bkrai]

The type="response" option tells R to output probabilities of the form P(Y = 1|X), as opposed to other information such as the logit.

[@valeriasanchez4910]

Thank you!

[@bkrai]

You are welcome!

[@yekhtiari]

Loved all of your videos.I've learned lots of good tricks in R with your videos.Could you tell what I did wrong the following code returns the same index?(I am thinking this way is not safe for splitting dataset into training and testing) x

[@bkrai]

You have a very small sample. Try sample(2,length(x),replace = T,prob = c(.5,.5))

[@yekhtiari]

@@bkrai How about using : idx

[@irfaneditzstatus9760]

Hi Sir, I am analyzing the data based on the traffic survey. I have Age, Gender, TripDist, TravelMode, TravelTime, DepartureTime, LectureTime information. What is the meaning of factor and margin in regression Modelling. can you help me in that. Thanks in advance

[@bkrai]

'factor' is another name for a categorical or qualitative variable.

[@muralitharanrap4534]

Can u plz explain . what would xtab does ?

[@bkrai]

It is for cross tabulation or for making a 2-way table shown in the example.

[@myakaramyakrishna4400]

Why do you use xtabs ? How we do find a dependent variable in data set?

[@bkrai]

xtabs is for cross tabulation. A dependent variable is based on the context of data. In the example I have used, it is obvious.

[@merumomo]

Great video! What do we do if we do have "0"(zero) in factor variables?

[@bkrai]

Do you mean missing values?

[@merumomo]

Bharatendra Rai yes, I meant missing values. We fill in missing values with mean/median in numeric variables but I guess we need to remove missing values if it is in categorical variables?

[@bkrai]

For categorical variables you can go with category with highest frequency.

[@merumomo]

Bharatendra Rai thank you!!

[@lutfyabdulah1321]

Thanks for your share. It is very helpfull

[@bkrai]

You are welcome!

[@chandrasekharkona6462]

excellent work

[@bkrai]

Thanks for comments!

[@kumarvarma942]

HI Sir Great video's and easy to learn topics. I have small doubt don't mind. before dividing data into train and test . we need to do null values removable , finding outliers, scaling, EDA, then sampling .... could you please please share if any video on linear regression or logistic with combination of these steps. because we need to check all above conditions to predict best output . I am bit confusion on finding outliers(or remove outliers) and null values removable and scaling (min max or z-score) . Please please share any video it will helpful to us. Thanks in advance .

[@me3jab1]

Hello , Before u remove gre residual deviance was 369.99 when u rerun the model without gre it became 371.81 I mean increased , PLease in this case we should not keep gre even its not significant ? or the value change is negligible

[@bkrai]

That change is negligible. When a variable is not statistically significant, we should remove it.

[@me3jab1]

@@bkrai thank you Boss

[@bkrai]

welcome!

[@me3jab1]

@@bkrai if we have only one Y result ( not 2 as this example ) which Family type we must choose ?

[@bkrai]

If Y has only one value then that doesn't need a classification model.

[@arunshowri7829]

Hi Sir, I have a question. how to predict the target variable if we have many independent variables( eg: around 60). what we have to do if most of the values in independent variable are NA's. Please suggest me Sir.

[@bkrai]

60 independent variables should be fine. But before applying the method, you need to take care of missing values to prepare your data ready for analysis.

[@sayamnandy5855]

Apart from sir's suggestion..you can go for information value concept if you have plenty independent variable.

[@navneetjain2507]

What about the case when we have a lot of independent variables that have zero as a response or missing values?

[@bkrai]

For missing values refer to this link: kzbin.info/www/bejne/d5-an4OCf5WZqck

[@hyunjungariuka1686]

can anyone help? I got about NSP, but in the regression in appears only 2 rows which is suspect and pathological, but in my regression there is 4 lines like that. I think that it is suspect pathological and the other 2 what can it be?

[@bkrai]

For response more than 2 levels, you need to apply multinomial logistic. Here is the link: kzbin.info/aero/PL34t5iLfZddvv-L5iFFpd_P1jy_7ElWMG

[@divyasree3261]

Sir i need robust regression using r..can u please post the next video for robust regression

[@bkrai]

Thanks for the suggestion, I've added it to my list.

[@ayush612]

This is an awesome video Sir...thanks for uploading this!!

[@bkrai]

Thanks for comments!

[@vairachilai3588]

In Logistic regression, how to check the linear relationship between the logit of outcome and each predictor values

[@bkrai]

That's not needed.

[@vairachilai3588]

@@bkrai Linear relationship between the logit of outcome and each predictor values. If this condition is not met, logistic regression is invalid log⁡〖𝑝/(1−𝑝)〗=𝑏0+𝑏1 ∗𝑋 I read in almost many article. If possible can you explain for this case study

[@dhanielaritonang3273]

Sir I have a question, how if we have three levels of categorical response variable.. what 'family' should I use ?

[@bkrai]

For 3 or more, use multinomial logistic regression: kzbin.info/aero/PL34t5iLfZddvv-L5iFFpd_P1jy_7ElWMG

[@OrcaChess]

Thank you very much! Your videos create high value. Kind regards from Karlsruhe Jonathan

[@bkrai]

Thanks for your comments!

[@marcorinaldo4139]

Hi Bharatendra, could you please share the R code and data? Thanks a lot!!

[@bkrai]

They are available in the description area below the video. Here are the links: Data: goo.gl/VEBvwa R File: goo.gl/PdRktk

[@marcorinaldo4139]

Thanks a lot! i have overlooked them! thanks

[@flamboyantperson5936]

Sir I have one more request if you don't mind can you please make a video on K Nearest Neighbor KNN? Thank you so much Sir.

[@bkrai]

Thanks for the suggestion! I've added this to my list.

[@flamboyantperson5936]

Thank you so much Sir