How To Handle Missing Values in Categorical Features

Hello All here is a video which provides the detailed explanation about how we can handle the missing values in categorical values
You can buy my book on Finance with Machine Learning and Deep Learning from the below url
amazon url: www.amazon.in/Hands-Python-Fi...
Buy the Best book of Machine Learning, Deep Learning with python sklearn and tensorflow from below
amazon url:
www.amazon.in/Hands-Machine-L...
Connect with me here:
Twitter: / krishnaik06
Facebook: / krishnaik06
instagram: / krishnaik06
Subscribe my unboxing Channel
/ @krishnaikhindi
Below are the various playlist created on ML,Data Science and Deep Learning. Please subscribe and support the channel. Happy Learning!
Deep Learning Playlist: • Tutorial 1- Introducti...
Data Science Projects playlist: • Generative Adversarial...
NLP playlist: • Natural Language Proce...
Statistics Playlist: • Population vs Sample i...
Feature Engineering playlist: • Feature Engineering in...
Computer Vision playlist: • OpenCV Installation | ...
Data Science Interview Question playlist: • Complete Life Cycle of...
You can buy my book on Finance with Machine Learning and Deep Learning from the below url
amazon url: www.amazon.in/Hands-Python-Fi...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY KZbin CHANNEL

[@mohitupadhayay1439]

This was such an amazing life saver. I didn't even knew I had this question and the video just popped up. Didn't find this tutorial anywhere else.

[@doop9134]

I was stuck for days trying to figure out how to predict missing data using ML. This helped me understand so so so much better! 😍 Thank you so much!! 🙏💚

[@aksontv]

Finally got right man to learn data science and ML. Thank you sir!

[@gabrielburgos2533]

You are the MVP, when no one has the answer, you do.

[@soumikchakraborty90]

You are just awesome bro. Please make a video on AIC, AUC, ROC curve.

[@duvanmartinez8586]

Great work, you're awesome, you're the best youtuber I've found.

[@abinashkumarsinha8958]

This helped me a lot in my project work. Very useful and very well explained.

[@keshavbansal5148]

started this playlist today, loving it

[@pallabsaha4098]

Very well explained. If you could show the same on a dataset and code that would be very helpful. Thank you sir for your videos. Love them all.

[@Susa270]

Hello @ Krish Naik Hope you are doing well 🙂 First of all would like to thank you for such knowledgable videos. Most of the times your videos are really beam of hope. Can you please let me know where can I check the actual coding for the above mentioned concepts. It is a little difficult to get it in live scenario. Please guide, a humble request.

[@sandeepnallala48]

doing a great work Krish. thanks a lot. Loved your Videos : )

[@AmitYadav-ig8yt]

One more question- in some data set we find columns with many categories like Cars name column will have many cars name..In such case if we use this Unsupervised technique to create clusters, Won't it be too many clusters ?

[@anandacharya9919]

Thank you for this video. Please also make video how to handle missing value and Outlier in continues variables.

[@amedyasar9468]

it was quite short explaination and nice points to undersdtand. Tanks!

[@another_hindu]

Hello sir, maybe I am here too late but I still hope that you would acknowledge this question as it might be of immense value. I have a disputed question which basically revolves around knn imputer, scaling and the concept of data leakage. As the knn imputer works on the principles same as knn algo, it does share the pros and cons of knn algo, right. So wont it be better to simply scale the data first ? Also, in case I am separating out the train and test data in order to avoid data leakage, should I split the data and then scale, impute ? Or should I impute and then split,scale it ? In case I split first...which is the most common preference which stats should I use for the user input. And lastly how should I handle the label encoded columns if any ? Nobody is discussing on this when it is one of the most imp problems a person would likely face. Can you please make a video on this ?

[@tumul1474]

thank you sir ! amazing video as always

[@thatguyadarsh]

Amazing !! Use ML model to predict the NaN values.. That is clever sir.

[@chandrasekarank8583]

Sir what if i can label encode the data then i can do a simple imputer which will replace the nan values by the mean or median as i wanted. Sir please tell me whether this is a way to do

[@hv3300]

Excellent video, as usual.

[@muzamilshah8028]

lets consider i want to predict value for f1 & row 2 as you have mention but what if we have also missing value in f2,f3 but not in same row ..what will we do in that scenario ????

[@shivambhayre5056]

I have no words to say just a thanks🙏

[@ankurbanerji6605]

Great explanation sir! Can you explain how to handle the missing values for multiple columns in a dataset

[@andyjackson4563]

Thanks for explaining these methods

[@AmitYadav-ig8yt]

Sir, Can we get code for Create a classifier algorithm method for Missing value?

[@lukaszmichalak9985]

Don't you increase correlation between features with those methods? If so - what that will bring to the output model - to the prediction?

[@aronpollner]

Is there a Multivariate Imputer implementation for categorical values like a class from sklearn?

[@Geethu_Mohan_DA]

Easy to understand. Thank you

[@mohiuddinshojib2647]

that is really informative

[@shaileshsahu9551]

Please add a video in the Data Science and ML playlist of how to create our own predictor or estimator classifier algorithm to predict both categorical and continuous variables.

[@daniellazarolazaro1033]

Thank you so much, this video actually helps a lot when you just got started like me hahahha, as I was saying, thank you so much for this great great great work!!!

[@madhurchaudhary5109]

Hi Krish, This is well explained!! I have an ID column which has unique value but for some records, ID is null how I can handle this type of data.

[@itsmoolya]

This is a good explanation!

[@user-vy4jo3lt2v]

If we want to apply classifier algorithm on multiple columns then its possible ?

[@divyaharshad9985]

For technique 3 will it lead to multicollinearity in the data?

[@ZUBINABRAHAM]

Thanks for the video it was informative. Can we use KNN?

[@fahimekheradmand5880]

Excellent, Thank you

[@abhipraydumka8587]

Can you tell me how to assign a unique cateogry lets say U(undefined ) to missing cateogrical data

[@theoutlet9300]

since we are using output to predict our feature and then feature to predict our output, wouldnt it cause problems in prediction?

[@abdulhakeem4715]

clean explaination

[@preetnandeshwar5331]

which missing catgorial method suit for which data set and why?or we just have to use it like HIT AND TRIAL METHOD? Plz anyone help me .I am begineer

[@mitultank7872]

If I have the missing values in numerical column, and I want to fill that based on other categorical variable column . Then how can I handle that?

[@Nursin-rg1ey]

thanks very much sir

[@ommehta4501]

If we have date categorical feature and have some missing values, please tell me how to do with this

[@Saikrishna-lx9it]

Hi bro can you make one end to end chatbot video using rasa nlu, which is useful for all who are interested in nlp.

[@raghavkumar8333]

Sir, I have a student attrition dataset where I need to predict the reasons for student dropping out in 2nd year who got admission in 1st year. An year consist of 2 terms and I have grades of student (a,b,c,d) in 6 different courses in 1st and 2nd terms now most of these grade columns of 6 different courses in 2nd term are missing. Intuitive I think it could be a reason for dropping out. My question is 1) Should I impute missing values in this case because it is possible that it is not missing those students already dropped out. So, should I create dummy variables 2) If I impute missing value what technique should I use to impute those missing categorical variables

[@aditya_baser]

Here, you only had one categorical column. What if you have multiple categorical columns, how do you go about with the missing value treatment in that case?

[@MegaJaivardhan]

love you bro.. could you make a video AUC and ROC curve?

[@kumarraju2923]

How the initial clusters are selected for missing values

[@amitjajoo9510]

sir thanks for making feature engineering playlist.

[@anuragmishra6262]

Can you please show practical implementation of the same. Thanks 😊

[@tahamansoor599]

its great it would be better if u show us a hands on the dataset

[@clivefernandes5435]

Is method 3 widely used ? Never heard of it

[@jaiminshah143]

How to handle missing(NaN) values in column having binary data values i.e Just 0 or 1 ?

[@madunishant6052]

Thanks! 😊

[@Analystmind]

What if my model's missing values are not categorically it's number

[@RAJI11000]

Sir how can impute if feature value like 100 mbps

[@chirumadderla8129]

If there are several missing values in the solar radiation data during the night times and early morning hours how to handle them .The dataset I considered is of one year

[@bismeetsingh352]

What do you do when you have missing values in textual data?

[@sachinborgave8094]

Excellent Sir, can you please provide a python source code i.e. how to fill missing category data using logistics reg

[@CheeseKransky12]

Thanks krish

[@RajaKumar-ne9bt]

Why we are skipping the output when doing clustering?

[@AmitYadav-ig8yt]

Just a request...May you please upload codes for this also..-, I saw in many videos codes are missing for techniques..it will be very helpful if you provide us code. Thanks a lot

[@nasiksami2351]

Amazing!

[@sachinborgave8094]

Hello sir... Please make a video that how to fill missing categories using logistic regression...

[@AutitsicDysexlia]

This is what I did in DAX, but I did it in a more complex way... because I was using DAX. But it's effectively a RandomForest method that I used.

[@hindajjouri9151]

thank you

[@VikasSharma-ye7pu]

Hi krish ... Pls make video on in explaining 2 kaggle competition projects ...

[@akshayvilayatkar7985]

How we can handle alphanumeric missing values in dataset. I can not got out of this problem ,Please help krish

[@sadikbilal5149]

Nice , plz u have code to implement that techniques?

[@ele_wings7521]

thank you sir...

[@Raja-tt4ll]

very nice video

[@RK-un6ou]

Why do we fill NaN values with mean or median? And why does it won't effect the dataset Can you explain a bit in this?

[@ashokpalivela311]

thank you😍

[@chinmaybhat9636]

Can you Share the Same thing by taking one dataset and showcase the same

[@RishikeshGangaDarshan]

How to handel in regression oroblem

[@sriraj8392]

sir will u teach offline classes ...?

[@saurabhpathare4157]

I am always reluctant to delete or use mode for categorical values. This video explains a lot. Good approach! In technique 3, which classifier do you recommend for best efficiency?

[@riteshmukhopadhyay6922]

KNN, there is no particular ways as such it depends on the dataset

[@192Kiran]

Krish . could please do with datasets

[@sandyjust]

Great explanation of the concept. With unsupervised technique we might be in situation that both male and female falls under group 2. Then what would our approach?

[@kaustabhmandal7483]

I have also observed that in this video. You can put the the category with max frequency in that cluster.

[@shaikhkashif9973]

Sir pehle outliers fill yah null values fill karna chahiye ols answer

[@pankajkar2008]

pure concepts

[@1a17890]

Sirji can you kindly show how it's done

[@napoleonx5259]

كفو كريشنا ❤

[@sandipansarkar9211]

finished watching

[@ashwinkrishnan4285]

If we apply classifier algorithm to predict the Gender feature if it is male or female through other features including output feature as well, in training dataset and get the missing values of gender feature (Test dataset), and then finally when we go for the model to predict the classification of output hope it would be influenced or the data leakage would have happened as we considered that to fill missing column values? Please clarify on this point Krish..

[@chirathabey7729]

It won't as much because even though we are training including the output feature, it only used for predicting the missing samples ONLY. Considering the fact that there is much less missing samples as compared to rest of the samples. If the missing samples are considerably high and have in many other features then it will certainly create a bias on the final prediction.

[@AmitYadav-ig8yt]

Sir, U took data set which has a missing value in just one column. You told about Predicting missing value my using other columns as Training set. Let's say we have a data set in which every columns have some missing values..In such case which columns should be use to predict missing values?

[@kannadarecipes-6626]

Following

[@habilmohammed5127]

Following

[@leilafakhraei78]

Following

[@barnadipdey8486]

yes Amit I have the same query ,if you had solved this please dm me.

[@mohammadarif8057]

Sir can you provide a practical approach with complex data set ...that would be great thank you

[@janinajochim1843]

Thank you for the video! Would you happen to know what to do in cases where the value is"Missing by design". I have a case where I am using the variable "Father's reaction to pregnancy" -- it has missing values for participants who did not know the father of the child because they didn't get this question :/

[@sawradipsaha5377]

May be you can consider that as a different catagory.

[@AmitYadav-ig8yt]

You said to Create a classifier to predict the missing values. What to do if we have Linear regression problem and Missing values there?, Should we create classifier for that too? Please response

[@chirathabey7729]

Yes, if you are trying to predict the missing value which belongs to a Categorical variable. Because when you are predicting missing value, your output variable will be the missing value variable and rest of the variables will become the input variables. You can think of you are trying to solve an entirely independent problem.

[@dineshkumar-kc7vt]

im unable to overcome this problem. I have initially done is get_dummies for the Dataset and i want to handle the missing values but i'm getting error so as TypeError: '(slice(None, None, None), slice(0, 2, None))' is an invalid key Please Help Me

[@chirathabey7729]

Before you apply One-Hot-Encoding, do the missing value treatment first

[@Justme-dk7vm]

Sir why do you have the same voice as my college chairman? 😩💓

[@analistaremoto]

Niiiiiice!

[@junaidlatif2881]

But how to apply!

[@archanapereira1333]

How to identify dependent n independent variables in a dataset ?

[@chirathabey7729]

It depends on the problem description. It describes what the problem is. So, your output variable / dependent variable will give the answers to your problem. Rest of the features will become your independent variables

[@shivambhayre5056]

If it is in quantitative variables we can replace missing value by mean

[@AmitYadav-ig8yt]

Is it a question?, If yes, Then Yep You can take mean to replace Quantitative missing values

[@vasusharma1773]

sir if you could just show this in a code, it will be very helpful

[@cutyoopsmoments2800]

Bro I want to make my career in Machine Learning. Kindly guide...

[@jaypatil4786]

I have one easy question ...but I not remember it now please tell me to view how many missing values in dataset

[@saravananm2280]

dataset.isnull().sum()

[@nhprml6324]

we can replace missing values with corresponding feature's mean value.

[@arjyabasu1311]

Sir please upload the implementation of these methods !!

[@harshtiwari8765]

can u send me the notes for feature enginerring which was given by Krish naik ? Help is appreciated

[@martinlyuba5105]

Great tutorila. your email please