Missing Value - kNN imputation in R

This video discusses how to do kNN imputation in R for both numerical and categorical variables.
#MissingValue Imputation
#KNNimputation
#MachineLearning

[@malvarez8588]

Nice explanation! Just an advice regarding the last step: You can also use the argument "imp_var = FALSE" in "kNN()" to skip the additional logical variables.

[@MrGozobozo]

You are awesome Mr. Gourab! Thank you very much for the simple yet so helpful tutorial!

[@nhosurkar]

Excellent explanation. Very simple and methodical

[@abrammakgalemele3491]

Thanks. Great demo!

[@niveditapaul2921]

Very informative video.

[@preeyank5]

Thanks for the videos dude, very informative.

[@RobertoSolanoM]

nice, short video! thanks for sharing

[@kukulemon]

Why did you use k=6 and how do you identify the most applicable value for this parameter?

[@TM-ip7vm]

Thanks. धन्यवाद

[@parijatkumar339]

Excellent!!

[@jindai8827]

Very Helpful! Thx

[@Protect_the]

Thanks sir! It was quite helpful. Just wanted to know for categorical variable imputation is there any other option available in R? EM or MI won't work because they treat even categorical variable as continuous ones which further distort the categorical variable values

[@meghalgandhi4357]

Great. That helps. Thanks a ton

[@larrylai4080]

Thanks a lot! In addition, how to deal with the following warning messages? "In [

[@APPUIT2KUTTU]

Thanks for the video. please explain how missing values for POSIXct format variables can be updated.

[@elmonovagales2929]

I dont know if the error I am getting is result of a lack of maintenance of the package. What it does when I impute, is to create a new column with a Boolean output (True for NA's and False for real values). Any idea?

[@ravindarmadishetty736]

Useful video

[@harishdhanarajan3297]

Under what assumption K was taken as 6? Basically, K has to be selected based on the elbow criterion / Sum of Squares and in order to plot it we require a data-set without NA's, I might be wrong on how to find k value but is there a way to determine the K value? Thanks in advance and great video BTW.

[@AliNavarro-tq6zy]

how do you identify what is the best value for the hyper parameter K?

[@ayushdudedon]

Should we treat missing values before removing outliers or after ?

[@pratikshah305]

after, as outliers will skew the treatment to a higher/lower value thereby changing the average/median or any other treatment that you are trying out

[@vedantsahay8750]

Hello Gourab, thanks for the video! I tried to impute categorical data using kNN as mentioned but after running the code, I still have the same number of blanks values in the column. Could you please provide some inputs on the same? Thanks!

[@juandavidlozadacalderon1356]

thanks.

[@petax004]

Can you make video for VIM package(GUI preferably)? I like the ability to visualize the missing values using VIM package along with imputations.

[@Protect_the]

start with install.packages(VIM) on R 3.3.0 version, it will keep on guiding you which additional package to add. Keep on adding all supporting packages as per pop up messages in R. It will work.

[@sainaveenbali]

can i impute this in panel data and then proceed to normally i.e. calculate fixed and random effect model and then apply hausmen etc. Please help

[@antoniorodriguezandres161]

Did you find out for panel data?

[@chonthanyayosbuth4648]

Thank you very much. Do you have other method ???

[@Mythical_motion]

Thanks

[@maticslearning_science]

hi prof.... how do you solve nominal and numeric attribut data with missing values data?

[@akrsrivastava]

The extra columns that kNN created were to keep track of imputed values. Isnt that so?

[@SA-gt8lx]

i need imputation for a data with 1.7 million rows and 27 columns. my pc cant handle this. what to do?

[@rookiedrummer6838]

Hi Gourab thank for the explanation, i need small information ... how do we decide the value of k?

[@TheFreezwater]

There is no standard as such. You need to give a couple odd choices and see how each model performs and try to choose the one the best fits your need. This could change as source data changes.

[@rajdipsaha]

Here we can take k = round(sqrt(n)), where n is the number of observations.

[@ayeledesalegn5367]

For further any body can see the vedio kzbin.info/www/bejne/fWPVkmqBqpiYfa8

[@fardazay]

there is no library(VIM) on my R, can you help me please?

[@gourabnath9789]

Did you install the package? If not run: install.packages("VIM") Then load the package library(VIM)

[@khachiakhadija7738]

I get this error :Error in `[.data.table`(data, , sapply(.SD, function(x) all(is.na(x))), : Some items of .SDcols are not column names (or are NA)

[@rookiedrummer6838]

I a also getting the same error

[@vaibhavkabdwal6275]

use variable=c("NAME OF COLUMN)"

[@miguelescobedorivas9763]

I already installed VIM package but its stil showing me this message. > library(VIM) Error: package or namespace load failed for ‘VIM’ in loadNamespace(j

[@hans4223]

ok

[@nhosurkar]

Excellent explanation. Very simple and methodical

[@kennetholelewe9932]

library(VIM) can't run on my computer. What could the problem be?

[@gourabnath5076]

+Kenneth Olelewe Probably the VIM package is not installed in your system. Try installing the package VIM using install.packages("VIM")

[@rishabharora5072]

@@gourabnath5076 Hi, even i am facing this issue. When i try to install this package it gives an error. Is it this VIM is under some other package?

[@gourabnath9789]

@@rishabharora5072 Hi Rishabh! No, VIM is an independent package. If install.packages('VIM') does not work check if ur version of the R software is compatible with this package or not. In that case upgrade R and try again.

[@rishabharora5072]

Gourab Nath could you please tell me any other package that can run on 3.5.1 version.. need to perform imputation for a project. Will be very helpful. I have some 30+ column and 6K + rows

[@gourabnath9789]

@@rishabharora5072 For your version VIM would have supported. However you can try using other imputation methods like MICE using the mice package in R.