The A to Z of Missing Value Treatment | Data Preprocessing in Python | Data Science

[@meetuchandra7688]

Thank you for this in-depth tutorial. Your complete playlist is awesome and helped clear so many of my concepts.

[@prosmartanalytics]

Thank you! Glad it helped!

[@rachitmakhija9703]

thank you for this hands on tutorial !!

[@janaosama6010]

how will we know if the variables are independent or dependent on each other to choose the right method to fill missing values? is there a methods that we can use to know that? and if there, what are those methods .. thanks in advance

[@prosmartanalytics]

Generally the features captured for a given context tend to have some dependence, that's why some models are called Naive because they assume features to be independent. Therefore, in practice algorithmic approaches are superior compared to the typical mean and median based treatments. A lot of the preprocessing steps are connected, you may refer to our playlist titled EDA and data preparation. Hopefully most of your queries will get answered.

[@ryymtareq5027]

@@prosmartanalytics can we calculate correlation to know if the features are dependent or independent?

[@prosmartanalytics]

@@ryymtareq5027 Pearson's correlation for numerical variables which are linearly associated, but it focuses on two variables at a time, pairing all possible combinations. Alternatively, we can also look at VIF because it checks entire independent feature space. Likewise, we can apply Chisq test of indepedence for categorical variables. Please note, these are not the pre-requisites for applying algorithmic imputers.