Python Missing Data Filling Techniques - Simple Methods To Handle Missing Values

Missing data is probably one of the most common issues when working with real datasets. Data can be missing for a multitude of reasons, including sensor failure, data vintage, improper data management, and even human error. Missing data can occur as single values, multiple values within one feature, or entire features may be missing.
It is important that missing data is identified and handled appropriately prior to further data analysis or machine learning. Many machine learning algorithms can’t handle missing data and require entire rows, where a single missing value is present, to be deleted or replaced (imputed) with a new value.
If you haven't already, make sure you subscribe to the channel: / @andymcdonald42
----
Books I Recommend:
As an Amazon Associate I earn from qualifying purchases. By buying through any of the links below I will earn commission at no extra cost to you.
PYTHON FOR DATA ANALYSIS: Data Wrangling with Pandas, NumPy, and IPython
UK: amzn.to/3HNycJ9
US: amzn.to/3DL7qPv
FUNDAMENTALS OF PETROPHYSICS
UK: amzn.to/3l1PgSf
PETROPHYSICS: Theory and Practice of Measuring Reservoir Rock and Fluid Transport Properties
UK: amzn.to/30UNWZS
US: amzn.to/3DNqBbd
WELL LOGGING FOR EARTH SCIENTISTS
UK: amzn.to/3FHsbfn
US: amzn.to/3CILAuE
GEOLOGICAL INTERPRETATION OF WELL LOGS
UK: amzn.to/3l2v2HV
US: amzn.to/30UOTkU
----
Libraries used in this video:
pandas: pandas.pydata.org
missingno: github.com/ResidentMario/miss...
-----
Thanks for watching, if you want to connect you can find me at the links below:
/ andymcdonaldgeo
/ geoandymcd
/ andymcdonaldgeo
www.andymcdonald.scot/
#missingdata #petrophysics #machinelearning #geoscience #missingno #python

[@angelomaurodiez4713]

Thank you for your videos!! m y carreer as DS has been boosted a high level!! GREETINGS FROM ARGENTINA

[@mohammadkeshtkar9655]

Thank you Andy for another useful video 🙏🙏

[@AndyMcDonald42]

No problem 👍

[@saramoeini4286]

Thank you so much. it helped me. but now i have a large dataset with lots of missing value. i think it's not good to remove that rows or fill that with mean value. does interpolation works good in this case? what solution you suggest?

[@hichemsevran]

Very informative tutorial, I just finished reading your article on Medium about Missingno am at Serie 11 now. Thank you very much