Python Pandas Tutorial (Part 9): Cleaning Data - Casting Datatypes and Handling Missing Values

[@coreyms]

Hey everyone. Hope you all had a great weekend! I will be traveling to Vancouver this week to visit a Quantum Computing company and learn more about the work they're doing, so I'm not sure when the next Pandas video will be ready for release. I will be working on it while I'm there, but I likely won't have it recorded and released until midway through next week. Let me know if anyone has any questions they would like me to ask them about Quantum Computing!

[@harshvardhan1156]

Hey, Corey. Thankyou for everything. I am not from Computer Science background, Out of curiosity I started learning to code and here I am now, has done more than 20 datascience project. Your videos are literally best, I have taken some courses for high price and I can un-undoubtedly say that your way of teaching is way more interactive, complete and easy to grab. I just want to know how you plan for any course, like in 1st or 2nd video You said that I will cover this topic in later videos. So do you make whole content, practice it? deepdive in it and make your own order and then start teaching? It would be very helpful for me if you share about how you prepare for any topic. Thank you very much Love from INDIA

[@harjotsinghbaidwan2204]

I have many times seen while using dataframe that column names are not at same level and this creates an issue during extraction of values. Do you have any idea about it?

[@JiminPark-ld2xx]

How do I download dataset after cleaning my data using Jupyter notebook online? Plzz ans..

[@malikdiallo9976]

I like this series in pandas. thank you so much Corey.

[@ahammadshawki8]

Please make a playlist on numpy after pandas.

[@ChetanAnnam]

Yeah please do that

[@saravanannatarajan6515]

Corey you're teaching is awesome!!! Much appreciated!!! Expecting series on Machine Learning/Deep Learning in the near future...

[@corben3348]

Good teaching is an art... This playlist is so helpful ! Thank you for your work !

[@gauravmarwaha8466]

this series on pandas is the most complete and informative series ive found till date...!!!

[@ashishdeora8522]

Thank you Corey for this. My parents urged me to join your community. They are saying you are doing wonderful job. Thank you Corey for enabling us

[@sayantanchakraborty75]

Best series on Python Pandas . Thank you so much Mate. Love from India

[@adamgdev]

You never disappoint!! And I never have to speed you up because you keep a great pace with no BS! Thank you!!

[@ishanpand3y]

This is the most amazing series on Pandas ever. I just finished watching number 9th. Sir thank you so much providing such great content. 🧡🤍💚

[@benhancock1541]

Thanks for this Corey - your tutorials are always great! I've been using pandas for almost 2 years and still learned stuff 👍

[@kuls43]

11:36 we can use df.replace(['NA', 'Missing'], np.nan, inplace=True) instead

[@AtlasIndustries101]

could've used in other df.replace(...) line too. But I think he is trying to keep it simple for us to understand it easily.

[@srivathsgondi191]

Now thats a lovely explaination, i like how u showed the function can be used in different scenarios!

[@minxxdia1132]

wow, this is the best playlist for python pandas. thankyou so much!

[@gagansoni9665]

i understand your pandas tutorials very clearly. this is helping me a lot. thank you so much corey. i wish to see your tutorials on machine learning using python.

[@codegeek8256]

Hi @ Corey Schafer I am very with your teachings, these are great building blocks towards data science, i hope one day we arrive there.

[@mapa5000]

You really care about making a video addressing many scenarios and possible issues … that’s phenomenal !! … I really appreciate it … thank you so much!!

[@stanislawjarzynski6133]

You're a great teacher, Corey!

[@YeekyYeeky]

can't wait for your numpy series , this channel is gold , Thank you Corey

[@kameshinipillay4587]

Thank you, learning so much :)

[@zixinlee2165]

Thank you so much for creating these videos!! They're really valuable for self-learners like me.

[@TopicalAuthority]

Great lesson!

[@darkmaraux]

This video was so smooth! Right in the point! Thanks!!!

[@finncollins5696]

Learnt a lot so far. Thanks so much Corey,.

[@zzzorgjanbatist564]

As usual Corey best of the best!!!

[@andreykaok9497]

Brilliant tutorials on Pandas! Very much looking forward to the time series lessons.

[@rockeyvalley]

Great stuff Corey!!! Keep up the good work!

[@anubhavrauniyar3192]

We love you Corey Schafer!!!! Lots of love from India🥰

[@gayatriwaghmare6293]

The series is very helpful to me. Thank you sir.

[@002_priyanshugoswami5]

love you coreyyyyy best channel

[@analyticswithothello8213]

Corey, you are teaching the best!

[@ABDULKARIMHOMAIDI]

Thanks man for such valuable series of videos, please add more video on new features on pandas !!!

[@davebeckham5429]

Many thanks for sharing excellent tutorials Corey.

[@rauberhozenplotz7009]

Great content - great style of speaking and explaining - thank you!

[@Ian-bb7vv]

I had to say, thank you!! I think you guys are really helping to fill the unequal educational resources between the rich and the poor. Great job and I hope you now that what you are doing is really meaningful

[@danielflorea3001]

Simple and clear explanations. Great job.

[@VikasGuptacherie]

Very helpful series with nice explanations !!!

[@alexthewebdesigner1856]

@Corey Schafer Something told me that I'hd better watch this video. Just when I thought that I'd sanitized a large data set, I realize now that there could potentially be some data (or missing data) that could crash my application. Great video. Thank you Sir!

[@ajinzrathod]

Corey you are great.❤️ Love from India ❤️

[@lucasbartomioli7861]

Man, i love you! Thanks a lot from Argentina!

[@kirannagar8295]

Hey , truly glad for your all series . If possible , please do make a course video on Pyspark .

[@stressfreetrading1341]

Love the way u teach. thanks a lot... Love from India

[@Al-Ahdal]

Boss, it is requested to kindly make videos on comprehensive data analysis series, covering all aspects in much detail, and covering all possible areas for data analysis. Your channel and vdos are awesome. Great work indeed...... 👍

[@ahmedhosny3855]

such a great work done by you , hope you all the best man

[@saraghafelehbashi5808]

much appreciated! could you please have more video like that? cleaning data and see the diffrent errors come with it! it would really helpful for juniors.

[@samratsengupta8881]

Thanks Corey, i have no words to say. As an inspiring data scientist, your pandas videos were really cool. I don't know if you will ever read this but this has helped and has put a smile on my 'confused about pandas' face. i have subscribed and will watch your videos for becoming a self taught data scientist. God Bless You

[@juancarcelen3437]

Hi Corey thank you so much for posting these videos. Your tutorials have helped me transition the concepts I know into actual useful code. I would like to test my progress and would really appreciate if you can put out a link with some data analysis projects (i.e. a database to download, questions to answer using data analysis, and the code that was written to answer those questions). Thank you so much and keep the videos coming you're an amazing teacher!!

[@Shkkmj6868]

It's very useful .You are great at articulating . Thank you so much .

[@kingjoshuamanatad2140]

In 27:28 of the video. For a one liner code. df['YearsCode'].replace(['Less than 1 year','More than 50 years'],[0,51]), inplace=True). Correct me if I'm wrong I'm new to Python. But great video again Corey! Hats off!

[@vladimirwimmer11]

this does not work anymore as Corey was mentioning, rather like this>> df.replace({'YearsCode': {'Less than 1 year':0, 'More than 50 years':51} },inplace= True)

[@FakeAccount]

You're a legend, my guy.

[@robertmnganya7533]

Excellent teaching. Thank you.

[@mikkybricks]

Thanks Corey

[@dadoll1660]

This is gold.

[@manishgpt25]

thanks a ton for this series..helped a lot in clearing concepts!!

[@njgaming4422]

instead of replacing separately you can just pass the list of strings that you want replace E.g : df['YearsCode'].replace(['Less than 1 year','More than 50 years'],[0,51],inplace=True)

[@mohammedkaifmirza7585]

Amazing tutorial 😍👌

[@interestingstudies4422]

Amazing video...solved my problems ☺️☺️🙏🏻

[@aegystierone8505]

Please do a video about your visit to the Quantum Computing trip in Vancouver!

[@teetanrobotics5363]

I love your tutorials. Could you also make tutorials for scipy and scikit learn?

[@NikitaSharma-bs4gg]

That was such a good video- thank you for sharing

[@andr101]

great series, thanks!

[@noureddineettayyeby5210]

Thank you

[@haiderali2050]

Thank you so much, i have learnt a lot and able to automize my daily Excel routine work

[@JoKaR80-d5r]

These are awesome! Thanks a million!

[@ironpolux]

Great vid, pls do one on multiple indexes!

[@gauravmarwaha8466]

good video again..!! thanks a lot

[@mohamedikbalguetout32]

hey bro always I fond the solutions in your videos thanks man

[@stephanie_ong]

Thanks again for such a helpful video.

[@muntadher8087]

Thank you so much!! You are the best

[@bharaths1396]

Your content is awsome....! How do replace nan values with other values only in a particular column? Please Help Thank You

[@nikhilb3880]

I love this series man, more than you could expect. If I may ask, what state and country are you from? Because I saw snow on your 2nd channel and now I'm confused about whether you live in the USA or in a European country. Thanks again for this series

[@coreyms]

Hey there. I currently live in Greenville SC in the United States. The snow videos were likely from Boulder Colorado where I lived for several years.

[@varunkrishnaKyathanpally]

Thank you , excellent tutorial as always :)

[@quoit99training83]

amazing series - hi Corey, how many PARTS u think will end up in this playlist? Thank you for helping the community :)

[@arkahm]

Great video! How about a video in spitting data and passing the split into a function? That would be great!

[@maheryagub]

Plus 1 for using Vimium plugin at 18:55

[@litan5006]

Good pandas video. Thank you

[@shivavijaya1537]

Hi Corey, please post a video on python sys module

[@codewithluq]

Thank you Corey again. My resume is getting more interesting everyday. Viva

[@ADNANAHMED-eo5xx]

Thanx a lot

[@pandeyUkislay]

hi Corey, at around 16:14 you say that "age column is a string, because it says object!" this is confusing me coz iirc everythin in python is an object, please correct me here

[@athas12]

for the last part of the video, you can actually create two lists and use these lists in replace method to change all values at once. It is slightly easier especially if the df has multiple values to replace

14:57 - Casting / fixing data types

[@saqibhussain1354]

Great video - I wonder if you can do a few on the business side like freelancing and how to get clients as python developers?

[@dhanraj112]

is brilliant give certificates after completion of course?

[@kinjalvora256]

Hi Corey, Thanks for the awesome series. While I have not yet finished the series, I would like to know, how we can deal with duplicates. If you have a column let's say with duplicate apps and the apps have reviews, size, installations and you want to let's say get a mean for the reviews, take the first size and sum of the installations and merge the rest of the columns for those apps as they were, like Ratings. How would one do that?

[@prasad1686]

Hi Corey, your videos are "the great". I am beginner can you please tell me how to get "cheet sheet" or ".py scripts" of your video playlist "Python Tutorials" "1 to 136", to speedup learning as i am slow in typing. Thank you.

[@muntadher8087]

useing this func ( df.fillna("Unfilled", inplace = True) ) to replace the missing values is good practice I belive, for me it's easier than replace and more dynamic

[@KevinTempelx]

Thank you!

[@ADNANAHMED-eo5xx]

Please continue the series sir

[@harishrudroju1379]

Hii corey, can u plz make a video on how to bypass captcha while scrapping a web site

[@KimJennie-fl3sg]

This also work if we want to drop a column if 0 and 1 index have NaN df.dropna(axis='columns', how='any', subset=[0, 1])

[@bobsalita3417]

Can you use join() or merge() to do multiple replacements?

[@SusanAmberBruce]

Corey, do you happen to know what Linux distro's ship with python 3 currently?

[@nayeemuddinmoinuddin2186]

@Corey Schafer - Please do a video series on PySpark.

[@md.abdullahalmasum4942]

thank you sir .

[@shadmanmartinpiyal4057]

excellent!

[@shanghainewbison7687]

Another way to change the dtype of df["YearsCode"]. def change_to_float(string): try: result = float(string) except: result = np.nan finally: return result df['YearsCode'] = df['YearsCode'].apply(change_to_float) df['YearsCode'].mean()

[@vipulsuthar3796]

Thanks man! Was stuck on this... really appreciated!!

[@md-ayaz]

@Corey Schafer Can you make video on getting started on Open Source Contribution?

[@d_omar1468]

great job brow

[@stayinawesum]

can you make video explaining: primitive data types vs data types vs adt vs data structure