Exploratory Data Analysis with Pandas Python

[@robmulla]

Chapters don't appear to be working for my videos for some reason. Here are the timestamps for the video: 00:00 Introduction 01:00 Imports and reading data 03:35 Data Understanding 06:40 Data Preparation 20:57 Feature Understanding 27:35 Feature Relationships 35:30 Asking a Question about the Data 40:00 Final Thoughts

[@oshogweikekhai5499]

amazing video, please can you make available the dataset used? thank you

[@Pluvo2for1]

Hi Rob. I'm new to notebooks. Could you plese explain why you don't need an explicit print statement to view cell output?

[@cocgamingstar6990]

This was one of the bad video and non industry knowledge

[@ABYHYDROS]

😮

[@saintsaens3517]

As a begginer in data this really opend my eyes as to how things works. Your explanations are very clear and I can feel how passionate you are. Great video

[@robmulla]

Glad it was helpful! I am passionate about it, and excited when I hear people are learning from my videos.

[@arunkumark6351]

So true,, though there are many projects and training videos outside. The way you think, step by step approach and the reason for doing so, is so relatable and feels very natural. Awesome video, Thank you so much

[@grecokap7881]

I am a beginner and I found it really really helpful. It the first totally clear tutorial I have seen so far. It made clear to me a lot of things. That you so much! I'm about to explore your Chanel. Hope to see more from you!

[@grandselenium296]

- import data Data understanding - filter columns by need - convert dtype of certain columns - rename columns - check isna in columns and dropna on row or column accordingly - locate duplicated rows in single or multiple columns - drop duplicated rows from dataset and reset index Data prep -univariate analysis of features - kde, histogram, box plot - use value counts to determine duplicates and unique values in feature - he creates bar plot for top 10 years introduced to highest # of coasters - he creates histogram to bin speeds of roller coaster and view their frequency distribution Feature understanding - scatterplot, pairplot, correlation, groupby - he creates scatterplot for speed and height with year based hue of points - he create pairplot to compare correlation between features, alongside hue from material type - creates a correlation heatmap for selected features Ask question - he uses groupby and query to create bar plot with sorted descending data on mean speed of roller coasters by location.

[@ViniBevilacqua]

Thanks

[@saptarshidey7672]

There are a ton of EDA videos on KZbin. This is one of the best I have ever come across. You just nailed it, Rob.

[@robmulla]

Thanks so much!

[@jz0401]

I agree with you 100%. Rob did a wonderful job.

[@sa-pt3kf]

By far one of the most clear and concise ways of teaching in a computer science related field I've come across in a while. I'll be binging all your tutorials for sure!

[@robmulla]

Whoa. I love this feedback. I'll try my best to keep them coming.

[@JHornsby89]

This is the second one of these I have now watched and coded along with! Genuinely awesome content, so precise and simple to follow. You make daunting tasks (for beginners getting into data) really accessible which is a sign of a great teacher!

[@robmulla]

Comments like this make me really happy that I made this video. So happy it helped you in your coding journey. Did you use the Kaggle notebook when you followed along?

[@darshantawte7435]

Lucidly explained. One thing i have learned that in order to be a great Data scientist what matters is your problem solving skills, understanding the business requirements and curiosity to dive deep into data (true to the name data scientist) . There is no need in remembering these codes as long as you know what to look for.

[@jackgarn8392]

This is such an amazing guide! I’m new to data analysis and had limited python exposure and have taught myself most of these things so far by googling or just reading the pandas documentation. Watching someone familiar with the process do it all together was really helpful and gave me a lot of insights as to how I can improve my skills and workflow. Thank you so so much!

[@chrisosomo2856]

I can’t get enough of your videos, especially the very hands-on practical approach to learning. Your explanations are clear and easy to follow along with. Please make more of these types of videos. You are definitely makes a change and contributing to the KZbin knowledge pool. Thank you so much.

[@Aarron-io3pm]

Hi Rob, this was super useful to me as a tired Excel veteran and python beginner. You explain and demonstrate everything so clearly, thank you

[@ahmetunaldi7601]

Rob is the greatest pandas teacher I have ever met... Thanks many many times for this wonderful video... Süpersin dostum..

[@jz0401]

Rob, I found your tutorials only recently and, to me, yours are one the the best out there - well-explained and highly effective. Thank you very much for sharing your expertise.

[@nigelkiernan1321]

This is a great refresher guide! Very nice coding style and I appreciate you using a simple Kaggle dataset to follow along. Great stuff - thanks!

[@mrahul2113]

Decade years of experience in churning data in xl and SQL .... Trying to get my hands on python... This video is the best I have come across on the internet. Thank you for letting us know a real life problem getting solved

[@robmulla]

So glad you found it helpful!

[@silver_soul98]

I have tried plenty of tutorials by now. This is the most precise and to-the-point tutorial so far. Well done.

[@shoaibahmed5616]

I completed my certification in data science but could not figure out where to start. Thanks for such a detailed and easy to understand instructions with very useful commands. It helps a lot. Keep it up

[@vietndk5437]

I did not practise pandas usually then I almost forgot the syntax or its application. Now I find your video with very clear instructions, it helps me remember better. Thanks alot

[@chess6802]

Perfect stuff what I love about this video is the simplicity and the clearness of the way you talk

[@robmulla]

I appreciate that! Thats how I learn best so it's also how I try to explain things.

[@chess6802]

@@robmulla your response reflect your both knowldge and wisdom please keep on 💞

[@itm1996]

Hands down one of the best tutorial I ever saw. Basic enough to follow as a newbie but demanding enough to be useful. ❤

[@Rantalytics]

wow... thank you so much rob. I come from a frontend background but just began a data analytics bachelor at SJSU. I was trying to grasp at a high level what DA might look like as it pertains to conducting an explorative project. This tutorial completely cleared up those questions!

[@AbdullahKhan-dl9lm]

Very crisp and concise explanation. Sometimes youtube videos can be better than paid courses.

[@wesleyweel8007]

The quality of your content is only surpassed by the ease at which it is to assimilate it, keep up the great content Rob, cheers!

[@robmulla]

Wow. Thanks for the positive feedback!

[@yjgg5882]

​@@robmulla bi L😅

[@yjgg5882]

🎉😊

[@rrestituti]

#1 Data science youtuber!!! You made easy to understand the basic commands e sintaxes. Thank you a lot, Rob. 😉

[@robmulla]

Tell all your friends. 😆

[@lashlarue7924]

Agreed, this is the best Python content on the entire internet, hands down. I'm going to be carefully watching these videos over and over for a long time.

[@tomparatube6506]

I dabbled in this 4 years ago at EDX. This is a wonderful refresher. Thanks Rob!

[@Paranorman2049]

Great work, bro. Thank you!

[@walterpark8824]

well organized, concise, very helpful to get grounded in Pandas. my explorations will continue. Thanks!

[@robmulla]

Glad it helped! Thanks for watching. Share with anyone else you think might also learn from it.

[@ZeuSonRed]

This was the greatest Tutorial I ever had. Thank you. Here I get to cnow about the corelation and some panda functions and ploting. But for Power counting and Corelation between Variables was very pleasend and satisfied my expectations. From Bulgaria Volga Sauvete! Thaank you ! 👑👑👑👑👑👑

[@jmmj5018]

thank you so much, you have made my EDA analysis easier and faster. :) also, it's easy to digest as I go along with the data you are working. thanks a lot. you are helping a lot of analysts or people who wants to study in data analytics. Great video, keep them coming.

[@sheshankjoshi]

This is perfect for my interview tomorrow. I just needed a refresher on how to approach the problem, ask right questions and then come up with exploratory options. Thank You so much for this video

[@chrismagee5845]

This is the best reference guide. I always find myself rewatching this whenever I'm cleaning a dataset.

[@robmulla]

So glad you find it helpful.

[@deneskalnoky7939]

This is a really good tutorial. I am new to Python and data analysis, and was completely lost. It was so hard to find a good, reliable source about it. This source just clarifies the basics for beginners so that I can start off with my own project.

[@sushanthpentakota4551]

hey I have a doubt..do we need to use pylab as the submodule of matplotlib or can we use pyplot...like does it have all the functionalities of pylab. I read that pylab is outdated and generally not recommended...btw really great explanation...loved it

[@alisonhenley2551]

Wow, what an informative fun tutorial. Thanks Rob!

[@robmulla]

Glad you learned from it and I appreciate the comment.

[@adamvoltemar420]

This is one of the best content related to Data Analysis and Python/Pandas, I am really glad I found it! Thanks!

[@walterpark8824]

Terrific introductory survey that answered so many of my questions, moving from SQL. Looks extremely efficient. Now, to plug into my data! Thanks.

[@robmulla]

Glad you liked it. Sql still has a place but when working with the data for EDA pandas can’t be beat.

[@TheGenerationGapPodcast]

I already know Pandas pretty well, but I still learn from this dude.

[@NeerajKumardeaf]

Thank you so much. I am happy for your teaching about EDA with data analysis for pandas. I am clearly explaining to you. I can continue my hands-on experience for EDA

[@ranahuzaifa147]

I was smiling at 39:23 . How easily you answered the question. Thanks for this amazing video tutorial.

[@robmulla]

My pleasure 😊 Glad you liked seeing it all come together at the end.

[@wahabamin6946]

Thanks Rob, you’re doing a great job for the data science community. Your videos here and on TikTok is helping me a lot in this journey. Thank you.

[@robmulla]

Love to hear that Wahab! Glad you learned something, and thanks for posting the feedback.

[@aydanlopresti2879]

Clear and applicable to any type of analysis. Thank you

[@Leerosasi]

As a beginner, found this quite useful and easy to follow through, your expalation is easy for beginners. Thank you 'Mwalimu mzuri' (Swahili word for 'Great teacher') 😊

[@100themagician]

Excellent video Rob, thank you!

[@robmulla]

Thanks for watching!

[@guilhermedesanctis]

Thanks for this lesson. It’s much valuable.

[@soumyadrip]

Wow so many things are covered, its a great tutorial for getting started with EDA.

[@robmulla]

Thanks @somuSan. Glad you liked the tutorial. It took me waaaaay longer to film than I expected but I'm happy with the result. I hope more people in the future find it helpful.

[@Eysh2009]

This video makes me feel glad to be alive. Great explanation, amazingly fast and on point. Thank you!

[@user-ei9jd7pw4s]

Thank you for the video. You have combined all my knowledge into one comprehensible picture.

[@anitagodwin8067]

I’m here in 2024 and this video has helped me a lot

[@secretperson8321]

This video is gold

[@bradleyfrueh2761]

Thank you so much. I appreciate the work you put into your videos. It shows.

[@robmulla]

I really appreciate the feedback! Please share with anyone you think might also learn from it.

[@hotbit7327]

Some of your other videos I found too fast paced, like about pandas mistakes, this one here has great content and also fantastic presentation. Thank you.

[@robmulla]

Thanks for the feedback. I’ve been experimenting with different editing styles so it’s nice to hear you prefer slower paced like this video. I’ll keep that in mind in the future.

[@irfanshaikh262]

This is really an eye opener stuff for rookies like me. Thank you Rob

[@robmulla]

Thanks for watching!

[@adarshtiwari7395]

I have been grinding through your videos lately in preperation for my data science job and you have been an absolute blessing! Thanks a bunch!!

[@robmulla]

Wonderful! Glad I could help.

[@hmx21]

Hey Adarsh! Other than this what else are you learning that will help you in the data science job, I'm also preparing for the same but kinda new to data science so any guidance would be appreciated. Cheers!

[@adarshtiwari7395]

@@hmx21 Hi Hemang. I'm a fresher in data science as well. I started with Python and statistics. Then moved on to EDA followed by Machine Learning algorithms. I then made a few projects on ML. Also tools like SQL, Power BI, Excel are preferred

[@hmx21]

@@adarshtiwari7395 Hey Adarsh! Thnaks for the reply, I'm done with EDA and made a dashboard using Power BI, and don't know how much machine learning or SQL is required for the role as I've studied SQL in college and know how to work with joins,etc. Any tips or resources you'd like to share would be a great help. Also from where did you learn stats for ds, whenver I try to learn stats online I get overwhelmed with the magnitude of tutorials.

[@adarshtiwari7395]

@@hmx21 depends on what you're going for. If you are interested in a data analyst position, EDA through Power BI is great but if you want to go done the data scientist or machine learning route you need to be hands on with Python. EDA using python is much more nuanced as compared to visualisation tools like Power BI. SQL is essential in all contexts so it's a must. But whether you should study machine learning depends on your career goal.

[@evansala7814]

Rob, I really like your videos. You are very concise with your language and make this content very digestible.

[@robmulla]

Thanks! Glad they are helpful.

[@shihaosun6861]

Thank you very much Rob for this wonderful walkthrough and explanation! Really Appreciate it!!!!

[@robmulla]

Thanks for the feedback! Glad to hear you learned something from it.

[@nunolopes3910]

This type of videos are amazing to follow, i am starting to use python for data analysis and i could not happier! Your channel is helping me alot, thank you!

[@robmulla]

So happy to hear this. Let me know what you would like to see in future videos.

[@nunolopes3910]

​@@robmulla One question i have is about the safety of using jupyter while working with company data. I am just starting to use jupyter and that is a big question that i'm sure other begginers would like to know to! Can you give your opinion on it? Thanks in advance

[@kryzzet]

Great video, the pacing was perfect to follow along without flashy visuals that disturbs.

[@AkramBedouiEPT]

while formatting column datatype, we can also convert 2 objects to 'category' and one int64 to boolean: df['Inversions_Clean']=df['Inversions_Clean']==1 df['Status'] = df['Status'].astype('category') df['Type_Main'] = df['Type_Main'].astype('category')

[@AkramBedouiEPT]

well, actually, converting 'Inversions_Clean' column from integer to bolean was a dumb idea as, later, we have to transform all categorical data to numerical (encoder or one hot encoding methods). But I do think that convrting from object to cetegory good to reduce the file size.

[@robmulla]

Yes, I have a whole video about converting dtypes to make dataframes more memory efficient. It does depend on how many unique values the categorical column has, but in general it's better.

[@AkramBedouiEPT]

@Rob Mulla yes, indeed, all the techniques I use in data exploration on Pandas are derived from 5 videos of yours ! Thanks a lot for democratizing the learning of data science, despite the tens of thousands of kilometers separating us 🥹

[@aayaanhasnain5143]

Hey Rob, really admired the way you explained complicated topics with ease!! Looking forward to learning from you more :)

[@robmulla]

Thanks so much for that feedback. I really apprecaite it.

[@thaanathaana4522]

Clear explanation for beginners.. will follow you more for tutorials

[@melomane01]

Hello Rob. I'm currently doing a graduate degree in business intelligence. One of my class this semester is Data Mining using Python. Your video was very helpful. Thank you.

[@DatascienceSiharath]

Thank you!!! I learned a lot from your course.

[@robmulla]

Glad you found it helpful!!

[@ark9255]

Thank you for being extremely clear and concise.

[@nyadokuamponsah04]

This is the best video I have watched so far. Thanksss!

[@robmulla]

Thanks so much!

[@Thorne2610]

I have watched more than 5 times its really eye opener and step by step teaching. Well done Boss

[@zhaozheng7704]

Excellent tutorial and immediately useful. Thank you!

[@robmulla]

Glad it was helpful! Thanks for watching Zhao.

[@careervest]

this was wonderful thank you

[@浩然李-c3y]

this is absolutely amazing! Follow your video step by step actually make me more confident of my coding!

[@lindyhopwithliz]

Great video: informative and fun; easy to follow along. Helped me feel motivated to tackle more Python Pandas. Thanks so much!

[@alikakavand3165]

It was fantastic. Every step you took was kind of amazing, specially the last bit where you visualized average coaster speed by location. Thanks.

[@TrendingUpdateCentral]

It's really hard to find good videos on this topic. This was fantastic. Thank you.

[@ricardorockthem3339]

Very well explained and quite nice difficulty level! Brilliant!

[@leonrobinson2053]

Late to the party but this is really really good. Helps you dig in to the detail (rather than you thinking, how do I do what I'm thinking I need to do). This should be a template to use as it general enough for you to pick it up but specific enough with examples to be used elsewhere

[@jonathanhody3622]

Extremely interesting! Thank you for the content, I really like the way you go step by step in your approach, it makes it very easy to understand for non-advanced people like me

[@robmulla]

Thanks Jonathan. Glad you found it helpful. Looking forward to making more videos like this in the future.

[@jonathanhody3622]

@@robmulla Can't wait! :)

[@pdrcouto]

It has been great to refresh some topics and learn new ones. Thanks a lot :)

[@robmulla]

Thanks Pedro. So glad you’ve found these as a good refresher.

[@thshantakumar]

Couldn't stop myself THANKing you.

[@robmulla]

Thanks you for watching and commenting!

[@SearchingforScraps]

Wow ! this is such a clean run through. You make it look so easy and easy to learn ! Thank you so much. This is giving me the confidence to finally start something on my own.

[@anishkumaranjan]

Just completed it along with coding it all!

[@muhammadfadliaktsar7172]

Thank you Rob for your explanation, before this it was hard for me to study and my mind just start pressured me of how to do EDA with Python language. And this video just open my mind to study it!

[@duchungluong]

Thank you Rob. I really appreciate your contents. One small thing I noticed in this though. Would it be unnecessary to use copy() if you are overwriting df with the new subset of columns? I think copy() is helpful if you create a new variable for the subset to avoid it referencing back to the original df. But in this case, it is overwritten so it shouldn't matter isn't it?

[@rajeevnair8815]

Thanks for the excellent video on EDA, Rob. It'd be great if you could do a complete EDA on a relatively complex dataset, wherein we have scope for feature engineering, by exploring feature interactions and stuff like that. Thanks!

[@robmulla]

Thanks for the feedback and wonderful suggestion. I have something like this in mind so keep your eye out for future videos.

[@elevated_minds09]

​@@robmulla Hey Rob, your videos are really helpful. But as @Rajeev suggested pls cover EDA for complex dataset which will help us to understand the advance concepts. Your xgboost timeseries videos were really awesome. It saved in my project work. Looking forward to upcoming videos.

[@oluwatobiadeniji4268]

@robmulla good day, can I get a link to the video?

[@5014eric]

30:30 Matplotlib's scatter plot also allows setting the colour. (just copying an example out of my code) plt.scatter(df1['lon'], df1['lat'], df1['sz'], c=df1['resid'])

[@robmulla]

Nice. Doesn’t the resid need to be a hex color for that to work?

[@avidlearning-gz2yp]

I have to change your code in the line where "pd.set.option('max_columns', 200)" to "pd.options.display.max_columns = 200" because there was an "option error : Pattern matched multiple keys"

[@mschuer100]

Awesome video! Thanks for putting the time into this. Very helpful

[@robmulla]

Glad it was helpful! Share with a friend!

[@mschuer100]

@@robmulla I certainly will. Thanks

[@khushi1385]

Amazing, within a short time you taught so many helpful features for meaningful analysis. Really appreciate.

[@bingolio]

Excellent and succint. THANKS!

[@robmulla]

Glad it was helpful! If you can, share it somewhere you think others might also learn from it too!

[@9jorge]

Thanks a lot, you explain concepts like no one, subscribed!

[@sowjanyakake5720]

Very detailed explanation for each step you perform during the analysis, helpful for beginners like me.

[@robmulla]

Glad to hear you found the pace was helpful!

[@Edward_Adams]

Hey, Rob! Thank you for your video! Haven't watched it till the end but it seems really cool and brings a lot of value. Thank you! 🥰 Your line of code "pd.set_option('max_columns', 200)" seems to not work properly and while executing it I got an error "OptionError: Pattern matched multiple keys". I'm not an expert but as I understood while googling this issue, this expression is outdated. I replaced it with "pd.options.display.max_columns = 200" and it worked as I expected.

[@ozrenopacic904]

thnx

[@xiaoyangshawnhuang1251]

it's a super great video, just enjoy the way you explained, it's long video but every part is so useful and informative, thanks a lot for sharing it, well done.

[@Davlet]

your content is pure gold! thank you

[@robmulla]

Glad you enjoy it! This comment is gold. 😎

[@vishwathapa6626]

Pair-plot looked absolutely beautiful!

[@hawx786]

That was simply amazing! 10/10 on all counts.

[@robmulla]

Really apprecaite the feedback 🙌 - please share it with anyone else you think might feel the same.

[@alasdairmunro1953]

Really great video. Clear and well presented. Thank you.

[@robmulla]

Glad you enjoyed it!

[@codecode6419]

I am spending my weekend with your videos and I would like to say that I have learned several tricks to use python's libraries efficiently. Thanks for your explanation and your time to provide the videos

[@282OJK]

This is a great video. So helpful and informative. Thank you.

[@robmulla]

Glad it was helpful! Thanks for watching and tell your friends!

[@naderbazyari2]

one of the most awesome videos about data cleaning and analysis ever. Many thanks

[@robmulla]

Wow, thanks! Glad you learned from it.

[@lashlarue7924]

Rob you are the MAN! Thank you for sharing your knowledge with the rest of us! 😅🙌

[@robmulla]

Glad you found it helpful Lash! Share it with anyone you think might also learn from it.

[@ilmankhairusidqi9146]

Your explanation is easy to understand and also show how the things work, ThankYou please make more videos about EDA in python Rob!!

[@gabrielacolen7281]

This video is really good. Thank you so much!

[@robmulla]

You're very welcome!

[@hardikacharya2664]

Great video. Look forward to your twitch streams!!

[@robmulla]

Thanks so much. Hope to see you during one of the twitch streams soon.

[@Dongnanjie]

Definitely amazing. Thank you so much, Rob!