My top 25 pandas tricks

[@dataschool]

THANKS for watching! 🙌 Which trick are you most excited to start using? CLICK REPLY and let me know!

[@TarmoSillajoeGoogle]

Thanks. Trick nr 9- building a dataframe from multiple files is the one i've been searching for. Comment to trick nr. 3. Sometimes modifying column names works by using Python's list comprehension. Say we want to capitalize all column names. Just write: df.columns = [my_col.capitalize() for my_col in df.columns]

[@dataschool]

Glad you like trick 9, that is one of my favorites! As for capitalizing column names, a list comprehension works, but a string method is even simpler: df.columns = df.columns.str.capitalize()

[@juanbetancourt5106]

Thanks, those tricks are really amazing, I wonder if I can take this magic to the web, is it a good choice to combine pandas power with django?

[@mritunjaykumar970]

The last trick will significantly reduce the amount of work we do understand the data through eda.that trick was amazing.thanks kevin

[@sreenivas7739]

Thanks for a great video. Bonus trick is the most useful.

[@Alok_Mandal]

you just saved my life, i was collecting these things over google wasted my good time but now i know where to look for. keep sharing , God bless you. Thanks

[@dataschool]

You are welcome! 😄

[@zackhicks2433]

Video should be called 'Magic with Pandas'. Thanks for this really useful resource!

[@dataschool]

Thank you! 🙏

[@BasselHaidar]

Kevin, love your content and it is amplified by your speech cadence, enunciation, and tone. My favorite tricks are 10 and 11 (Building a dataframe from multiple files rows/columns). Thank you for sharing your knowledge, mad respect!

[@dataschool]

Thank you so much! 🙏

[@rvg296]

No other trick mentioned in the video is comparable to the Bonus Trick (Profiling).. This is simply outstanding !!!!

[@dataschool]

Thanks!

[@Goodhealthall2015]

I had some trouble with my dataframe. So I was stumbling over the internet to find solution,and your video came. Although I didn't find solution in your video, but one of the trick gave me a logical idea of how to resolve the issue. It told me where I was wrong. And as far as tricks are concerned, they are amazing. Some went over my head but that's okay. This whole panda python thing is new to me. I'd prefer to bite slow instead of gulping down entire meal in one go

[@dataschool]

Thanks for your kind words, and glad I could be helpful!

[@valeriemolignano8127]

Kevin, you are a fantastic instructor. I'll sign up for anything you offer. My 2 favorites: clipboards, and, what made my mouth drop - the dataframe profiling.

[@dataschool]

Thank you so much for your kind words Valerie! I have a lot of resources that might interest you here: www.dataschool.io/start/ And here is my email newsletter: www.dataschool.io/subscribe/ Thanks again!

[@giangtu6000]

I love you! KEVIN!

[@naqeeb16]

It is one of the best videos on Pandas. I really love some of the tricks e.g., the Clipboard', and 'The Excel copy & paste ones.

[@dataschool]

Glad it was helpful! Thanks for your kind words!

[@satty22]

The long awited video is out now. Thank you so much mate. Great learning with you.

[@dataschool]

You're very welcome!

[@kislaylove]

Last Bonus tip, Best in 2020 till date. Thank you so much. You have made my life easy!!!

[@dataschool]

Thank you!

[@mustafabohra2070]

Man, you are just so awesome! Giving all these valuable tricks for free on your channel. God bless you!!!

[@dataschool]

Thank you so much! Glad you like them 👍

[@Ashr2230]

Your contents are to the point, your teaching style is exceptional. I am learning so much from your videos. Thank you!

[@dataschool]

Wow, thank you so much!

[@thomasmendy9802]

Kevin Markham, you by far one of the best Data Science Instructor, I came across. Thank you for providing us with such high standard content at no cost.

[@dataschool]

Thank you so much! 🙏

[@balasubramani2126]

this video is awesome.Maybe knowingly or unknowingly i worked with the following tricks. trick_no={2,6,13,15,17,18} and the remaining tricks are new to me.thanks for the video.i will practice the remaining tricks in upcoming projects

[@dataschool]

Sounds good! Thanks for your kind words!

[@NitinSawant1988]

This video was awesome.. Very Brief, to the point. It will be very easy to revise the concept/code by just revisiting them.. Thanks a lot Sir.....

[@dataschool]

You're very welcome!

[@sushedbubai]

All the tricks were mind blowing. But the bonus one was out of the world. It is surely going to help those who struggles Exploratory Data Analysis. Many thanks for sharing this.

[@dataschool]

Awesome! So glad it was helpful to you! 🙌

[@Asparuh.Emilov]

This is one of the most useful coolest videos about Pandas I have ever watched. Thank you sooooo much for compiling all these tips and tricks! Absolutely amazing!!!!!!

[@dataschool]

You're very welcome! 🙌

[@stretch8390]

This is a nice vid! When I was learning clojure someone had compiled a '100 most used functions' and boy would I love that for pandas functions or methods: the docs are too extensive but tutorials are typically too brief.

[@dataschool]

That's an excellent idea!

[@adithiaa4385]

The last 5 tricks were fantastic.!!

[@dataschool]

Thank you!

[@mikekramer7737]

Tnx. The .style.format(dict) is a game changer. I used to convert my parameters to ensure a fixed number of parameters would show up well, i.e convert a fraction to a percentage. Not sure why this had never been mentioned in the many videos I have watched before.

[@dataschool]

You're very welcome!

[@irfankhuddus1974]

Superb ! Excellent ! introduction for Beginner (myself) to get introduced and master these Pandas at the same time thereby saving me a lot of time. Thank you so much Kevin !!! You Rock !!!

[@dataschool]

That's great to hear! Thanks for watching 👍

[@christopherteoh3094]

The bonus content totally blew my mind off! Thank you very much Kevin! You're simply the best! Looking forward to purchase your ML series!

[@dataschool]

Thank you so much for your kind words!

[@kamleshgangaramani9445]

all tricks superb, continue good work🎉

[@dataschool]

Thank you so much 😀

[@OmarAbdelaziz__47]

Thank you Kevin, You are really a good instructor. Also, I really loved how you shout out people your learnt tricks from, that's very awesome!

[@dataschool]

Thank you!

[@asn6543]

Thanks you sir, your tutorials are very clear and very easy to learn even for none english speaker. Lots of thanks

[@dataschool]

Glad to hear that!

[@siddhartha8886]

Thank you, for giving me shout out, and thank you for including my trick in your video (pd.profiling) :)

[@sanjaykrish8719]

Wow... Veryyyy cool tricks. Saves a ton of time.

[@dataschool]

Thanks Sanjay! 🙌

[@tellkorok]

The bonus is cherry on the cake!

[@dataschool]

🍒

[@manojl3388]

Hi Kevin, Thank you very much for sharing your knowledge to the world :) The most fav trick I loved is, its your video editing (the highlighting of cells while you were explaining). Now its hard to play in 2x speed :) But you have condensed so much information in a very short video. Thank you!! Also finally, the way you thanked everyone.. taught me how to be humble and acknowledge others effort. So your videos are more than just pandas videos.

[@dataschool]

Thanks Manoj for your really thoughtful comment! I wouldn't be anywhere without the amazing pandas resources and tips provided by others, so I definitely wanted to acknowledge them. And I'm glad you liked the highlighting! Credit for that goes to my excellent video editor 🙌

[@MadLadsAnonymous]

Wow, that last tip! What a phenomenal overview and diagnostic tool. I've learned so much from your videos. So grateful for your content.

[@dataschool]

Thanks very much for your kind words!

[@meghasyam427]

Holy Moly What a master piece this is. I liked the clipboard trick a lot man like that literally saves a lot of code and time. And the bonus blew me off. It looks literally like a final report LOL. Can't wait to implement these tricks in my analytics work. Thanks a lot Data School

[@dataschool]

You're very welcome! Thanks for your kind words!

[@tunkyi7162]

It helps me a lot in Data Wrangling. I really love your videos. Keep doing!

[@dataschool]

Great to hear!

[@maxxpool]

I'm so glad I chose to click on this video. Thank you for this amazing video!

[@dataschool]

You're so welcome!

[@pldeepesh]

Fantastic! Been using pandas since a year now, You have opened new possibilities for me to keep looking and learning.!! Please keep doing the great work!

[@dataschool]

Thank you!

[@Yuri-to8uj]

the best guy in this business!!!

[@dataschool]

Thank you so much! 🙏

[@kishlayamourya3141]

Best pandas video ever!

[@dataschool]

Thank you! 🙌

[@shoboyjr]

Thanks a lot. I learnt and liked the new tricks below: The apply function Using ufo.isna().mean() to find percentage of missing values. Using the transform function combined with the groupby function reminded me of SQL Windows Functions pd.cut was good to help convert a continuous data into a categorical one. The fancy display option was an added bonus.

[@dataschool]

Awesome! I'm glad you learned so much! 🙌

[@saeedrehman5085]

Pandas profiling is a new thing for me. It is good to learn new thing everyday. Subbed !!

[@dataschool]

Thanks!

[@YourMakingMeNervous]

Great video. I didn't know about the cut function and it will make a portion of my daily data processing tasks a lot easier. Thanks!

[@dataschool]

That's awesome to hear!

[@pradeepgb986]

Kevin, you are simply amazing. Thanks for sharing these wonderful tricks!

[@dataschool]

Glad you like them!

[@yingkitchow5160]

this is my first comment on youtube, ive been following your channel for learning pandas/scikit learn. Please keep doing what youre doing! it is so useful and fantastic for people who has no programming background. THANK YOU and i hope to see more and more of these example based video.

[@dataschool]

That is awesome to hear! I'm so glad it was helpful to you 👍

[@maxvinella941]

This is a jewel video! All tricks left me without words! thanks

[@dataschool]

You're very welcome! 💎

[@brendafosmire6519]

Best Pandas trips ever. I’m going to make myself some flash cards for each of these. Thank you.

[@dataschool]

You're very welcome! Glad you like them!

[@anastasiiag8518]

This is incredibly useful! And explained perfectly! Thank you! It would be awsome to see from you some content on advanced plotting, data plotting tricks :)

[@amandal8170]

Agree, if we can see the plotting ticks will be fan jolly tastic. :p

[@dataschool]

You're very welcome, and thanks for your suggestions!

[@homeuse8879]

Awesome pandas tricks!!! Thanks for this great resource

[@dataschool]

You're very welcome!

[@vadivelan4228]

Thanks for this consolidated and a very crisp video on Pandas....

[@dataschool]

You're welcome!

[@narendraprasath5745]

It is really nice and very useful to many people. Special thanks for your effort to post this.

[@dataschool]

Thanks for your kind words! 😄

[@hectoralvarorojas1918]

Well done Kevin! Thanks for sharing this great material. My best regards!

[@dataschool]

You're very welcome, Hector!

[@hoschi60]

Nice compilation of usefull pandas hacks.... I watched your video to find help on the .merge() method, to combine two dataframes in a sql manner..

[@riessm]

First time ever I honestly wish I could upvote this more than once! Great tips, great explanation! One short addition to trick 19: I enjoyed getting to know the transform method, but as most of the times in pandas, there is another way with (for me at least) more often used functions: Just take the aggregated data-frame and merge it with the old one (left merge) instead of concatenating. I don't know if there are any runtime benefits of any of these methods, but to me it's one less command to remember - even though now I will look into uses of the transform method! :-)

[@dataschool]

Thank you so much! 🙏 As for trick 19, you are totally right that merge could accomplish the same thing. I like the transform code much better, but I also encourage people to learn fewer functions overall, so there's nothing wrong with just using merge! Anyway, thanks again for your kind words 😊

[@dataschool]

Want even MORE pandas tricks? I share a new trick here every weekday 👉 www.dataschool.io/python-pandas-tips-and-tricks/ 👈

[@Dara-lj8rk]

Please make a video on time series analysis in Pandas

[@dataschool]

Thanks for your suggestion!

[@vishalwaghmare3130]

Super duper useful video. Amazing!!

[@dataschool]

Thank you!

[@leonardoalvarado7632]

Thank you for all these tricks.

[@dataschool]

You're welcome!

[@kwanpakshing]

Very helpful and useful tricks for day to day Data operations

[@dataschool]

Thank you!

[@sachavanweeren9578]

I was revisiting dataschool video's. In my view these are still one of the best resources out there. Thanks a lot ...

[@dataschool]

Thank you!

[@mrmuranga]

simply awesome.. helpful, well layed out..Thanks

[@dataschool]

Thank you! 🙏

[@92soyeon]

Fantastic! It really helped me escaping from a lot of itchy situations!

[@dataschool]

Great to hear!

[@ScottLinne]

Great video. The bonus tip is very useful.

[@dataschool]

Thanks!

[@jasonwong8315]

it is amazing because every time I watch this video i can learn sth new! thank u!

[@dataschool]

That's awesome to hear!

[@gregf9160]

Excited to try ALL of these! Thanks!

[@dataschool]

You're welcome!

[@youngzproduction7498]

Yeah, 25 tricks!! I would like to thank you a lot for sharing these tips.

[@dataschool]

You're very welcome!

[@sarahmacdonald1342]

Awesome video! Thanks for reminding me of the transform function to combine aggregates back onto a df. Also, the df profiling is a great idea that I've never heard of.

[@dataschool]

Thank you!

[@dew_reddit]

Wow the last one was fantastic. styling the dataframe was also pretty neat. Thanks for this 👍

[@dataschool]

You're very welcome!

[@amandal8170]

Best pandas tutorial !! Thanks

[@dataschool]

You're welcome!

[@DiegoTerzano]

Fantastic tips, thank you!! Super valuable and with clear, practical examples. Once again, thank you!

[@dataschool]

Thank you so much! 🙏

[@neesargmistry2071]

I hardly ever comment on any videos but I had to for this one. You're amazing, keep up the good work :)

[@dataschool]

You are too kind! :)

[@norwegiandud]

Gret video series! I've watched 35/38, and I've learned a lot! 👍🎉 Regarding tip 18&19 I think you forgot to multiply with quantity. E.G. order_id #2 should have a total of 2x16.98 = 33.96

[@dataschool]

Glad you like the series! Regarding tips 18 & 19, the item_price column already incorporates the quantity. (Check out the rest of the dataset, and you will see.) That's a good thought though!

[@norwegiandud]

@@dataschool I don't know if I agree with you. In the dataset I find that order_id 484 and 822 ordered the exact same item_name + choice_description. However, while they ordered a quantity of 1 their item_price is $8.99 which is not half of $16.98. Did order_id 2 get a quantum discount?

[@snehashinde21]

This was amazing video to sum up basic Pandas usage. Thanks for this useful resources!

[@dataschool]

Thank you for the kind comment!

[@mohammadj.shamim9342]

Don't know how to thank but you are a Nobel man.

[@dataschool]

Thank you!

[@yahyahamid5762]

Thank you for the video, really useful stuff!

[@dataschool]

Glad it was helpful!

[@bikideka7880]

Thanks for the tricks, specially for the pivot_table() function and pandas_profiling . For me this is a nice summary of pandas. Thanks

[@dataschool]

Awesome!

[@jongcheulkim7284]

Thank you so much. I learned a lot of good things. Thank you.

[@dataschool]

You are so welcome!

[@FabioKasper]

Great stuff. Thanks for the tips.

[@dataschool]

You're welcome!

[@da_ta]

Pandas_profiling is BEST Can't wait to use it! Thanks Kevin for doing these

[@dataschool]

You're very welcome! 😄

[@TheNewDont]

Hello Markham, how are you doing? First I want to thank you for all the quality content, I'm learning a lot. You're a wonderful teacher! Second, for the tip number 9, I did some tests (with may not lead to a solid conclusion...) but when I read multiple files, the following method showed a significant better perfomance (I'm not sure about memory efficiency though): df = [] for file in glob.glob('files_*.txt'): df.append(pd.read_csv(file)) df = pd.concat(df, ignore_index=True) Again, thank you so much for all the knowledge shared!

[@stephenpatti2375]

Great Pandas resource! Difficult to say which trick most 'excites' me, but will go with using transform to add aggregation functions to the data frame. Thanks!!!

[@dataschool]

I like that trick as well... I only learned about it recently!

[@jagggyjazz8010]

I liked it to see someone who puts their cam much above their face level baing successful on YT

[@user-cc8kb]

Thanks a lot. This is super useful! I'll try to use as many of these tricks as I can whenever I can :D

[@dataschool]

Great to hear!

[@robertodilisio7535]

Brilliant as usual! From beginning to end! The bonus was also mind blowing :)

[@dataschool]

Thank you so much, Roberto!

[@danielperico2806]

One of my favorite videos on KZbin, thank you so much!

[@dataschool]

Wow! Thank you so much for your kind words! :)

[@cradleofrelaxation6473]

I can’t have enough of this!!

[@dataschool]

Thank you!

[@annawilson3824]

19:50 this is super-cool, always was confused by the groupby return value and how to make is a dataframe!

[@dataschool]

Thanks!

[@yashagrawal8352]

25 and Bonus tips are the best.

[@dataschool]

I like those ones as well!

[@keepcontinue]

This video helped me so much. Thanks a lot :))))

[@dataschool]

Great to hear!

[@archowdhury007]

Amazing tips!!!......keep sharing more such videos

[@dataschool]

Glad you like them! 😄

[@minglee5164]

The bonus trick is so amazing

[@dataschool]

Thank you!

[@kasiasama7322]

Great job Kevin. Learned a lot.

[@dataschool]

Thanks! Great to hear 👍

[@pavel5362]

Great tricks. Very helpful as always. Could you please make a video to explain functions and classes. Honestly, it's not clear for me when to use functions and when to use classes.

[@dataschool]

Thanks for your suggestion!

[@BabakKeyvani0]

Thanks for the great tips Kevin. Very useful 🙌🏼🙌🏼 Still going through the video, but loved all the ones so far. Will let you know my favorites after I finished 😊

[@dataschool]

Awesome! Glad to hear you're finding them useful!

[@BabakKeyvani0]

Wow, so hard to pick favorites. I have used most of these in my projects, and was planning to use the styling functionality on some dataframes. certainly didn't know about the bonus trick! Looking forward to trying that. Thanks again for sharing all these 🙌🏼🙌🏼👍🏼👍🏼

[@dataschool]

Love the styling functionality! You're very welcome 🙏

[@matthewwallace2512]

Really informative Kevin, appreciate your effort in creating these video's. Such a valuable resource. Trick 19 was probably my favourite.

[@dataschool]

Thanks for your kind words, and thanks for sharing! 🙌

[@kishoreyarramshetty2930]

Nice video on pandas.. great help to me. Thanks

[@dataschool]

Great to hear!

[@erickcampos50]

Useful is the word! Thank you

[@dataschool]

You're welcome!

[@lebedev_michael]

Hi Kevin! Thanks from Russia 🇷🇺) Your video is awesome 👏🏻

[@MrDEBONTUBE]

Thanks for compiling it at one place

[@dataschool]

You're welcome!

[@clementineh2565]

Thanks for your help. It's very clear and easy to understand.

[@dataschool]

You're very welcome!

[@jianbingding4087]

Thank you, it's very useful!!!! I have subscribed your video, thank you for your summary.

[@dataschool]

You're welcome!

[@kalekalekale]

Excellent video, thank you for cutting to the chase.

[@dataschool]

Thanks! This video took a very, very long time to create 😄

[@somasundaramgurunathan3087]

Kevin - you r simply awesome man.. thanks for sharing wonderful tips.. wish you to post more which will help us to gain

[@dataschool]

Thank you so much!

[@MaxoticsTV]

I'm getting back into Python and found this video very useful AND inspirational, which isn't easy to do with this 58-year-old ;)

[@dataschool]

That's awesome to hear!

[@ankushsharma-gu7co]

Needless to say. Thank you once again for the excellent video, like always very neat explanation and clean deployment of ideas.

[@dataschool]

Thank you so much! I'm glad the video was helpful to you!