How do I apply a function to a pandas Series or DataFrame?

Have you ever struggled to figure out the differences between apply, map, and applymap? In this video, I'll explain when you should use each of these methods and demonstrate a few common use cases. Watch the end of the video for three important announcements!
Subscribe to the Data School email newsletter: www.dataschool.io/subscribe/
Join "Data School Insiders" for exclusive rewards: / dataschool
== DATA SCHOOL VIDEO TUTORIALS ==
Data analysis with pandas (30 videos): • Data analysis in Pytho...
Machine learning with scikit-learn (10 videos): • Machine learning in Py...
Version control with Git and GitHub (11 videos): • Version control with G...
== PANDAS RESOURCES ==
GitHub repository for the series: github.com/justmarkham/pandas...
Series "map" documentation: pandas.pydata.org/pandas-docs/...
Series "apply" documentation: pandas.pydata.org/pandas-docs/...
DataFrame "apply" documentation: pandas.pydata.org/pandas-docs/...
DataFrame "applymap" documentation: pandas.pydata.org/pandas-docs/...
== RELATED PANDAS VIDEOS ==
loc, iloc, and ix: • How do I select multip...
string methods: • How do I use string me...
== JOIN THE DATA SCHOOL COMMUNITY ==
Blog: www.dataschool.io
Newsletter: www.dataschool.io/subscribe/
Twitter: / justmarkham
Facebook: / datascienceschool
KZbin: kzbin.info...

[@sak8485]

Dude , you're the Corey Schafer of Pandas.

[@dataschool]

Ha! Corey is a friend of mine and a great educator, so I take that as a huge compliment 😄

[@RoylanMartinez]

haha

[@philiperiskallaleal6010]

Thanks for everything. Every minute here invested has been most valuable. Awesome videows. Super didactic!

[@dataschool]

Thanks!

[@aleksandramazurek1364]

This is amazing! The way you explain the complicated stuff is fenomenal, thank you

[@dataschool]

Thanks, I really appreciate your kind words!

[@NroShock]

Thank you so much for the time and effort you have put into your video series, and the amazingly well documented Jupyter Notebook with everything from the series!

[@dataschool]

Thanks for appreciating!

[@marcosylinafeijoo9966]

Great Series: I have learned so much. You are one of the best teachers I have ever seen.

[@dataschool]

Wow, thank you so much! I really appreciate your compliment!

[@chaithanyavamshi2898]

Thanks a ton for the great series, Kevin!. I have watched all your videos and Pandas series videos are still go to watch even in 2021. You are a life savior, Sir!

[@dataschool]

You are so very welcome!

[@Matt-tn2on]

I've watched a lot of tutorial videos in my time, but you my friend, are hands-down the best!

[@dataschool]

Wow, thank you so much! 🙏

[@MrNicfeller]

Just finished all 30 videos. Thank you very much Kevin! You are a great teacher!

[@dataschool]

Awesome! Congrats on finishing the series! :)

[@potentialada]

Thanks for the great series. Your videos have given me a good foundation in pandas. With what I have learned here I have already been able to generate several data clean-up scripts, and run data analysis that I have presented at an academic conference using only pandas. Great work!

[@dataschool]

Wow! That's awesome, I'm so glad to hear this series has been helpful to you!

[@sagarghimire]

Hi Kevin, how do I lookup a column values in one datra frame against the second data frame, say column 1 and get the corresponding data from column 2 of second data frame and add that results to first data frame.

[@NemoNemoNemo.]

Sometimes when I have free time I watch your videos. Always walk away with something new and useful. Thanks for the great vids.

[@dataschool]

Awesome! That's so nice to hear.

[@user-ep1xm2ty8c]

Awesome. I think it is especially super useful for non-English speakers as your English is clear and slow. Thank you very much.

[@dataschool]

You're very welcome!

[@hyakushiki23]

I just found your videos and they are awesome. Your explanations and examples are easy to understand, but also very practical. I am looking forward to watching the other 29 videos

[@dataschool]

Great to hear!

[@hpchen5402]

This is a great video series on pandas. Really concise and clear. Many thanks for the effort!

[@dataschool]

Thanks for your kind comment! I enjoyed creating the series :)

[@twofour1969]

Thanks Kevin for devoting your time to provide such good series videos.

[@dataschool]

You're welcome!

[@vaibhavkathale5527]

Great videos, perfect explanation. Best teacher on KZbin for Data science... Waiting for your videos

[@whatwasisaying1338]

I'm glad I found your videos! You make understanding these concepts much easier!

[@dataschool]

Thanks very much for your kind words!

[@bloomingtonvolleyball639]

You are such an awesome tutor that I felt this is the best channel for pandas. Please keep posting. Thanks a lot.

[@dataschool]

Thanks so much!

[@PradeepKumar6]

Thanks Kevin for such a nice tour on pandas....I enjoyed it a lot. By the way, In future if you decide to start it again, please enlighten us with the "Reduce" function as well. Thanks again for helping people so much. I know it is really hard to make a video. It takes lot of time and effort. Take Care, Good luck to your projects

[@dataschool]

Thanks so much for your kind words! Glad the pandas series has been helpful to you. You're right, it is a ton of effort to make the videos, but definitely worth it :)

[@TheOntheskies]

Thank you. Your explanations are well-paced and easy to understand and most importantly sticks to memory.

[@dataschool]

Thanks for your kind words! 🙏

[@saurabh75prakash]

Excellent tutorial, your video series is a must watch for anyone serious in data science.

[@dataschool]

Thanks very much for your kind words!

[@samc2481]

Best pandas tutorial so far on the internet... Thanks Kevin. Great work. hope you can do more tutorial on applied Tensorflow & ML

[@dataschool]

Thanks for your kind comment, and also for your suggestion!

[@7791merc]

All videos are brilliantly curated and explained thoroughly. Great way to teach and learn.

[@dataschool]

Many thanks!

[@deborahweissner8527]

Thank you for your clear and concise explanations. Such a pleasure to watch your videos.

[@dataschool]

Wow, thank you so much for your kind comment!

[@cradleofrelaxation6473]

No doubt one of the greatest teachers on KZbin!! So calm and graceful ‼️

[@dataschool]

You are so kind, thank you!

[@carlosportillo6815]

Hands down, one of the best Pandas instructor in YT!

[@dataschool]

Thank you!

[@heidelms]

Omg I really loooove your videos... I'm going to use everything you share, it's amazing. I quit on learning python 2 years ago and I want to give it a try again now. And by seeing your pandas videos I truly think it will woth it. Thank you so so much for your effort... keep going 💪🏽

[@dataschool]

Thank you so much! 🙌

[@parthibank280]

Incredible explanation which is very easy to understand. One can give this much of a clear explanation on stuffs only if they are confident with what they are talking about and you are nailed it buddy.

[@dataschool]

Thanks very much for your kind words!

[@rajsoniism]

every minute i spent here more than worthy.The way you teach to really good.

[@dataschool]

Thank you! 🙏

[@youngzproduction7498]

Thanks for your effort. It never gets old.

[@sandeepvarma4540]

Thanks for the series. Very helpful. Would recommend to everyone who want to kick start.

[@dataschool]

You're very welcome! And, thanks for your kind comment!

[@jongcheulkim7284]

Thank you again. whenever I watch this again, I am learning new.

[@dataschool]

Great to hear!

[@gopalagr85]

I followed all the videos of this series and course content is awesome and your teaching style is tremendous

[@dataschool]

Thank you so much!

[@brendensong8000]

Thank you for sharing! I learn so much from every video!

[@dataschool]

I'm so glad!

[@lindsaybreidenbach8658]

Saved my butt for finishing my hw today. Thx!

[@dataschool]

That's awesome to hear!

[@nas209]

exceptional videos, it has the depths and info needed to catch up with data scientist and with remarkable teacher, well done, thanks

[@dataschool]

You're very welcome! Thanks for your kind words!

[@scottlucas3710]

Wow. Excellent job of explaining map vs apply methods not simply explained elsewhere (even by Wes)

[@dataschool]

What a nice compliment, thank you so much! :)

[@aminsama]

this helped me in better understanding the fundamental difference. thank you so much.

[@dataschool]

You're very welcome!

[@apostolosmavropoulos177]

You are a sweetheart bro! Thank you so much, the whole playlist is perfect

[@dataschool]

Thank you!

[@vlinkin22]

Nice, finally I understood the apply method Many Thanks !

[@dataschool]

Great to hear!

[@nasreenbanu2245]

Hats off to you .You made me pro in coding.Thanks a lot..keep going

[@dataschool]

Thank you!

[@Ganeshkakade454]

Hey...great Series Sir..learning Pandas module from your side is such a boon..Thanks,Love from India!!!

[@dataschool]

Thank you!

[@nghiepcrypto7034]

Corey brings me here. Your videos are great! Keep up the great work

[@dataschool]

Corey is great! 🙌

[@thethirdkulture3314]

Thank you so much for this! You're so clear and concise!

[@dataschool]

Glad it was helpful!

[@taikoktsui_sithlord]

Superb tutorial. Thanks so much!

[@dataschool]

Thanks for your kind words!

[@ItsWithinYou]

Excellent! I am much clear now on this topic...Many thanks!

[@dataschool]

You are welcome!

[@ankhts]

This is amazing technique...Many thanks for the video

[@jrochagrupo]

Hi Kevin, Great Tutorials, learned a lot. Thanks.

[@dataschool]

You're very welcome!

[@meitalostikar3122]

Thank you so much! All your videos are amazing!! I learned so many new things

[@dataschool]

Thank you for the kind words 🙏

[@boubacaramaiga4408]

Clear explanation, excellent tutoriel. Many thanks.

[@dataschool]

You're very welcome!

[@aminasgharisooreh9243]

Thanks. it is a great series about DATA SCIENCE

[@WaltterValdez]

Thanks a lot for your time It has been very helpful for me

[@dataschool]

You're very welcome! Thanks for joining me for the series :)

[@erenhan]

you are the best pandas teacher I've ever seen

[@dataschool]

Thank you!

[@ahmadaburizaiza6598]

Thank you so much for this video and for all the other videos in your amazing channel. I just found a solution in this video for a problem I was struggling with :)

[@dataschool]

Awesome! Great to hear! :)

[@Edy54321]

Thanks for your videos. They really do help build the knowledge base for beginners like myself

[@dataschool]

You're very welcome! Good luck in your daat science education :)

[@yellowbug772]

Really appreciate it, thank you, Kevin!

[@dataschool]

You're welcome!

[@chinamyx]

Hey thanks for sharing! You lessons are clear

[@dataschool]

Thanks!

[@anAlokDubey]

You are really a best teacher :-) your videos are very helpful and very informative. Thanks you so much 😊😊

[@dataschool]

Thank you! 🙏

[@sureshkumartwarakavivenkat6760]

Very good explanation please run a case study from problem definition onwards

[@serdarb8995]

It was a great video series. Thank you very much for your great effort. You are great Kevin

[@dataschool]

Thanks! :)

[@Amitsingh-mh3lx]

your video helping me to much thanks from india

[@joro8604]

One of the best Pandas videos ever. If i had saw this a few days ago, it would have saved me a few days of frusteration

[@dataschool]

Glad it helped!

[@kenwallace127]

I really appreciate how clear and thorough you are with your explanations. Thank you for this! One question I have, when you split on the comma, I understand how it creates a list of strings. Can you explain why the strings do not show up in quotes, though? Don't most other string list elements? Thanks!

[@dataschool]

Glad you like the videos! Regarding your question, pandas doesn't display quote marks around strings.

[@wildlife_and_drones1320]

Clear and concise. Thanks

[@dataschool]

You're welcome!

[@jorgev4656]

thank you very much. the best explanation of youtube.

[@dataschool]

Wow, thanks!

[@yossarian2909]

Lucid explanations.. i love your channel!!

[@dataschool]

Thanks!

[@riderblack6401]

awesome vid! I learn so so Much from you Kevin! Keep it up!

[@dataschool]

Thanks! I appreciate your support.

[@esdrasdebrito]

very good examples. Thank you so much

[@dataschool]

You're welcome!

[@Genetics101]

super clear and very helpful. thanks!

[@dataschool]

You're very welcome!

[@gulshankumar17]

thank you so much for putting so much effort.

[@dataschool]

You're welcome!

[@xinwang68]

The video is perfect, and i studied a lot from it, thanks.

[@dataschool]

You're very welcome!

[@khairuleinuddin936]

Thank you very much. You've been a great help.

[@dataschool]

Glad to hear that!

[@spytheman]

Thanks! I overcome a lot of questions that many online videos couldnt gave me the answer! Btw, when doing a lambda expression, while we can lambda x: x[1] to use its neighbouring data, is there a way to get the data below it rather than on the right or left of it?

[@MrGleyce1]

Just amazing! thanks a lot for sharing your knowledge.

[@dataschool]

Thank you!

[@mansiarora5899]

Great videos 👍 I have learnt a lot from them. I have one question it would be very great if you find time to answer this. I have a list of string values and I am trying to create a new column and the value should be 1 if the value given in the list exists in any of the column of dataframe else the value should be 0. Eg. There is a dataframe with cols A,B,C,D and I have a list names test which has some values in it. If the value exists in either A,B,C or D then the first row of new col E should have 1 else 0 and similarly for the next row

[@veggiet2009]

Really like these informative videos! I was hoping you'd get to a video on multi-index

[@dataschool]

Glad you like it! I cover multi-index a tiny bit in my DataCamp course: www.datacamp.com/courses/analyzing-police-activity-with-pandas?tap_a=5644-dce66f&tap_s=280411-a25fc8

[@dataschool]

You might like to know that I just released a video about the MultiIndex: kzbin.info/www/bejne/qpS1eJRoqNSWY8U

[@UncleLoren]

Though in some cases, APPLY may not be as fast as using other approaches, that difference is usually negligible. Kevin doesn't mention that here, in the interest of time/audience; HOWEVER, he did make this very point in an insightful Stack Overflow post that I recently came across. In fact, there was a subtle advantage to using apply, instead of the other alternative, in that it preserved the index. So, yes: speed and optimization are great and all, but most of the time, writing code that is simple / readable -- which Kevin acknowledges as being subjective -- will just as effectively get the job done. This video series is wonderful because it's not only palatable to beginners, but I find myself coming back to it and picking up certain details of why things work as they do, that may not have *stuck* the first time around. It can initially feel overwhelming that Pandas gives you so many options on HOW to do things, but I've come to appreciate the value in wielding a proverbial swiss-army knife, and I've started to recognize which situations favor using one instrument over another. Thank you, Kevin, for being a great teacher, and for presenting content -- beyond this video series, at that -- that appeals to users having a range of experience.

[@dataschool]

Thank you *so much* for your thoughtful comment and your kind words! 🙏

[@MrBhargavafirst]

very knowledgeful thanks for this

[@dataschool]

You're welcome!

[@nimesharya909]

This was great, i did the exercise and it was great..I will sign up to your news letter.

[@dataschool]

Awesome! Here's the link to sign up for my email newsletter: www.dataschool.io/subscribe/

[@MrNitKap]

Thank You. Your effort is appreciated.

[@dataschool]

You're welcome!

[@NeoKailthas]

you are really good at teaching.

[@dataschool]

Thanks so much!

[@fernandotorres1203]

Thanks man for making these videos.

[@dataschool]

You're very welcome! I enjoyed creating them :)

[@DonDynAyt]

First of all, thank you very much for the time and effort you put into this video series it has been very helpful. Moreover, would you know how to subtract from a column a substring that is in another column?

[@dataschool]

There's probably a string method that can help you, but I don't know which one off-hand... sorry!

[@SugamMaheshwari]

beautifully explained brother !

[@dataschool]

Thanks!

[@navishagarwal1736]

A clarification on the apply function. In the example where you apply it on the Name series, how is it that you can use the str.split function on the 'Name' series without the apply function in the dataframe but for 'len' you need to use the apply function?

[@aayushmalde3707]

most helpful video man

[@dataschool]

Thanks!

[@ravinduabeygunasekara833]

This is an awesome tutorial! LEarned a lot from this. One question, what if I want to create 2 new columns named "First Name", "Last Name" and then split the values in Name column into those 2. Ex:For 1 st row, the "First Name" = Mr. Owen Harris and "Last Name" = Braund. Could you share the workaround to get this done?

[@riderblack6401]

Best python tutorials in the sea of the net you can dig. I promise subscribe it!

[@dataschool]

Thanks so much for subscribing, and for your kind comments!

[@arzoo_singh]

This can be used in an efficient way for feature selection.

[@alexsandroignacio761]

Very nice teachers i ever seen.

[@dataschool]

Thank you!

[@learnsimplebhima4059]

Good to watch and learn

[@dataschool]

Thanks!

[@dineshpandey5008]

Thanks for this , It is really helpful, but in my case, it returns 0 and 1 instead of column label df.loc[:,'Parle':'Soap'].apply(np.argmax,axis=1)

[@ilyastrojnov7627]

for simple operations like len and sum, no functions are needed, for example: df ['Len'] = df [Column_with_text] .str.len (), it will be much faster. To calculate the sum or average, we can choose an axis: df ['Total_for_rows'] = df.sum (axis = 0) or df.loc ['Total_for_columns'] = df.sum (axis = 1)

[@visheshmangla8220]

Really great videos!!

[@dataschool]

Thanks!

[@jundou7858]

thanks for the tutorial, very helpful.

[@dataschool]

You're very welcome!

[@gilberthor1988]

Congratulations for this video! I have one question: how can I, for exemple, get each string of the column "Name" and put them into a sigle list? For example: [Futrelle, Mr., Jaques, Health, Lily, May, Peel, Allen, Mr., William Henry]. Thanks in advance!

[@samshah3667]

Thanks for the great videos. I just finished watching all the 30 videos in this series and enjoyed learning Pandas. You are a great teacher! I also signed up for the newsletter. I have a question and hope that despite your busy schedule, you can either answer me, or at least point me to the right direction: How would I add a column to a data frame, which holds aggregate data from the same dataframe? Let's say I have a dataset of "expenses" over the last 5 years. I want to add two columns to the dataset, and for every row, the first column will hold the "year to date" sum of all expenses (including this one), from the beginning of the year, and the second column will hold the "month to date" sum of all expenses. How can I accomplish this? Thanks!

[@dataschool]

Thanks so much for watching the videos, and for your kind words! Regarding your question, I feel like it's possible but I'm not sure exactly how. If you are interested in joining Data School Insiders, you could post this question in our private forum and see if someone happens to know: www.patreon.com/dataschool

[@robertbriscoe277]

You make great videos! Have you ever made one to address how to iterate through a pandas data frame row by row: and for each row look at the previous row, column x, apply logic, then calculate current row column x. I can not find a way to do this that does not return a copy of original data frame for the logic instead of the values that are being calculated in the iteration. Thanks

[@fangyuancao5026]

Super helpful!

[@dataschool]

Great!

[@alejandrozaizar1329]

How would you find the best optimal cut off point for continuous variables having a binary outcome variable?

[@benben341]

Thank you for your video. If your table has a large amount of catogories ie coutries, colours etc - i use pd.factorize(d.columnname) to create a number list for scikit as it cant use the label data - so a question for later videos - whats the best policy map or factorize ?

[@dataschool]

Thanks for your question! I think the short answer is that you should use whichever one makes the most sense to you and is the least work :)