How do I apply a function to a pandas Series or DataFrame?

[@sulaiman1408]

Once I am looking for any help in python, I always search in your videos. And I ended up knowing more than what I want. Great tutor 👍

[@dataschool]

That's awesome to hear, I'm glad I could help!

[@cradleofrelaxation6473]

No doubt one of the greatest teachers on KZbin!! So calm and graceful ‼️

[@dataschool]

You are so kind, thank you!

[@파이썬으로머신러닝배]

Awesome. I think it is especially super useful for non-English speakers as your English is clear and slow. Thank you very much.

[@dataschool]

You're very welcome!

[@marcosylinafeijoo9966]

Great Series: I have learned so much. You are one of the best teachers I have ever seen.

[@dataschool]

Wow, thank you so much! I really appreciate your compliment!

[@sak8485]

Dude , you're the Corey Schafer of Pandas.

[@dataschool]

Ha! Corey is a friend of mine and a great educator, so I take that as a huge compliment 😄

[@RoylanMartinez]

haha

[@carlosportillo6815]

Hands down, one of the best Pandas instructor in YT!

[@dataschool]

Thank you!

[@potentialada]

Thanks for the great series. Your videos have given me a good foundation in pandas. With what I have learned here I have already been able to generate several data clean-up scripts, and run data analysis that I have presented at an academic conference using only pandas. Great work!

[@dataschool]

Wow! That's awesome, I'm so glad to hear this series has been helpful to you!

[@sagarghimire]

Hi Kevin, how do I lookup a column values in one datra frame against the second data frame, say column 1 and get the corresponding data from column 2 of second data frame and add that results to first data frame.

[@erenhan]

you are the best pandas teacher I've ever seen

[@dataschool]

Thank you!

[@Matt-tn2on]

I've watched a lot of tutorial videos in my time, but you my friend, are hands-down the best!

[@dataschool]

Wow, thank you so much! 🙏

[@NroShock]

Thank you so much for the time and effort you have put into your video series, and the amazingly well documented Jupyter Notebook with everything from the series!

[@dataschool]

Thanks for appreciating!

[@heidelms]

Omg I really loooove your videos... I'm going to use everything you share, it's amazing. I quit on learning python 2 years ago and I want to give it a try again now. And by seeing your pandas videos I truly think it will woth it. Thank you so so much for your effort... keep going 💪🏽

[@dataschool]

Thank you so much! 🙌

[@taikoktsui_sithlord]

Superb tutorial. Thanks so much!

[@dataschool]

Thanks for your kind words!

[@philiperiskallaleal6010]

Thanks for everything. Every minute here invested has been most valuable. Awesome videows. Super didactic!

[@dataschool]

Thanks!

[@7791merc]

All videos are brilliantly curated and explained thoroughly. Great way to teach and learn.

[@dataschool]

Many thanks!

[@joro8604]

One of the best Pandas videos ever. If i had saw this a few days ago, it would have saved me a few days of frusteration

[@dataschool]

Glad it helped!

[@samc2481]

Best pandas tutorial so far on the internet... Thanks Kevin. Great work. hope you can do more tutorial on applied Tensorflow & ML

[@dataschool]

Thanks for your kind comment, and also for your suggestion!

[@bloomingtonvolleyball639]

You are such an awesome tutor that I felt this is the best channel for pandas. Please keep posting. Thanks a lot.

[@dataschool]

Thanks so much!

[@jongcheulkim7284]

Thank you again. whenever I watch this again, I am learning new.

[@dataschool]

Great to hear!

[@NemoNemoNemo.]

Sometimes when I have free time I watch your videos. Always walk away with something new and useful. Thanks for the great vids.

[@dataschool]

Awesome! That's so nice to hear.

[@chaithanyavamshi2898]

Thanks a ton for the great series, Kevin!. I have watched all your videos and Pandas series videos are still go to watch even in 2021. You are a life savior, Sir!

[@dataschool]

You are so very welcome!

[@TheOntheskies]

Thank you. Your explanations are well-paced and easy to understand and most importantly sticks to memory.

[@dataschool]

Thanks for your kind words! 🙏

[@aleksandramazurek1364]

This is amazing! The way you explain the complicated stuff is fenomenal, thank you

[@dataschool]

Thanks, I really appreciate your kind words!

[@parthibank280]

Incredible explanation which is very easy to understand. One can give this much of a clear explanation on stuffs only if they are confident with what they are talking about and you are nailed it buddy.

[@dataschool]

Thanks very much for your kind words!

[@rajsoniism]

every minute i spent here more than worthy.The way you teach to really good.

[@dataschool]

Thank you! 🙏

[@UncleLoren]

Though in some cases, APPLY may not be as fast as using other approaches, that difference is usually negligible. Kevin doesn't mention that here, in the interest of time/audience; HOWEVER, he did make this very point in an insightful Stack Overflow post that I recently came across. In fact, there was a subtle advantage to using apply, instead of the other alternative, in that it preserved the index. So, yes: speed and optimization are great and all, but most of the time, writing code that is simple / readable -- which Kevin acknowledges as being subjective -- will just as effectively get the job done. This video series is wonderful because it's not only palatable to beginners, but I find myself coming back to it and picking up certain details of why things work as they do, that may not have *stuck* the first time around. It can initially feel overwhelming that Pandas gives you so many options on HOW to do things, but I've come to appreciate the value in wielding a proverbial swiss-army knife, and I've started to recognize which situations favor using one instrument over another. Thank you, Kevin, for being a great teacher, and for presenting content -- beyond this video series, at that -- that appeals to users having a range of experience.

[@dataschool]

Thank you *so much* for your thoughtful comment and your kind words! 🙏

[@Ganeshkakade454]

Hey...great Series Sir..learning Pandas module from your side is such a boon..Thanks,Love from India!!!

[@dataschool]

Thank you!

[@nas209]

exceptional videos, it has the depths and info needed to catch up with data scientist and with remarkable teacher, well done, thanks

[@dataschool]

You're very welcome! Thanks for your kind words!

[@scottlucas3710]

Wow. Excellent job of explaining map vs apply methods not simply explained elsewhere (even by Wes)

[@dataschool]

What a nice compliment, thank you so much! :)

[@hyakushiki23]

I just found your videos and they are awesome. Your explanations and examples are easy to understand, but also very practical. I am looking forward to watching the other 29 videos

[@dataschool]

Great to hear!

[@twofour1969]

Thanks Kevin for devoting your time to provide such good series videos.

[@dataschool]

You're welcome!

[@whatwasisaying1338]

I'm glad I found your videos! You make understanding these concepts much easier!

[@dataschool]

Thanks very much for your kind words!

[@vaibhavkathale5527]

Great videos, perfect explanation. Best teacher on KZbin for Data science... Waiting for your videos

[@gopalagr85]

I followed all the videos of this series and course content is awesome and your teaching style is tremendous

[@dataschool]

Thank you so much!

[@nghiepcrypto7034]

Corey brings me here. Your videos are great! Keep up the great work

[@dataschool]

Corey is great! 🙌

[@killthedark7283]

4-8 hours to make a less than 20 mins video? Gee, I appreciate your work

[@dataschool]

It's true!

[@nasreenbanu2245]

Hats off to you .You made me pro in coding.Thanks a lot..keep going

[@dataschool]

Thank you!

[@saurabh75prakash]

Excellent tutorial, your video series is a must watch for anyone serious in data science.

[@dataschool]

Thanks very much for your kind words!

[@MrNicfeller]

Just finished all 30 videos. Thank you very much Kevin! You are a great teacher!

[@dataschool]

Awesome! Congrats on finishing the series! :)

[@hpchen5402]

This is a great video series on pandas. Really concise and clear. Many thanks for the effort!

[@dataschool]

Thanks for your kind comment! I enjoyed creating the series :)

[@deborahweissner8527]

Thank you for your clear and concise explanations. Such a pleasure to watch your videos.

[@dataschool]

Wow, thank you so much for your kind comment!

[@LatinMusicLover88]

Thanks!

[@dataschool]

Wow, thank you so much, I truly appreciate it! 🙏

[@brendensong8000]

Thank you for sharing! I learn so much from every video!

[@dataschool]

I'm so glad!

[@PradeepKumar6]

Thanks Kevin for such a nice tour on pandas....I enjoyed it a lot. By the way, In future if you decide to start it again, please enlighten us with the "Reduce" function as well. Thanks again for helping people so much. I know it is really hard to make a video. It takes lot of time and effort. Take Care, Good luck to your projects

[@dataschool]

Thanks so much for your kind words! Glad the pandas series has been helpful to you. You're right, it is a ton of effort to make the videos, but definitely worth it :)

[@anAlokDubey]

You are really a best teacher :-) your videos are very helpful and very informative. Thanks you so much 😊😊

[@dataschool]

Thank you! 🙏

[@lindsaybreidenbach8658]

Saved my butt for finishing my hw today. Thx!

[@dataschool]

That's awesome to hear!

[@kenwallace127]

I really appreciate how clear and thorough you are with your explanations. Thank you for this! One question I have, when you split on the comma, I understand how it creates a list of strings. Can you explain why the strings do not show up in quotes, though? Don't most other string list elements? Thanks!

[@dataschool]

Glad you like the videos! Regarding your question, pandas doesn't display quote marks around strings.

[@ilyastrojnov7627]

for simple operations like len and sum, no functions are needed, for example: df ['Len'] = df [Column_with_text] .str.len (), it will be much faster. To calculate the sum or average, we can choose an axis: df ['Total_for_rows'] = df.sum (axis = 0) or df.loc ['Total_for_columns'] = df.sum (axis = 1)

[@astitvasrivastava7197]

we can also get the first element of the list of strings by using : train.Name.str.split( ' , ' ).str.get( 0 )

[@surbhiagrawal3951]

Hey Yours above solution worked fine for me but when i am trying with apply function , it is giving me error TypeError: 'float' object is not subscriptable Also the list which i am passing is a list of strings only . def tryfunc(list1,position): return list1[position] hi["new"]=hi["WebframeDesireNextYear"].str.replace(".", "").str.split(";").apply(tryfunc,position=0)

[@aminsama]

this helped me in better understanding the fundamental difference. thank you so much.

[@dataschool]

You're very welcome!

[@apostolosmavropoulos177]

You are a sweetheart bro! Thank you so much, the whole playlist is perfect

[@dataschool]

Thank you!

[@vlinkin22]

Nice, finally I understood the apply method Many Thanks !

[@dataschool]

Great to hear!

[@ItsWithinYou]

Excellent! I am much clear now on this topic...Many thanks!

[@dataschool]

You are welcome!

[@sandeepvarma4540]

Thanks for the series. Very helpful. Would recommend to everyone who want to kick start.

[@dataschool]

You're very welcome! And, thanks for your kind comment!

[@thethirdkulture3314]

Thank you so much for this! You're so clear and concise!

[@dataschool]

Glad it was helpful!

[@ravinduabeygunasekara833]

This is an awesome tutorial! LEarned a lot from this. One question, what if I want to create 2 new columns named "First Name", "Last Name" and then split the values in Name column into those 2. Ex:For 1 st row, the "First Name" = Mr. Owen Harris and "Last Name" = Braund. Could you share the workaround to get this done?

[@chinamyx]

Hey thanks for sharing! You lessons are clear

[@dataschool]

Thanks!

[@mansiarora5899]

Great videos 👍 I have learnt a lot from them. I have one question it would be very great if you find time to answer this. I have a list of string values and I am trying to create a new column and the value should be 1 if the value given in the list exists in any of the column of dataframe else the value should be 0. Eg. There is a dataframe with cols A,B,C,D and I have a list names test which has some values in it. If the value exists in either A,B,C or D then the first row of new col E should have 1 else 0 and similarly for the next row

[@navishagarwal1736]

A clarification on the apply function. In the example where you apply it on the Name series, how is it that you can use the str.split function on the 'Name' series without the apply function in the dataframe but for 'len' you need to use the apply function?

[@DookyButter]

@Data School, I did some testing and I have found that if you type the lambda function @10:35 a little differently, you get a more efficient processing time: import timeit tic = timeit.default_timer() train.Name.apply(lambda x: x.split(',')[0]) # Method 1 toc = timeit.default_timer() method1 = toc-tic print('Method 1:', method1) ticktic = timeit.default_timer() train.Name.str.split(',').apply(lambda x: x[0]) # Method 2 toc = timeit.default_timer() method2 = toc-tic print('Method 2:', method2) if method1 < method2: print(f'Method 1 is faster by {method2/method1} times.') else: print(f'Method 2 is faster by {method1/method2} times.')

[@dataschool]

Thanks for sharing! 🙌

[@I..Not..IVY11]

Hi sir , No doubt you have a great command in data analysis and a excellent way of teaching as well . Can you start a Machine learning and deep learning course too.? it will be a great help for me who wants to learn machine learning and deep learning

[@MagnusAnand]

EVERYBODY: remember to always use vectorized operations when possible. They are many orders of magnitude faster than using "apply", which is not vectorized. - For instance, instead of: drinks.loc[:,'beer_servings':'wine_servings'].apply(max, axis=0) - Use: drinks.loc[:,'beer_servings':'wine_servings'].max()

[@Edy54321]

Thanks for your videos. They really do help build the knowledge base for beginners like myself

[@dataschool]

You're very welcome! Good luck in your daat science education :)

[@riderblack6401]

Best python tutorials in the sea of the net you can dig. I promise subscribe it!

[@dataschool]

Thanks so much for subscribing, and for your kind comments!

[@calluma8472]

At 10:35, could we use a lambda function as follows: train.Name.apply(lambda x: x.split(',')[0]) or is that a bad way to do it? PS very much appreciate this video and your whole way of walking through these topics.

[@dataschool]

Yes, I think so, but I'd have to verify. Glad you like the videos! 👍

[@samshah3667]

Thanks for the great videos. I just finished watching all the 30 videos in this series and enjoyed learning Pandas. You are a great teacher! I also signed up for the newsletter. I have a question and hope that despite your busy schedule, you can either answer me, or at least point me to the right direction: How would I add a column to a data frame, which holds aggregate data from the same dataframe? Let's say I have a dataset of "expenses" over the last 5 years. I want to add two columns to the dataset, and for every row, the first column will hold the "year to date" sum of all expenses (including this one), from the beginning of the year, and the second column will hold the "month to date" sum of all expenses. How can I accomplish this? Thanks!

[@dataschool]

Thanks so much for watching the videos, and for your kind words! Regarding your question, I feel like it's possible but I'm not sure exactly how. If you are interested in joining Data School Insiders, you could post this question in our private forum and see if someone happens to know: www.patreon.com/dataschool

[@jorgev4656]

thank you very much. the best explanation of youtube.

[@dataschool]

Wow, thanks!

[@sureshkumartwarakavivenkat6760]

Very good explanation please run a case study from problem definition onwards

[@ahmadaburizaiza6598]

Thank you so much for this video and for all the other videos in your amazing channel. I just found a solution in this video for a problem I was struggling with :)

[@dataschool]

Awesome! Great to hear! :)

[@serdarb8995]

It was a great video series. Thank you very much for your great effort. You are great Kevin

[@dataschool]

Thanks! :)

[@Amitsingh-mh3lx]

your video helping me to much thanks from india

[@aminasgharisooreh9243]

Thanks. it is a great series about DATA SCIENCE

[@mobix6354]

Hey, np.argmax returns the indicies not the column name. I'm guessing the function changes over time. Could you tell use how to get the column names instead?

[@chicorebello]

I have the same issue. Did you figure it out?

[@RavinderSaluja]

@@chicorebello Use df.idxmax(axis=1). That should do the trick.

[@youngzproduction7498]

Thanks for your effort. It never gets old.

[@DonDynAyt]

First of all, thank you very much for the time and effort you put into this video series it has been very helpful. Moreover, would you know how to subtract from a column a substring that is in another column?

[@dataschool]

There's probably a string method that can help you, but I don't know which one off-hand... sorry!

[@nimesharya909]

This was great, i did the exercise and it was great..I will sign up to your news letter.

[@dataschool]

Awesome! Here's the link to sign up for my email newsletter: www.dataschool.io/subscribe/

[@xinwang68]

The video is perfect, and i studied a lot from it, thanks.

[@dataschool]

You're very welcome!

[@dineshpandey5008]

Thanks for this , It is really helpful, but in my case, it returns 0 and 1 instead of column label df.loc[:,'Parle':'Soap'].apply(np.argmax,axis=1)

[@ankhts]

This is amazing technique...Many thanks for the video

[@boubacaramaiga4408]

Clear explanation, excellent tutoriel. Many thanks.

[@dataschool]

You're very welcome!

[@alexsandroignacio761]

Very nice teachers i ever seen.

[@dataschool]

Thank you!

[@meitalostikar3122]

Thank you so much! All your videos are amazing!! I learned so many new things

[@dataschool]

Thank you for the kind words 🙏

[@denisesagan]

You should be an English teacher. Very pleasant pronunciation and easy to understand for non-native English speakers

[@dataschool]

That's very kind of you to say, thank you!

[@veggiet2009]

Really like these informative videos! I was hoping you'd get to a video on multi-index

[@dataschool]

Glad you like it! I cover multi-index a tiny bit in my DataCamp course: www.datacamp.com/courses/analyzing-police-activity-with-pandas?tap_a=5644-dce66f&tap_s=280411-a25fc8

[@dataschool]

You might like to know that I just released a video about the MultiIndex: kzbin.info/www/bejne/qpS1eJRoqNSWY8U

[@sandhya_exploresfoodandlife]

hi - Thanks so much for this! i have a question.. how would you use the apply function on two columns at the same time?

[@19aavila]

with applymap(), it's at the end of the video :)

[@wildlife_and_drones1320]

Clear and concise. Thanks

[@dataschool]

You're welcome!

[@louisbademosi9250]

You know you like a channel when you find yourself trying to subscribe only to find that you already have

[@dataschool]

Thank you! 🙏

[@yossarian2909]

Lucid explanations.. i love your channel!!

[@dataschool]

Thanks!

[@mygcoder]

Hi Kevin and thank you for this video series. I wanna ask you a question, I used np.argmax function, but it didn't give me the column names. It gave me the orders of the max columns. I checked many times, and I'm sure that I exactly write the codes as yours, and tried argmax with some other dataframes, always it gives me the order of the columns not the column names. Can you explain this, and how I get the names?

[@sachindeshpande1]

Yes, I am also facing same issue. Have you solved this problem?

[@mygcoder]

@@sachindeshpande1 Yes I solved it. That is because of different versions. Kevin uses an old version. New versions are like what we faced.

[@sachindeshpande1]

@@mygcoder Thanks a lot, I thought so but not checked with old version.

[@dataschool]

Thanks for figuring it out!

[@gilberthor1988]

Congratulations for this video! I have one question: how can I, for exemple, get each string of the column "Name" and put them into a sigle list? For example: [Futrelle, Mr., Jaques, Health, Lily, May, Peel, Allen, Mr., William Henry]. Thanks in advance!

[@yellowbug772]

Really appreciate it, thank you, Kevin!

[@dataschool]

You're welcome!

[@benben341]

Thank you for your video. If your table has a large amount of catogories ie coutries, colours etc - i use pd.factorize(d.columnname) to create a number list for scikit as it cant use the label data - so a question for later videos - whats the best policy map or factorize ?

[@dataschool]

Thanks for your question! I think the short answer is that you should use whichever one makes the most sense to you and is the least work :)

[@robindesbois07]

Hey Kevin, thanks for your videos. They are so helpful. One question: how to modify a data frame by adding an additional column if nested if statement? I tried to use apply lambda with nested if,but it looks like it only accepts the single if else, that it. thanks in advance.

[@dataschool]

Glad the videos are helpful! As for your question, I'd have to see the code to understand what you are trying to do.

[@spytheman]

Thanks! I overcome a lot of questions that many online videos couldnt gave me the answer! Btw, when doing a lambda expression, while we can lambda x: x[1] to use its neighbouring data, is there a way to get the data below it rather than on the right or left of it?

[@esdrasdebrito]

very good examples. Thank you so much

[@dataschool]

You're welcome!

[@jrochagrupo]

Hi Kevin, Great Tutorials, learned a lot. Thanks.

[@dataschool]

You're very welcome!

[@riderblack6401]

awesome vid! I learn so so Much from you Kevin! Keep it up!

[@dataschool]

Thanks! I appreciate your support.

[@mamtavenugopal841]

I wanted to know how will you use apply method to compare two columns say beer_serving and spirit_servings and then if beer_Serving is greater than print in another column called 'More servings'

[@dataschool]

You wouldn't use apply, you would probably just start by writing a filter: kzbin.info/www/bejne/aHKpeIOag9NnfK8

[@annalukowska7080]

Hi , How do I select specific columns based based on row condition please? If Row= value1 select only 2 columns or if Row=vaule2 select 3 columns so on. Thanks

[@GreyCosSil]

Just amazing! thanks a lot for sharing your knowledge.

[@dataschool]

Thank you!

[@MrBhargavafirst]

very knowledgeful thanks for this

[@dataschool]

You're welcome!