Method chaining in Pandas

What is method chaining in Pandas? How does it work, and how can you use it?
In this video, I introduce method chaining, including the use of .loc, lambda, and assign. If you've been confused by any of these topics, you should get a better understanding from here.

[@joaquinurruti2628]

I come back to this video again and again every few months to recall, It's gold powder

[@ReuvenLerner]

I'm so delighted to hear it; thanks for the warm and kind feedback.

[@jingangmiao8367]

Instead of lambda functions, I find .query a lot more readable: ( df .assign(...) .query('amount_per_mile > 5') )

[@ReuvenLerner]

I've never really gotten into using query, even though I know that it can have some performance benefits. But it can also fit nicely into this kind of chain.

[@VladPalacios]

total newbie to Pandas here, I like this aproach, but seems to me, where it beraks the flow, is when you need to transform the data and keep it, .... like fill the na's, delete some columns... I'll try figure it out on my own and learn.... but really like the video

[@ReuvenLerner]

Thanks for your kind remarks; it definitely takes time to get into the Pandas way of doing things. I've really enjoyed using method chaining in the last year or two, and I encourage you to try it before you knock it.

[@VelkoKamenov]

Great content! This concept is especially useful for people coming to Python from R and the tidyverse syntax with the pipe operator (%>%) for chaining functions. About filtering rows how do you feel about using query() instead of loc?

[@ReuvenLerner]

Thanks so much! I'm not a big fan of query, just because it introduces a totally new and different syntax -- it's sort of like embedding SQL inside of a program. That said, I admit that query can sometimes make things run faster, and can also look a bit clearer.

[@taqial-shamiri3377]

Love the way you teach it and the examples.

[@ReuvenLerner]

Thanks so much!

[@onenabeable]

thanks... clear explanation❤

[@ReuvenLerner]

Thanks so much; happy it helped!

[@horoshuhin]

same here. i got converted by Matt Harrison a few years back. and never looked back!

[@vigneshkrishnan424]

The one thing I felt was missing coming to python from R background! Thankyou !

[@ReuvenLerner]

I'm delighted to hear it!

[@elu1]

Simply brilliant! 👍

[@ReuvenLerner]

Glad you enjoyed it!

[@Lnd2345]

Great video. I think chaining is great and I use it but sometimes gets overused which makes those codes unreadable with zillions of lambda functions.

[@ReuvenLerner]

Yes, indeed!

[@imothar]

I love method chaining as well, but do you know if the copy_on_write changes in pandas 3.0 will make it more difficult to use? Maybe I've just misunderstood the explanation 😅

[@ReuvenLerner]

On the contrary, I just looked through the docs for 3.0 and copy on write, and it looks like this sort of chainig will definitely continue to work, whereas more direct assignment will not. When they talk about "chained assignment" not working, they're referring to something else, such as df['a'][100] = 200. That is already bad, and will stop working altogether.

[@user-fn5rx3gp2t]

Hi I wanna ask you two questions where can I contact You?

[@ReuvenLerner]

Feel free to e-mail me at [e-mail] -- I'm not always able to answer individual questions, though.

[@pietraderdetective8953]

with all due respect to Harrison, his version of Pandas method chaining is too much overdone IMO...combined with his love for lambdas, it makes it even harder to read. too little whitespace, too many lines of codes getting cramped together...could be just his style of writing code which is perfectly fine for his use case. If you take a look at the profile of those who complained about this style, many of them writes much more difficult languages than Python...but they complained about the readability!

[@ReuvenLerner]

Aha, got it. Like any tool or technique, it can be abused. I think of method chaining as a way to help clean up my code and my thinking. But if it makes things *more* complex, then that's not very helpful.

[@RafsanSiddiqui77]

Great one as always, Reuven! Apart from the readability of method chaining, is it better to use .loc to find total_amount > 100 and passenger_count > 5? I usually do it like this: df[(df['total_amount'] > 100) & (df['passenger_count' > 5)]. I would also do the amount_per_mile like that within [] without assign. Why shouldn't I do it that way?