Understanding (and avoiding) the SettingWithCopyWarning in Pandas

Just about everyone using Pandas for data analytics encounters the SettingWithCopyWarning. What does it mean? How does it happen? And how can we avoid it? In this video, I explain all of these things, including how to fix the problem if and when you encounter it.
The Jupyter notebook for all of my videos are at github.com/reu.... And don't forget my free, weekly "Better developers" newsletter at BetterDevelope...!

[@cstephens16]

This was the best explanation for that warning i have ever come across. can't wait for you pandas workout book to come out.

[@ReuvenLerner]

Thanks so much!

[@aflous]

This is by far the best explanation I've ever seen for this issue 🤩

[@ReuvenLerner]

Glad you enjoyed it, thanks for the kind words!

[@_truthful_q_]

Nice video, thanks Reuven! The why is just as important as the how 🙂

[@ReuvenLerner]

My pleasure!

[@tomkmb4120]

Great video Reuven, this warning always boils my blood lol

[@ReuvenLerner]

Excellent, I'm glad that it helped!

[@TheCJD89]

Great explanation. This problem happens so, so often! :D

[@ReuvenLerner]

You're not alone; glad this helped!

[@DookyButter]

Best explanation I've seen so far and under 10 min! Thank you because I absolutely loathe that error.

[@ReuvenLerner]

Thanks for the kind words; I'm glad you enjoyed it! (And yeah, that error is the worst...)

[@KevinBurns]

Great overview Reuven and something I have fallen into a number of times when working in Pandas 😀

[@ReuvenLerner]

You're not alone! Glad to hear it helped.

[@samarthnikhra8827]

Thank you so much, this explanation was so precise and easy to understand. It helped me a lot. Now I am a subscriber of your channel :)

[@ReuvenLerner]

Fantastic; I'm delighted to hear it helped!

[@shadrinan90]

Great explanation, thanks!

[@ReuvenLerner]

I'm delighted it helped!

[@RenatoCosta1]

Crystal clear explanation! Thank you very much for that!

[@ReuvenLerner]

Delighted to know you enjoyed!

[@horoshuhin]

that's a good one. like always excellent explanation. thanks

[@ReuvenLerner]

Glad you enjoyed it!

[@vincentcrescente3036]

Awesome, thanks man

[@ReuvenLerner]

Glad it helped!

[@somethingness]

This was great. Thank you!

[@ReuvenLerner]

Glad you enjoyed it!

[@minhdzung]

Thank you sir!

[@videotape-]

sir, you are great!

[@ReuvenLerner]

Thanks so much for the kind words!

[@mosama22]

Man, I “LOVE” you …..

[@ReuvenLerner]

Thanks so much! You have no idea how much this kind of comment means to me.

[@mattp6460]

amazing thanks

[@ReuvenLerner]

Glad you enjoyed it!

[@marcdelabarreraibardalet4754]

Excellent video. But I'm still puzzled by my case. Oversimplifying, I'm trying to do data['y'] = np.log(data['x']) This should bring no warning. The curious thing is that if I do it in a jupyter cell, I don't get a warning. However, I put this inside a function, and I get the SettingWithCopyWarning. Can you think what can be going on? To clarify, the ONLY thing that changes is that this line being inside a function or outside.

[@kmayumij]

You could rewrite it to: data['y'] = data['x'].apply(lambda x: np.log(x)). This solution will not explain why are you getting that warning, but probably will get rid of the warning.

[@ReuvenLerner]

Hmm, I'm not sure why you would be getting that, except that (maybe) the data frame is itself a view, or is sharing data with another data frame?

[@albertjimenez6605]

Mil gracias! Tienes un nuevo subs.

[@ReuvenLerner]

Delighted that you enjoyed it!

[@codeministry]

Best explanation for human

[@ReuvenLerner]

Glad to know it helped!

[@tanzinabinteyousuf6184]

what if i fix it still the warning shows. what should i do?

[@ReuvenLerner]

If you have a single set of square brackets and you're still seeing the warning... that's weird! Would you please post your code here? I"ll see if I can understand and help.

[@tanzinabinteyousuf6184]

@@ReuvenLerner thanks for your response. i had issue with two lines. fixed both the line and the warning finally went off.

[@tizianrettig5509]

This just does not apply to the issue si am seeing.

[@ReuvenLerner]

Sorry to hear that - I'm guessing that it's a related problem, even if not this precise one.