Lambda in a Loop is a Code Smell

[@salamonandris]

You can do this with keyword binding: `lambda i=i: print(i)`, where i is the loop variable. Partial application is nice but not necessary.

[@gabi_kun3455]

But generally in python is a good practique to use the things to what they're made for, with this you can guarantee that your python logic is fine

[@faremir]

@@gabi_kun3455 But lambda and more precisely lambda arguments were proposed like this for this exact reason. What Salmonandris said is the correct usage.

[@JoQeZzZ]

@@gabi_kun3455 keyword binding is exactly made for this though. The mistake stems from a misunderstanding of how lambda's work. A lambda is defined by: lambda [arguments] : [expression] So obviously you need to pass the arguments you want to use to the function. If you don't pass the arguments it would make sense that it uses something in from outside of the narrow lambda scope. It might be good to have a linter warn you "hey, you're using a variable outside of the most narrow scope", but that IS how Python uses scopes and lambdas are no different.

[@gabi_kun3455]

I don't think that guys, if we analyze this more deeply, this only works if the lambda is meant to take no arguments, also you should try to take more care of the argument signature, please avoid making them ugly

[@salamonandris]

@@gabi_kun3455 You are right, binding the loop variable to the function's input parameter is somewhat abusing the way closures work in Python, and it can be confusing to newcomers (especially if you name them the same). I almost never do this, but it is good to know nonetheless. One place a regularly do use this is creating threads in a loop during testing code for thread-safety or load management.

[@terra_creeper]

One big factor of this bug, at least to me, is how lambdas are percieved. If I used a named function in this example, it would seem obvious to me that the value is only looked up when the function is called. But for some reason a lambda doesn't feel like a normal function to me, which is why I almost always make this mistake and forget to use partial.

[@dualbladedtvrecords4383]

This "problem" is not really a problem, at all. The lambda is created within the same **closure**, so the value of the variables isn't captured.

[@terra_creeper]

@@dualbladedtvrecords4383 Yes, but in this specific case this behaviour leads to a bug. And I was just stating that, to me, this behaviour feels unintuitive with lambdas, even if it feels intuitive with regular functions.

[@gregorymorse8423]

Not only lambdas created closures. Normal functions can be defined in loops too.

[@Erdnussflipshow]

Literally had this problem 2 days ago. Was trying to add callbacks to buttons, all buttons did the same thing. scratched my head for a while before I figured it out Great video and explanation

[@waffleman4503]

I dealt with this exact same problem in C#. Same situation as yours.

[@mika2666]

This looks like the same problem we had in javascript before the addition of "let" and "const", filling the arguments at lambda creation is a workaround to a problem created by the language that should not exist.

[@Kenionatus]

I don't see how const would fix that. You need a variable that changes/gets recreated for every loop iteration if you want to have any kind of iteration over something or even a counter.

[@kaosce]

​@@Kenionatusby referring to const and let, he meant that before them was var which basically create a global variable which you can use before it's declaration and update when you don't expect it (deprecated BTW)

[@Gamesaucer]

@@Kenionatus Being able to declare things as "const" means that you can be confident that it hasn't changed from when it was first assigned, which is an incredibly helpful feature. Things should be immutable unless they have a reason not to be, and with "const" you can enforce that so you get no nasty surprises when something doesn't do quite what you expect. You get an error, or the result you want, ruling out the third option, a result you don't want. When it comes to this specific case though, in JavaScript, for..in / for..of loops close over the loop body _including_ the declaration of the variable extracted from the iterator, so every iteration you have an entirely different variable instead of reusing the same one repeatedly. I don't really know why Python _doesn't_ do that (why should the variable be accessible outside the scope of the loop body when it's just going to get automatically overwritten anyway?) but you can write JS like "for (const [taskId, func] of work)" whereas something like "for (const i = 1; i < 10; ++i)" just raises an error almost immediately, and thus is obviously wrong. It's made JavaScript a lot more predictable.

[@snoozbuster]

@@Kenionatus kaosce is essentially right, but to be more specific - variables declared with “var” are function scoped and hoisted to the top of the function (so you can even use a variable “before” it was declared). Variables with “let” or “const” are both block scoped, and their declarations are not hoisted to the top of their containing blocks (so using them before their declaration, even if its in the same scope, raises a ReferenceError). Then you have your traditional ways of making block scopes - for loops, while loops, case statements (at least as long as you use braces, which are unfortunately optional), and the such. bonus fact: for(const i = 0; i

[@PhilmannDark]

My workaround for this was lambda fut, l_task_it=task_id: ... which makes a copy of the value of task_id when the lambda is being created (and not when it's being evaluated). In the lambda, it's then safe to use l_task_id. But it looks a bit weird and there is a chance that a coworker will think this was a mistake and delete the "useless" lambda parameter.

[@unpythonic]

What a wonderfully subtle bug! Great example, and I'll be on the lookout now.

[@b4ttlemast0r]

So this is basically just capturing a variable by reference vs by value. In C++ you have to specify this when using a lambda expression, whereas in Python people probably aren't aware of this and the two ways of capturing require completely different syntax.

[@mayureshkedari3799]

I would love to see more content in the form of series on 1. threading vs multiprocessing 2. distributed computing in python using ray framework or its equivalent

[@cestlacroix]

🙌🏻

[@Cookie-mv2hg]

I haven't got this kind of error YET. It's just interesting that every once in a while I come back to see the problems that I don't understand at first and recognized that you have make a video explaining that. I see this as a sign of my improvements

[@pants64]

This is why I like C++ lambdas. They let you choose whether to capture by value or by reference, so this probably wouldn't be a problem in the first place and it certainly wouldn't need a library to fix.

[@gabi_kun3455]

But that would need new syntax, in the time i have programming with python i noticed that they don't add innecesary new syntax, i guess it is to avoid complexity, in C++ would be hard to implement something like functools.partial i think

[@pants64]

​@@gabi_kun3455 Ironically the simple lambda syntax lead to more complexity in the video, because a library was needed to solve the problem it had caused. The last two Python versions both added a new syntax (the except* syntax and the match statement), so I don't think a new lambda syntax would be out of the question. Btw the C++ standard library has something pretty similar to functools.partial, it's called std::bind.

[@faremir]

There's no issue with lambda itself in python. Correct usage of such case is defining the variable in lambda argument and not in the body of the lambda function. "lambda fut, t_id =task_id : ... "

[@pants64]

@@faremir Oh, good to know. I kinda just assumed there wasn't syntax like that since it wasn't shown in video.

[@gabi_kun3455]

@@pants64 in python it is very common to import libraries in that way, you only need to catch the style and you'll love it, about the new features they are pretty new so i still don't use them, but that there's more than one way to do something does not mean that one replace the other or that is innecesary, i'm refering to things like match vs if, that goes completely against python design principles, all have its specific use case, surely the new features have reasons why they were added, also i don't know C++ so i'm gonna not enter in that theme

[@dwarftoad]

Another way to understand this problem. Consider replacing the lambda with a reference to a normal function defined elsewhere (with the exact same parameters as the lambda!) and see what would happen -- and how you would fix it -- you must either add an argument, i.e. the value is supplied when called, or do some kind of partial application of the function that captures a copy of the value that is used later when invoked. (e.g. use a function taking the task_id and returns the lambda `def make_completion_func(task_id): return lambda fut: print_task_completed(fut, task_id) ... add_done_callback(make_completion_func(task_id))` , or the equivalent `add_done_callback( (lambda id: lambda fut: print_task_completed(fut, id))(task_id) )` or the IIFE pattern mentioned.)

[@fionnbracken]

Golang has same issue but they have just accepted a proposal to change the behaviour of lambdas in loops lol.

[@NaifAlqahtani]

I LITERALLY had pylint call me out on this yesterday and couldn’t for the life of me figure out why. Even after googling and reading online. It just didnt make sense. What a coincidence to see you upload a video on this exact topic one day later

[@mCoding]

Great to hear, I'm excited this vid was able to help you!

[@markasiala6355]

I hit this issue this afternoon. Luckily googling took me to the Pylint documentation which showed a workaround and linked to a StackOverflow page on the topic which helped me solve it (using functools,partial). Of course this was after ignoring the pylint error for some time thinking it didn't matter. :(

[@maccsguitar]

this should be "closures in a loop is a code smell"

[@GabrielSoldani]

more specifically, capturing the loop variable in a closure. this isn’t even a code smell imo because it results in a bug, not a design weakness. the fix actually creates lambdas in a loop as well, but they capture the functools.partial parameter, which is in an inner scope, rather than the loop variable. imo this video was way too simplified for views rather than the usual simple but technically correct explanations that we’re used to in this channel.

[@abadhaiku]

OH, so _that's_ why the Java compiler says that "values used in lambda expressions must be final or effectively-final" and makes you copy it to a new variable before using it.

[@snoozbuster]

Weird that it can’t just… compile in the copy to local variable itself. Most other languages (JS and I think C#) basically do that automatically for you. But yeah, this issue is exactly what that message is warning you about.

[@NotAUtubeCeleb]

I had this happen trying to create a loop which created streamlit buttons. I wanted the lambda to set a "current" value in a dictionary. Instead, every button set the value to the last item in the loop.

[@pinch-of-salt]

I felt this was very obvious. Almost all languages do this, this is a classic example of variables from an higher order functions causing confusion for the inner function's arguments. Understanding variable scope and function argument binding/partial functions is very important for all.

[@Slackow]

Java doesn't do this at all, passing in variables from an outer scope to a lambda in Java requires that variable be "final or effectively final" and it passes the references at runtime, not lazily. This completely sidesteps this issue.

[@MilChamp1]

I usually call myself self-taught coder. However, when I am pushed to think about it, I honestly feel that this channel has taught me more of what I know than anything or anyone else (other than perhaps StackOverflow). Every video is interesting, and every old video still somehow maintains relevance. Thanks James.

[@beyondcatastrophe_]

In my opinion, it's not the lambda itself that is the problem, it's the asynchronicity. If you were to call something like `map`, it's perfectly fine, because it is evaluated right away

[@jacquesfaba55]

Haskell solves this using Seq instead of Map (python is cringe)

[@faremir]

In my opinion - well in fact - it's not asynchronicity or the lambda itself. Calling the lambda correctly "lambda fut, t_id =task_id : ... " would fix the issue. It's just that people are using things they do not understand so code such as that becomes painfull to read.

[@JoQeZzZ]

@@faremir Yes, I agree. This is how python uses scope: L(ocal), E(nclosing), G(lobal), B(uilt-in). The lambda is just following this rule and task_id DOESN'T EXIST in the local scope, so it uses the enclosing scope. This makes perfect sense and rewriting the lambda to an explicit function definition and a function call shows exactly how this dumb mistake happens. I'd expect any linter to show a warming though, like "are you sure you want to use something out of the lambda scope?"

[@0xCAFEF00D]

map still has the problem if you don't immediately iterate the map in the for loop. But I've stretched the example I feel. You shouldn't be creating a list of mapobjects or similar and iterate them later. It's a very indirect way to do work. Depending on scope variables at the same time like in the mCoding example is a step worse.

[@poke_champ]

@@jacquesfaba55 yet you watched a python video. cringe

[@asdfghyter]

i feel like this is as much of a python footgun as a code smell. it’s a bad choice in the design of the language that runs counter to how people would expect it to work. in many other languages you would not have this issue and calling lambdas in a loop is perfectly fine. for example, in rust with its explicitness about lifetime would make this bug impossible, since you are not allowed to have mutable aliasing. and in many other languages the loop variable is fresh and immutable, so one iteration of the loop can never affect another. i believe that if you loop using a lambda, e.g. a forEach loop in js, it’s fine in most situations, except if the lambda catches a variable from an outer scope, which is modified in the loop, in which case you’ll run into the same issue again.

[@lpt_7]

The reason why people make this "mistake" is because in some languages like Java task id would actually be captured and not looked up from the scope.

[@faremir]

Yeah, python lambda is weird, but correct usage would fix the issues. even though it's not mentioned in the video at all...

[@mCoding]

Indeed, I don't mean to fault anyone for this mistake, clearly I even made it myself and there are many reasons lambdas defy programmer intuition. Nevertheless it helps to be aware!

[@ParkourGrip]

Languages with strict mutability never have this problem because you are encuraged to always use immutable variables. The source of the problem is that task_id variable is mutable.

[@snoozbuster]

JS also doesn’t have this problem anymore (at least when using “const” or “let” to declare the loop variable) because the loop creates a new scope. When the function goes to lookup the name in its closure it gets the scope created for that iteration and so everything “just works.” I haven’t worked in Python for close to ten years but i remember having this issue and it was surprising to learn that y’all still don’t have a solution provided by the language (and lambda _t_id=task_id is definitely an obnoxious hack, just like how IIFEs were obnoxious overhead in JS loops for the same purpose back in the day).

[@opticalreticle]

this is definitely not a smelly python channel. keep up the high quality work

[@Christian-mf4jt]

Some languages like C# and Go have previously fixed this problem by creating a new variable instance for each loop iteration. Python is definitely not following the principle of least astonishment here.

[@Pscribbled]

You can scope the variable inside the signature of the lambda as a keyword argument where the default is the value you want. That’d work too and you don’t need to know the functools

[@kyletech4878]

I discovered this exact issue 2 weeks ago and thought I must have just missed the mCoding video covering it.

[@von_nobody]

Funny that recently Go Language fix similar behavior in their loops, and some time ago C# had similar problem, should Python change too this behavior?

[@NoNameAtAll2]

6:58 it's just that function redefinition is considered "expensive" intuitively so def gets moved out of the loop lambdas are considered cheap to create

[@stonemannerie]

Are they? Or are they just "considered"? I couldn't find any source saying they are more expensive. All sources said there is a negligible difference.

[@coarse_snad]

​@@stonemannerie From my limited understanding, there is no difference at all. The same process happens whether you capture variables in a function def or a lambda. Some time back i was wondering whether defining a function within another function was costly, and the answer is no. The function is only actually defined once.

[@mCoding]

Lambdas are functions and they are very nearly exactly the same amount of expensive to create under the hood. Also note that function creation at runtime like a lambda or local function does NOT mean compiling the source to bytecode, which is relatively expensive. The source is compiled to bytecode, lambdas and local functions and all, before runtime. At runtime, creation of the function or lambda object is little more than creating a dummy object and sticking a few attributes like the globals dict and cell variables onto it.

[@Ziggity]

Funny thing, this is a very common issue in Go when using goroutines (Go's implementation of coroutines\green threads) in a loop. So much so that it's being addressed directly by the developers in Go's next version.

[@AllMightGaming-AMG]

Isn't this already addressed? I had to learn go recently and their docs on closure suggest that it is there. Do you have the link for the issue?

[@quillaja]

Really "fixing" the issue just means people are going to be doing something bad without suffering and learning from the consequences, and therefore continue the bad behavior.

[@kisaragi-hiu]

This pattern also bit me once in Emacs Lisp. Essentially for in Python and cl-loop in elisp both create the variable once for the entire loop and mutates it for each iteration, so lambdas in the loop body will all see the same local variable. You can also just introduce a binding local to the iteration: in elisp this is an explicit let, and in Python this appears to be keyword binding. I don't think it should be thought of as a problem with lambdas in a loop. That implies the fix is to use named functions instead, which is completely wrong. The problem is capturing loop variables directly, and honestly I consider this a language flaw: `dolist` in Emacs Lisp doesn't have this problem, for instance. (The behavior in Emacs Lisp is the same as in Common Lisp.)

[@rikschaaf]

I don't think that the issue is using a lambda in a loop, but using a mutable variable in a lambda. the fact that the loop created the mutable variable is besides the point. This is also why in java all variables used inside a lambda that are defined outside that lambda have to be (effectively) final.

[@shahaffrsshahaffrs5190]

Once again, great content and a very deep understanding of python. I had this problem years ago, and I couldn't find any solution to this "lambda function in loop" problem. At the and I finally understood the problem and how to solve it. after years of watching your videos and gaining so much from your content - thank you for making the process of learning to program better and easier!

[@ZiplawDev]

the first time i encountered this bug i had only been programming for a few months, and it took me the best part of an evening to figure out what was going on

[@benjamin7853]

This is why I love how explicit you have to be with lambda functions when they are done the same way as c++. You know exactly what you're getting and there's no guessing games.

[@palapapa0201]

This is just because there is no way to capture by value directly. I just found out that C# also has this issue that lambdas always capture by reference.

[@mat4x]

I used to use lambda function to make multiple buttons in tkinter and had faced the same problems. I "solved" it by asigning the looped variable as a default to a lambda input and pass that input as the function argument: As in 4:09, it would be lambda fut, t_id=task_id: print_task_completed(fut, t_id)

[@faremir]

Good job. That's actually what the lambda args are for!

[@MiTheMer]

Lambdas in Python feel awkward and so "loose" and limitting. Love to use them in C# but tend to avoid in Py... This particular problem described here is also way easier to mitigate in C# (copy the loop variable to a local variable right before lambda creation).

[@franciscoflamenco]

Everything about lambdas in Python is "smelly". I use lambdas all the time in other languages, but I almost never catch myself writing one in Python.

[@JoQeZzZ]

Like in lambda _task_id=task_id: print_ts(f"{_task_id} completed")?

[@Christian-mf4jt]

In C# the whole problem is fixed since version 5.0, because each iteration gets its own loop variable.

[@Slackow]

@@franciscoflamencothey're not as necessary in python since you can use generators and comprehensions a lot of the time

[@berndeckenfels]

That’s why Java only binds effective final variables (so it can only happen when the content is shared/changed)

[@yxh]

Great video! I also appreciate you breaking away from overuse of memes and keeping the 💯 content unsullied

[@Veptis]

Does the threadpool executor keep track of any childprocesses spawned and can timeout those too? I have a major issue where a subprocess crashes the python process and it seems to be not possible to safely execute it and also catch it timeing out.

[@NoNameAtAll2]

does "task_id = task_id" in the beginning of the loop fix the issue? go has just fixed this exact problem by changing scope of loop variable (new pointer for each iteration)

[@faremir]

Yes. Somethin like "lambda fut, t_id =task_id : ... " would fix the issue. As the issue is not lambda but whoever wrote that atrocity.

[@NoNameAtAll2]

@@faremir no, not in a lambda I'm talking about solution that was too common in go before the change for x in y: x = x ...

[@faremir]

@@NoNameAtAll2 I see (at least i think i see) what you mean. I'm actually not sure but I don't that would fix the issue - or at least is not defined behaviour in python - as the variable is still defined out of the lambda body scope. You can actually have same issue (if you don't properly set lambda args) with lambda if the for loop call is multiple layers up and passed down through functions.

[@0LoneTech]

Not in Python; the for statement creates a block level but not a new variable namespace. def and class do that, so you could do it with another nested function. for x in y: def inner(x): pass inner(x) This evaluates the outer x that for deals with only in the call to inner, and inner has its own local x. But this isn't good style either; inner should be moved out of the loop, to be created only once, and probably out of the function so it won't accidentally access other changing locals. This leads to using a higher order function that produces functions, just as functools.partial does.

[@Gamesaucer]

This is good to know. In JavaScript, extracted values in a for..in / for..of loop are constants rather than being updated each loop, so you can capture them in a closure without any issues arising. The Python syntax looks to me like it should allow the same, but it doesn't. I'm not super familiar with Python though, so perhaps it's a predictable outcome if you're a bit more familiar with the language. Still a yikes from me though, superfluous usage of variables over constants in core programming language features is something I think of as a language smell. Doesn't mean the language is bad (though I do think Python is highly overrated), but it does call for further investigation.

[@snoozbuster]

Old JS (back when we only had “var”) suffered from the same problem for basically the same reason. We fixed it by inventing “const” and “let” and giving them different block scoping semantics so that closures in loops would “just work.” As someone who used to work in Python a lot and really enjoyed it, it’s sort of sad to see that Python hasn’t caught up with that yet. No block scoping feels like such a miss.

[@corejake]

Not sure if functional calls with lambda funcs in Rust are a fair comparison, but I only run them like this, within a for loop that is. Why? It's faster in rust, and a pretty big difference at that too.

[@volbla]

I've come to almost only use lambdas as key arguments in min/max/sorted. It seems extraneous to define a full function for that if i only need to fetch an index or convert something to an int, etc.

[@re.liable]

Wow. I encountered the same issue back then. I got around it with the "default argument" approach. Maybe a shorter example: ``` names = ['name1', 'name2', 'name3'] fns = [] for name in names: fns.append(lambda: print(name)) for fn in fns: fn() # name3 name3 name3 fns = [] for name in names: fns.append(lambda n=name: print(n)) # Default arg for fn in fns: fn() # name1 name2 name3 ``` If my understanding of scoping and closures is correct, I guess the problem was I was expecting the loop (or each iteration) to create its own scope, and therefore the lambdas to be defined within those. Also explains why I was always feeling iffy when accessing loop variables outside of the loop itself, e.g. ``` for item in some_seq: ... item # Last item in some_seq for item in some_seq: if some_condition(item): break item # Offending item that broke the loop ``` --- Maybe the same issue in JS: ``` const names = ['name1', 'name2', 'name3'] let fns = [] for (var name of names) // Issue caused by var fns.push(() => console.log(name)) for (const fn of fns) fn() // name3 name3 name3 fns = [] for (const name of names) // const/let introduces block scope fns.push(() => console.log(name)) for (const fn of fns) fn() // name1 name2 name3 ``` Now I kina wish Python would introduce block scoping lol

[@snoozbuster]

You’re pretty much bang-on. I wish they’d add block scoping to Python too, block scoping is great. Python and JS both had this issue about ten tears ago but sadly modern JS has pretty much eradicated it and Python still has people falling into this trap. Pretty sad because I use to enjoy writing Python a lot more than I enjoyed JS 😄

[@andrewmenshicov2696]

similar to the functools solution, task_id can be put as the argument of lambda w/ default value like this: lambda fut, task_id=task_id: print_task_completed(fut, task_id) not very beautiful, but working and i kinda expected to see this 😃

[@mCoding]

A solution, yes! Recommended by me, no! Clever workaround, though 👌

[@tobb10001]

Nice. I think go recently got an update that mitigates this exact problem from the language side, right?

[@AM-yk5yd]

Never had this porblem in python(as I rarely use it for such thnigs), but I defintely hit it in javascript in the past and used this IIFE stuff to pass variables

[@SteinGauslaaStrindhaug]

It's been a while since I coded in Python, and when I did I don't remember using lambda much (probably because most of it was very straight forward server rpc calls and database retrieval, since I'm mainly a frontender); but lambda is basically the same as an unnamed function (usually used as a closure) in JavaScript, right?

[@snoozbuster]

Basically, except Python seems to begrudge them instead of embrace them for some reason

[@Kenionatus]

I'd rather call it "lamda if you don't immediately execute it". A common case where you do immediately execute it is using it in things like map().

[@danielwilkowski5899]

I'm wondering if that's just behaviour of python, or does it work similarly in other languages with lambdas? I can imagine some langauges have lambdas which capture the state of the variable at the moment of creating the lambda.

[@somenameidk5278]

lua closures are like in Python, they store references to their captures, so if the variable is modified elsewhere (including by another closure that captured the same variable), it will change in the closure.

[@ranexia04]

What about defining a wrapper function (on the global scope) that itself defines the callback function?

[@mCoding]

Possible fix, yes. Recommended fix, no (at least my opinion). :)

[@gicknardner]

now the trick is, will I remember this next time? Or will I just forget and do it again like every other time this has caused problems for me?

[@mCoding]

You won't forget if you just repeat "No lambda in a loop" 100 times. Say it out loud, I'll wait ;)

[@gaborfekete3777]

why is partial in this case better than just write lambda fut, task_id=task_id: print_task_completed(fut, task_id) ?

[@snoozbuster]

It makes it a lot more explicit what you’re intending to do - most any dev from most any language would understand what’s going on. The default argument thing is kind of a hack that works because of how default arguments are evaluated in Python and requires Python-specific knowledge to understand.

[@ethanyalejaffe5234]

This is probably even smellier, but would add_done_callback((lambda x : (lambda fut : print_task_completed(fut,x))(task_id)) solve the issue by immediately calling the outer function with the correct argument?

[@GabrielSoldani]

You’ve just implemented a non-reusable functools.partial ;)

[@faremir]

"lambda fut, t_id =task_id : ... " no need for smelly code.

[@snoozbuster]

@@faremir we may disagree on this, but that’s smelly too 😄 (it is functionally the same as what OP wrote, but the Python runtime is doing it for you - so now you have to know some extra details about how Python itself works rather than just programming fundamentals.)

[@faremir]

​@@snoozbuster No it's not. The outer lambda creates a new anonymous function, which itself returns another anonymous function when invoked. So, there is an additional layer of function invocation, which could potentially result in a performance hit. (Those are the fundamentals you are talking about?) And in most threaded applications that's really undesired overhead. But... you're just making my point. Understanding the nuances of the language you're working in is an essential part of being a skilled programmer. Just as you wouldn't expect a carpenter to be proficient without understanding the characteristics of different woods or tools, you shouldn't expect to write efficient code without understanding language specific features, such as default arguments in lambda functions. Using workarounds that 'just work' without understanding what's happening is a sign of smelly code. Even the usage of the partials is better, but still adds unnecessary overhead. You can run few tests and compare the results.

[@doc7115]

I just made this type of error, and it gave me wonky results. I did something even more stupid, submitting the task using a lambda with local var.

[@alexkolokolov]

Why not just pass the lambda like this - lambda fut, task_id=task_id: print_task_completed(fut, task_id) ?

[@tauiin]

python lambdas feel half baked tbh, I generally avoid them

[@marckiezeender]

I only use lambdas for trivial functions that don't actually perform any logic. I think in this case it is more readable than defining whole functions. for example: lambda: False; lambda x: x

[@codegeek98]

7:02 yep, LOL, there's a voice in the back of my head that says that every time i break out the lambda 👀

[@MaxBerson]

Anybody know why Python uses these scoping rules? The original code would work fine in LISP, because the value of "task_id" would be captured at the time of invocation. Also, 7:06 - Alonzo Church would like to have a word....

[@hawkanonymous2610]

Why can't you just use an defaulted argument in the lambda? Like lambda fut, task_id=task_id: print_task(fut, task_id)?

[@snoozbuster]

He didn’t mention it, but you can. IIRC, it works because default arguments are evaluated when the function expression is evaluated to create the function object and stored alongside it, so the loop variable gets looked up and copied for each iteration instead of closing over the same loop variable over and over again.

[@NFSHeld]

This is a Python problem, right? Because I'm like 99.9% sure that in C#.NET you can write "task_id", and the compiler determines if the value of that variable could be different by the time the lambda is called, and if so it creates a "captured context" into which the variables' current values are copied, and then the reference to the original task_id is replaced with a reference to the invisible captured context.

[@vlad_cool04]

This is why I spent more time debugging my project. This is very unexpected behavior

[@DeathSugar]

I wonder if there's any actual lambda usage in the python world? they looks useless in general and some scoped named function looks like way better choice then lambda.

[@JinTsen]

Is this a python specific thing or does such also happen in other languages?

[@snoozbuster]

Used to happen in JS back in the day, but it’s been mostly eradicated there. Any language that allows a person to write function definitions inside of loops or conditionals but doesn’t have block scoping will probably have this sort of issue.

[@0LoneTech]

It happens in other languages (e.g. someone else in these comments encountered it in Lisp). The issue is sharing mutable state, as Rustaceans would put it; the closure created by the lambda closed over (shared) the variable (not the value it pointed to), and the variable was then changed (mutated) by the loop. This is the same core issue as languages that lack namespaces (like classic BASIC) and why we generally avoid global variables. Python's dynamic nature does make it a little more expensive though, as the closure covers the entire enclosing namespace, not just the variables it ends up using.

[@RoamingAdhocrat]

first time I've encountered the abbreviation IIFE. I love C++

[@RoamingAdhocrat]

(from a safe distance)

[@szaszm_]

@@RoamingAdhocrat lol

[@archardor3392]

When I read about lambdas in a for loot, I thought to myself: "Yeah, obviously, cus you redeclare the function every time, right? Why do that." In actuality, this turned out to be so much more worse than I expected.

[@aron2922]

Would this also hold for .apply?

[@jadonbelezos2583]

this has alot to do with how a variable/data is passed around. this is where any language where you have to be very explicit about it how something gets passed works really well and any other language can fall apart quickly. even if most the rules are simple, they need to be like stupid simple like less than three sentences or this problem would occur with any language

[@trag1czny]

Great vid!

[@gonderage]

That's strangely unintuitive. If the lambda is set off after each iteration in the loop, and the task_id changes each iteration but before the lambda, then how come the lambda uses the previous value if it should have already been updated?

[@davea136]

Like in a clojure?

[@silp-en-cailloux]

What is a gooey application?

[@tomlombardo6051]

GUI - Graphical User Interface

[@akirakosaintjust]

Funny that Python forces you to pass self to class methods but then can't use self to solve a problem JavaScript could solve by omitting a call to the this object - paradoxically one of the most dreaded aspects of the language.

[@wenbozhao4325]

The take away is loop variable shouldn't be used in a closure, and I need to remember it's called cell variable 😂

[@poijmc606]

Does nonlocal solve this issue?

[@mCoding]

Good guess, but no. In this case, the variable is already determined to be nonlocal at compile time. See my video on closures for more explanation of the nonlocal keyword. In short, you normally only need nonlocal if you want to write to a nonlocal value, not if you want to read one.

[@adityakiran2956]

What is this theme?

[@faremir]

It's not a problem of lambda, it's not a problem of loop. It's problem of whoever wrote that code, that doesn't understand how lambda works. Correct usage of lambda in cases like that is to specify the variable in own lambda argument, that's why those arguments are even definable in the first place.

[@mCoding]

I do not recommend using lambda default arguments as a fix for the issue in the video because the behavior of default function arguments in Python is yet another quirk of Python that many beginner and intermediate programmers often subtley misunderstand. From my experience, I continue to recommend functools partial as the simplest and most straightforward solution to this mistake that many programmers make. I also find that the frequency with which this programming error comes up indicates this is not primarily a "problem of whoever wrote that code" but rather a failure of the language to do what most programmers expect or to otherwise educate users effectively on the design choices that led to this choice of behavior in how lambdas work.

[@faremir]

@@mCoding I do partially agree. Not that partials is correct way, but that it's definitely from huge part language fault. I'm just maybe too oldschool and read documentation. For me expecting something behaving same way just because languages x and y do that without actual understanding is just "mediocre dev" sign.

[@jullien191]

Me gusta comer el hueso. Gracias por los subtítulos en español. 💕

[@mCoding]

(Traducción de Google.) ¡Gracias por tu aliento! No estaba seguro de tener espectadores de habla hispana, así que me alegra que hayas comentado.

[@nimaforoughi3008]

(Lambda x=x: lambda: print(x))() Is an easier solution

[@nnirr1]

This is really unintuitive to me. I would expect the lambda/function object to store its own references for variables defined outside of its scope

[@simon3812]

Looping is a code smell

[@Yupppi]

I tried to look up what is lambda and got really confused about what it's good for, seemed like extra typing in every example. Like return lambda a : a + b + c seems like there's just an extra word there. Or it's used to shrink two lines into one?

[@reimusklinsman5876]

When I hear people talk about code smell, more times then not, I'll devalue their opinion. 95% of the time when I hear someone call something a code smell it's usually because they either don't understand the complexity of the code or more likely, they don't understand time constraints given by management. If management wants to hurry development along, sometimes corners need to be cut. It's easy to say, we'll refactor this later but management never allocates time for cleaning up code. This is one of the few times where it really makes sense.

[@ABaumstumpf]

or rather - Python with lambdas and Asynchronity is a codesmell. Got nothing to do with Lambdas in loops in general, not even lambdas in python loops.

[@zlorf_youtube]

Establish code cheese!

[@stevenhe3462]

Since when do people use this single-threaded language for UI?

[@AlexAegisOfficial]

what, python doesn't capture those variables in a clojure?

[@mCoding]

It actually captures a reference to the local variable in a closure, so if that local variable is modified then the value seen in the closure (cell) is also modified. The terminology Python uses for this is a "cell variable" if you'd like to investigate further on your own.

[@mr.bulldops7692]

Threads, man. They aren't too complicated, but they can force some very interesting code decisions.

[@tanavamsikrishna]

My name is not cheese

[@dschaedler]

Yeah we have a team policy to never use lambdas.. if you don't have a really good reason for it. I think so far we have no lambda in our code..

[@rdococ]

This isn't a "code smell". (1) This causes a bug, not design/architectural issues. (2) This is entirely the language's fault for not creating a new loop variable each iteration as it should.

[@shauas4224]

Is this only me or it's just insanely stupid thing to even consider while designing a language?like what the actual hell is this? This is not thread safe, adds overhead, etc

[@gregstarr2]

Execute-er

[@BraxtonMeyer]

ohhh you're finally awake.

[@JustSomeAussie1]

Why the hell do you pronounce "executor" like that

[@pw5687]

What about for i in range(10): function(lambda i=i: callback(i)) This is what I use, but I wonder if it is better/worse or situations where it should be used vs functools.partial?

[@illegalsmirf]

Pythonistas are like the fashion police, always telling other people what to do (because it's the latest most stylish thing) but never contributing anything of substantive value.

[@NoNameAtAll2]

???

[@mCoding]

Hi and welcome to the channel! Here we don't play language wars. Many languages are popular for many different reasons and we keep things respectful in the comments even if the language in the video isn't our personal favorite. If you have questions, doubts, or disagreements about the code, you are always welcome to voice your concerns. Blanket generalizations about millions of people are not welcome.

[@gregorymorse8423]

This has to do with scope context not lambdas. You could add a def func() in a loop with the same problem. This video declaring lambdas as the problem is WRONG. Update: nevermond he addressed it at the end of the video after enraging us until then :)

[@mCoding]

Keepin' me on my toes!

[@gregorymorse8423]

@mCoding indeed, you always give the fine details at the end. You know the audience best though. A few of us theoreticians are initially a bit irked though

[@stysner4580]

People writing Rust 😮