Do you even need loops in Python? A Python short by Michael Kennedy

[@jabuci]

Note that there is a for loop inside the list comprehension... So yes, you do need loops in Python.

[@Colstonewall]

Always loved List Comprehensions. Much cleaner and succinct.

[@mikeckennedy]

Indeed! Especially for one-line transforms (dict -> class or something).

[@buzzdev]

You still have the for loop in list comprehension

[@Drakkle]

Does list comprehension work with updating additional variables in a standard for loop? For example, I have a bit of code nested in a while loop that will check if a file exists, then update the version name to the next incremental value if true, and compile the file results into a list. I would be blown away if list comprehension is that robust but I don't know enough about it admittedly. I will update my code immediately if that's possible! Edit: found a post that specifies that while loops are not compatible with list comprehension. Ah well, it would be nice.

Have an idea for this short series. How about an async version of this short? maybe going as far as async generators or async generator expressions (if there is such a thing)

[@mikeckennedy]

Thanks Alvaro. Good idea and I'll definitely do some on async and generators. :)

[@mertyertugrul]

Great one.

[@mikeckennedy]

Many thanks Mert!

[@Minkiu]

I do like list comprehensions, but tend to write them in three lines if i got the conditional statement :D

[@mikeckennedy]

Me too! The ones in this video are just transforms basically.

[@guidyouguy7306]

Still same. can you show how they compile?

[@mikeckennedy]

They are not the same. List comprehensions are about 25% faster. Here's something you can run: gist.github.com/mikeckennedy/2ddb5ad84d6e116e6d14b5c2eef4245a

[@galanu]

I've read this before, but generally for readability, I have kept with for-in loops. What is the performance gain by doing a list comprehension? Is this more focused on massive data sets? Another great video...thank you Michael.

[@mikeckennedy]

Thanks RJL. For me, it's for transforms. I have a list or iterable of one thing and want it in another. I have a list of dictionaries but I want a list of Pydantic objects.

[@elatedbento]

there are generally performance gains with list/set/dict comprehensions. In general, they make code more readable and succinct. However, they should not be used for cases where loops should be used, like looping with side-effects. [print(x) for x in xs] is just an abhorrent practice. They also have a minor advantage: they delegate the iteration logic to python, whereas with loops you have to take care of the iteration or explicitly append to a list. I see them as more idiomatic and easier to read and understand.

[@HimanshuYadav007]

Do you even need loops in Python? - Yes that you are using in list comprehension - Violla Genius LOL

[@Slacquerr]

Is there a reason to not get_raw_data() in your comprehension?

[@mikeckennedy]

Actually, no! That’s a good idea. One reason to keep it in an intermediate variable is for seeing what you’re starting from in a debugger. But in general, I like your suggestion.

[@Slacquerr]

@@mikeckennedy Thanks for the sanity check!

[@spouwn]

Great video Michael. One comment could be a follow-up video that explains using Generators for "Larger-than-your-PCs-Memory" datasets. Using the same data, for clarity.

[@mikeckennedy]

Thank yo7, and that’s a great topic idea. I’ll definitely put it on my list.

[@Drkttv-b2d]

I love vectors, they're my friends.

[@guidyouguy7306]

Vec required same type

[@victorklimov5254]

Thanks Michael! Love your work! By the way, why do you annotate with list[Something], not List[Something] (from typing module)? More simple? Is there any difference?

[@mikeckennedy]

Thank you Victor! Python used to have typing.List for this purpose. But since 3.9 (?), you can now use the native types, list, dict, etc for the same purpose. This means you no longer need the from typing import List statements.

[@victorklimov5254]

@@mikeckennedy Thank you for your answer!

[@AbstractObserver]

I find it misleading to say a list comprehension helps to build better or more reusable code. It does not help with reusability. Its just what it is, a way to describe the generation of a list. The "you dont need loops" approach made some people come up with weird things like using list comprehensions for thing that dont require the generated list, or dictionary or set, and do that as a syntax sugar to for loops.

[@mikeckennedy]

List comprehensions can be used in locations that are impossible with for statements. That makes them more flexible. I don't know I said that they were more reusable but that they could be used in places that loops cannot (not the same thing). I'm not sure I can put code reliably into the comments and keep the formatting, so have a look at the discussion here for some examples: stackoverflow.com/questions/40646458/list-comprehension-in-pandas One example: df2['optimal_fruit'] = [x[0] * x[1] - x[2] for x in np.array(df2)] That is super clumsy to create a temp list, run through the comprehension, the write back into the dataframe. That doesn't make it more readable, it makes it less. It's these types of situations I was thinking about with those statements. Also, when lists are taking as a function argument: products = load_products(data.id for data in api_result) Is cleaner that doing a loop just to call that function, as long as it's simple enough of a projection.

[@DuarteMolha]

I like list comprehension.. But there is a for look written right there in the middle of the comprehension... So saying you are avoiding a for loop is disingenuous

[@mikeckennedy]

It runs different instructions in CPython compared to a loop. So while the word 'for' is involved, it's not the same as a loop. But clearly there is a close relationship.

[@DuarteMolha]

@@mikeckennedy really? Given that the input iterable can be composed or any typeof variable I did not think there was much optimization that could be done to that for loop. Good to learn something new.

[@mikeckennedy]

@@DuarteMolha Hey, yes, a bit surprising. Check out this gist: Perf numbers at the bottom: gist.github.com/mikeckennedy/2ddb5ad84d6e116e6d14b5c2eef4245a

[@DuarteMolha]

@@mikeckennedy see... But that I can understand.. When you have a pure counter as an iterator the input variable is all of the same type (int) If the iterator was a list containing different types of variables would it will perform faster?

[@DuarteMolha]

My guess is that similarly to numpy arrays... The performance optimization comes from the fact that all elements are of the same type. I think if python cannot collapse the iterator in the for loop onto a single type like int or float it will revert back to the same regular pure python for loop implementation