Loop like a native: while, for, iterators, generators

Ned Batchelder
Python provides powerful primitives for iterating over your data in ways that let you express yourself clearly and directly. But even programmers familiar with the tools don't use them as fully as they could. This talk will cover Pyt

[@fatihakilma2098]

"So my name is Ned Batchelder and I'm not gonna tell you who I am. You can google me if interested. Let's get into the meat of the talk." What a great introduction... People should learn to google first and then ask the question if needed. Time is gold.

[@rameshsampath6998]

Best talk of PyCon for me. Clear, concise and reveals the power of Python. Don't just write code, write pythonic code. Thanks Ned.

[@joseville]

Just taking notes: 3:47 those `dict` methods were renamed to `.values()` and `.items()` 10:43 I already knew what zip does but "zip takes a pair of streams and gives you a stream of pairs" is such an elegant succinct explanation. 20:32 Now you can just use `for x, y in itertools.product(range(height), range(width)):` 21:32 "A book is an iterable; a bookmark is the iterator -- you can have two bookmarks in the same book" - great analogy! 22:30 files are iterators too

[@PippyPappyPatterson]

good notes

[@tolpacourt]

I'm so glad youtube allows me to slow down the playback. He covers so much ground so quickly.

[@SwastikSwarupDas]

@The Magnificent Whalens i guess this talk wasnt meant for beginners, i found it rather helpful, after having a week of python and 2 years of C

[@Post1690]

I have watched this so many times over the last few months - if you know someone who's teaching him/herself Python, I'd highly recommend you share this!

[@pluckthelivingflowers]

Easily the best explanation of these topics I’ve seen.

[@Guinhulol]

That's my kind of python talk, he just go right to the chase.

[@MuneerKhan1]

Probably one of the very few videos that don't let you blink! 🤙🏻

[@yztyzt1]

I love how clean and aesthetically pleasing compact python code is! OMG> I am in love with the beauty of this language!

[@edterrell5664]

Learned more about loops and itersables in the first 5 minutes than in the past two years studying python!!!

[@pablom695]

this guy is a fucking genious. you can see it espcially at the end when answering questions

[@hangugeohaksaeng]

Thanks for this great talk. Just finished watching for the second time. Much to get out of this. Thanks again!

[@szaydel]

Great talk, covers what we do all the time, and mostly badly, and teaches us how to do it better!

[@user-tb4ir5ny8n]

Very good and densely packed info for beginners. Thank s for a great job.

[@MrTigerstyle80]

This guy knows his business.

[@drygordspellweaver8761]

PURE GOLD especially making your class objects iterable

[@H0r4c3]

Clear and concise explanations! Thank you!

[@curiousbit9228]

Wow, Mindblown! Have to watch it like 20times to actually understand it.

[@zeroxcub]

"your code has interesting streams of values that you are not producing in a nice iterable form yet" Thanks, really nice talk.

[@janhutar]

Thanks. Very nice talk - now I'm confused about generators/iterators/iterables for sure, but I guess I can actually use them :-)

[@edt11x]

Great video. It is jammed with useful information for me.

[@paddym27]

really nice presentation - cheers!

[@meunomejaestavaemuso]

20:50 with generators you can use the 'send' method, so you can 'ask' for a value using 'generator.send(desired_value)' and having 'while True: x = yield; yield x' inside your generator, useful to make co-routines. Another useful thing about generators in Python 3.3 or higher is the 'yield from' statement which can link several generators and yield values from them.

[@joseville]

Wow, I didn't know about this! Now, I need to go and learn about it because it looks very usefule.

[@drygordspellweaver8761]

More on this please

[@nickmcneely5601]

Question: For the evens function, why is a generator better than a list comprehension? Answer: For small inputs, it's not that big of a deal. For big lists, generators use muuuuuuuuch less memory.

[@Lolwutdesu9000]

Nick McNeely thanks, the guy in the video could have mentioned this.

[@ZapOKill]

so lets use generator comprehensions

[@yoganandchowdary6371]

one of the bestttt tutorials....it helped me a lot.....thankyou!!!!!

[@Neceros]

13:40 so basically, try and keep all of the choosing code in the head of the loop via a function and not in the body. Smart and clean!

[@yztyzt1]

it looks like you can iterate over a list of x values, send that x value to a function. Then, put the result of that function into a list of results. It look like very clean code!

[@smlasdhaosf7764]

Always thought this talk should belong to Raymond Hettinger, well, Ned did a good job

[@shahrushabh87]

Nice video Ned.. Thanks :)

[@aliabedi6163]

It was very beautiful and informative (2021)!

[@pradeepnim]

Good one!! clean and fast!

[@yztyzt1]

you don't need range of len??? I can't wait to see what you say, so that i can learn from you and get rid of the integer. (I am guilty of using range of len) and I do think it is very ugly. I don't like it. I paused at 8:00 to type this confession. Now I am going to unpause in order to see how I can get rid of all range(len) cases! I am excited!

[@maguas01]

Learned a lot today

[@TommyCarstensen]

Awesome talk! Thanks for uploading! Ned should replace the image of Superman with an image of himself!

[@PhilippeSantanaCosta]

Cool! Nice stuff...

[@amaterasu48]

Wow this guy knows his stuff!

[@dlednirg]

Outstanding presentation! Any chance you could point me to a link of your slides. Trying to get my head around Python and OOP and your talk really hit some points NOT covered in my other sources. Thanks much

[@mangelozzi]

Great presentation in terms of conveying lots of info quickly, cleanly, and easy to understand. I learnt a lot thank you. One confusing thing for myself I found at 21:19 On the slides: iterable: produces an iterable iterator: produces a stream of values But he says: iterable: Any value that can produce a stream of values iterator: Is the object which knows where you are in that stream. The wording is confusing, the "slides iterator" definition sounds like the "spoken iterable."

[@renyuanzhang9706]

What he said is correct, and what on the slides is: iterable: produces an iterator # not iterable iterator: produces values off a stream one at a time # more precise way to put it

[@Murkz21]

@@renyuanzhang9706 If I put a Pass down the def __iter__(self):, then what will happen? Is it still ok to use for the for/while loop?

[@swapnagupta6756]

At 20:55, is spreadsheet.cells() a built-in function in Python? If not , is it just a generator with 2 for loops on rows and columns returning cell values?

[@RaymondJerome]

to break out of nested loops I, as a BASIC programmer for well over 35 years I use GOTO. the only time i ever use goto. not elegant but practical. I love all the 'new' thinking python has taught me.

[@vayunandu]

pure gold

[@ZenKloom]

Python is beautiful.

[@TommyCarstensen]

11:15 "zip is a function that takes iterables and produces an iterable" Ned ignores the existence of Python 2 and itertools.izip :)

[@MatthewTaylorAu]

I get the concept, in my newb days I had a mental block over readability. Is there a speed/resources benefit?

[@mangelozzi]

At 12:32 what is the advantage to using .get in the key function? Is it not adding an extra dict look up, instead of accessing the tuple directly? (k, v) being passed into the .get(), so it's like: seq.get(k, v) k is treated as the key, and v is as the default value if not found. Am I the only one who finds its a bit more obscure?

[@matthewherzog6155]

This is for Python 2.x. For example, when he uses _print repr(line)_ it will throw an error in Python 3.

[@vishnuvarthan8304]

yeah right concepts matter

[@yomajo]

14:20 - very nice.

[@736939]

At 14:00 in function should be append(n) not append(x)

[@RamkrishanYT]

Can anyone explain the second command at 12:36 key=lambda b:b[1]

[@deepakuttam5435]

tall_building.items will return the tuple of (buildingname, height), we are sorting this by the height , so lambda b : b[1] will take tuple as input (buildingname , height) and will return the height i.e b[1] as key value for sorting . if we want to sort by the buildingname it would have lambda b :b[0].

[@Will_Keatinge]

The entire function range_2d in 20:23 could be replaced with itertools.product

[@jazej5516]

my teacher wants us to iterate the first 4 digit sum to a single digit. How would i go about this? His hint was # can you iterate over a string or a list or a tuple? mylist=list("MyVeryOwnString") for counter1 in mylist: print(counter1) # can you iterate over a number? mynumber=12345 for counter2 in mynumber: print(counter2) Next Previous so im assuming you cant itterate a number so somehow i would need another string but that is where i am stuck...

[@ericmacleod8605]

+JazeJedikai convert your integer to a string then slice print str(yournum)[3]

[@jazej5516]

hmm i did it differently. This seems simpler though lol

[@MrSkatefuture911]

I am fairly new to Python and I am wondering if anyone with more in-depth knowledge could tell me to what degree these speeches from 5-10 years ago are still applicable to current version of Python? for replier in repliers: print(f"Thank you {replier}")

[@joseville]

They're very applicable still. Some of the function names have changed, but still very applicable.

[@me4901]

Genius within 2:25! Integers? if you want to print your list, just print your list!

[@bmurph24]

Cool lecture I'm guilty of some of the horrible looking needlessly complex loops he mentioned. It's a shame how much time this probably could have saved me, especially the generators.

[@kalidasa_in]

Expert users should start at 14:00.

[@mctrjalloh6082]

what tool is he using for presentation content?

[@nedbatchelder]

It's not a convenient tool, it's a hand-rolled tool-chain: nedbatchelder.com/blog/201504/how_i_make_presentations.html

[@RobertLugg]

Good basic talk.

[@enyoc3d]

22:18 note to self: use this in your CSV code

[@andrewsanchez5895]

>>> def hi_world(): ... yield "Hi" ... yield "world" Why does next(hi_world()) always produce "Hi"? but you get: >>>for x in hi_world(): ... print(x) Hi world

[@Grelleum]

Because each time you use next(hi_world()) you are creating a new iterable object. The for loop is reusing the same iterable object. Instead try instantiating a new generator object: >>> my_hi = hi_world() >>> next(my_hi) 'Hi' >>> next(my_hi) 'world' >>> next(my_hi) Traceback (most recent call last): File "", line 1, in StopIteration >>>

[@pgoeds7420]

8:30 s/Sears/Willis/

[@marshabner7731]

Found typo for variables at 14:19: "them.append(x)" ==> "them .append(n)"

[@tommyli6916]

6:26 said: for list function will take an iterable and will pull all the values out of it and give you a list of all of them. Don't use this on itertools count because you will run out of time. Why ? ??

[@jimstanley_49]

Because itertools.count() generates an infinite stream. new_list = list(itertools.count) will loop forever (or until you run out of memory) as it tries to generate a list of all integers from 0 to infinity.

[@RonJohn63]

This is a very interesting video. 10:51 zip is possibly the Worst Name Possible for this function, without resorting to politics. The word already has a specific meaning.

[@kipsta1993]

may somebody please explain the joke at 12:00 :D

[@Grelleum]

"Python can loop tall buildings in a single bound" is a play on Superman can "Leap tall buildings in a single bound."

[@owaisgen]

HI I am getting error dictTower=dict(zip(names, heights)); TypeError: zip argument #2 must support iteration... Are you getting some error?

[@Zeturic]

This is an excellent talk, but I generally hesitate to refer it to beginners because of the Python 2-isms. Getting into the minutia of how Python 3 and 2 are different can be interesting, but I feel it's just an unnecessary distraction for beginners who are starting with Python 3.

[@ThunderAppeal]

Not sure why there is so much praise for this talk. He spends about 18 minutes talking about various loops, all well and good. His presentation on generators is weak at best and misrepresented realistically speaking. At 18:05 he shows the pre-generator example, at 18:19 he takes out the comments, tightens up the lines and puts a yield rather than the do_something return. This indicates to me that poor little Ned is having difficulty truly explaining the diffrences between a generator and a regular loop. At least he is able to cover zip with some understanding so good on you Neddy. This should serve as an indication of what is wrong with the software development industry today. A lot of frauds running around with the talent of being able to pull off the appearance of understanding what they are doing. Hello to all the python and nedbat fan boys.

[@ThunderAppeal]

Took out comments to make it seem as if the generator uses less lines of code. All generator does is yield an individual result. Generators eat up more CPU time, versus a regular loop that may eat up more memory. With clusters...I'm sorry I mean clouds all that may be a moot point depending on what someones subscription plan to the cluster is.

[@Lolwutdesu9000]

I'm not sure why he even bothered to make a separate "evens" function at around 16/17 minutes. Can't really see that working faster, or being more efficient memory-wise. Sure, making a generator sounds great, but who would actually use one in that situation?

[@darrinmc]

@@Lolwutdesu9000 It makes the code easier to read

[@psypher514]

iter

[@dokwonsuh]

Half Python, Half Cocaine

[@erichepperlewp]

Feels like a powerful talk ... lost me at #generators.

[@voiceofprime]

Is it my ears or he talks really fast?