Tap to unmute

Python Generators

  Рет қаралды 144,705

mCoding

mCoding

Күн бұрын

Пікірлер: 224
@biffenb7534
@biffenb7534 2 жыл бұрын
I love the relatively monotone presentation, which helps clarity, with only a few incredibly well placed jokes sprinkled here and there. Keep it up!
@Holphana
@Holphana 2 жыл бұрын
Monotone? I hear many inflections. I would be offended at the implication towards my accent if I were in the KZbinrs shoes. 😢
@biffenb7534
@biffenb7534 2 жыл бұрын
@@Holphana You’re right, I just didn’t choose my words carefully enough. What I mean is that the presentation is that of a factual text, instead of creating unnecessary clutter through adding emotion or emphasis like so many other KZbinrs do.
@Ptolemarch
@Ptolemarch 2 жыл бұрын
Indeed, these are vastly the best videos on coding I've found.
@RoamingAdhocrat
@RoamingAdhocrat 2 жыл бұрын
What's really missing is a title card with a photo of M with chin in hands looking puzzle, or head-in-hands despairing at some topic that was exhorted in many keynote talks - TDD, say
@NewLondonMarshall
@NewLondonMarshall 6 ай бұрын
What a wholesome reply :) ​@@biffenb7534
@l4luchi123
@l4luchi123 2 жыл бұрын
No matter how experienced you are, in your videos there is always a new small detail you didn't know about. Love it!
@georgplaz
@georgplaz 2 жыл бұрын
yes! I thought "oh. I know everything about yielding.." and then.. bi-directional?! 🤯
@SwissTHX11384EB
@SwissTHX11384EB 2 жыл бұрын
Mate, I have years of Python experience, yet everytime you release a video I am floored by some of the stuff you showcase and I feel like I'm learning about the language for the first time. Stellar job, as always, and your presentation style is excellent for the job.
@luiz8755
@luiz8755 Жыл бұрын
same it's a little bit overwhelming lol
@adsfwef1331
@adsfwef1331 2 жыл бұрын
Dude, you are a genius. I wonder why your vids aren't getting more views than those other millions of python hacks out there. Keep up the good work.
@GlutesEnjoyer
@GlutesEnjoyer 2 жыл бұрын
“Even if a file is gigabytes large you’ll only need enough memory to handle a single line” great point!
@dhravya
@dhravya 2 жыл бұрын
Always struggled with generators. Thanks for the amazing explanation!
@LettuceAttack176
@LettuceAttack176 2 жыл бұрын
I swear every time I start watching a video on this channel I think “ah here is another thing I already know in python I probably won’t learn anything new this topic is pretty straight forward” and I end the video feeling like I never knew anything about it to begin with 😂😂 keep up the great work
@Fikerus2
@Fikerus2 2 жыл бұрын
I write things like "sum(x for x in range(10))" very often and I didn't know that was generator comprehension! I was thinking it was the same as writing "sum([x for x in range(10)])" (with brackets). Thanks a lot for your content!
@yazanalhariri3700
@yazanalhariri3700 2 жыл бұрын
Me too!
@enkryp
@enkryp 2 жыл бұрын
Why using a comprehension at all? Can't we just do sum(range(10)) ?
@MrRyanroberson1
@MrRyanroberson1 2 жыл бұрын
@@enkryp i was just about to mention! yeah range is directly iterable
@lawrencedoliveiro9104
@lawrencedoliveiro9104 2 жыл бұрын
That’s the difference between an “iterator” and an “iterable”. An “iterable” has an ‗‗iter‗‗ method, which returns an iterator when it is called. The iterator is what returns the elements of the sequence, until it is exhausted and can’t be called again. Because range() is an iterable, you can reuse a single range object to return any number of iterators over that range.
@Geza_Molnar_
@Geza_Molnar_ 2 жыл бұрын
@@lawrencedoliveiro9104 Yeah, your explanation helps me a lot.
@ssholum
@ssholum 2 жыл бұрын
Async (vs threading vs "other kinda-multitasking methods") video would be greatly appreciated. It's not the most useful stuff for your average script or simple program, so I rarely see good explanations of it for any language.
@Gamecrazy721
@Gamecrazy721 2 жыл бұрын
Very insightful video! One other place I've seen generators used frequently is for API pagination. If, for instance, if you want to get all 100 records from an API, and the API limits your query to 10 records per call, you may not want to wait for all 10 calls. Instead you create a generator that calls the API only when you've completed working with the first page and are ready to move onto the second, etc.
@lawrencedoliveiro9104
@lawrencedoliveiro9104 2 жыл бұрын
Database queries is a good example. A very common sequence is * Create a cursor object * Execute a query on the cursor * Iterate over the results returned from the cursor * Close the cursor. You can wrap all these steps up in a single generator function, which takes the query and lets you iterate over the results in a single step.
@mabml
@mabml 2 жыл бұрын
Your async explanation would be much appreciated! :)
@ahmadhesham1389
@ahmadhesham1389 2 жыл бұрын
This is why I like your channel: Even when I think I know a topic well, I still expect to find something new in your videos, and I always do. Keep it up!
@vincentperrollaz5261
@vincentperrollaz5261 2 жыл бұрын
Outstanding video as always. itertools might have deserved a mention since it is very useful with generators. The jump from iterator to coroutine was a bit steep and I would personally love an async video
@rituchandra6325
@rituchandra6325 2 жыл бұрын
"Where the only limit, is your imagination... and your download speed" - LOL I lost it here
@69k_gold
@69k_gold Жыл бұрын
Thank you! By learning this I was able to create a toy version of the async coroutines, helped me a lot to grasp the concept of event loops
@MessLeadingProgramming
@MessLeadingProgramming 2 жыл бұрын
Great video. It was eye opening for me when I learnt about generator comprehension. I realised that I was doing something very inefficient passing list comprehensions to sum and other functions. Also worker example is amazing. Never used it this way.
@lex_darlog_fun
@lex_darlog_fun 2 жыл бұрын
Just as I was about to suggest a video on async, James announced it himself. 👍🏻 Can't wait to see this convoluted topic explained in mCoding simple comprehensible manner.
@wojasek1223
@wojasek1223 9 ай бұрын
Thank you so much for this video, and all of others. I've started reading Fluent Python a couple months ago, but that was quite hard for me to go through it. And you on your channel describe those hard for me things and I finally get them somehow. Thought I was dumb, but all i needed was to find a great lecturer. Thank you so much!
@technowey
@technowey 2 жыл бұрын
Thank you for another great video. I’ve been using generators for years, and still didn’t know some of this.
@TylerLarson
@TylerLarson 2 жыл бұрын
Yep, definitely interested in an async video. Especially interested in learning the "python native" async features and components so as to detangle it from all the third-party contributed stuff.
@pa1nk1ller14
@pa1nk1ller14 2 жыл бұрын
Your videos are going into great depth mate! Im awaiting for that async vid of yours
@jewpcabra666
@jewpcabra666 2 жыл бұрын
always love the videos - would love an async video! Always learn a little tidbit from these
@Cookie-mv2hg
@Cookie-mv2hg 2 жыл бұрын
Every coding courses of yours it like math classes where you start at 1+1 and a simple zone out and suddenly you're on advanced mathematics.
@mistervallus185
@mistervallus185 2 жыл бұрын
literally just had to learn about generators 2 days ago for a proj. the video timing is immaculate
@dmdeemer
@dmdeemer 2 жыл бұрын
Watched this video because I'm like, "I know what generators are, but I bet I'll learn something new about them." Found out that yield is an expression. Mind blown. Not disappointed. Please make the async video.
@lawrencedoliveiro9104
@lawrencedoliveiro9104 2 жыл бұрын
yield became an expression in Python 2.5.
@murphygreen8484
@murphygreen8484 2 жыл бұрын
Made me realize how much I still need to learn. Great video!
@DebashishGhoshOfficial
@DebashishGhoshOfficial 2 жыл бұрын
"Thank you, next" was a nice touch.
@ashishjain518
@ashishjain518 2 жыл бұрын
I probably only got 20% of this video. But it was enough for me to fiddle around and learn more about generators' daily use cases to improve my performance... Great video, Mcoding 😄
@FlorianLinscheid
@FlorianLinscheid 2 жыл бұрын
Didn't know about generator chaining, that sounded very useful! Thanks
@LastTigerEyes
@LastTigerEyes 2 жыл бұрын
I really appreciate your videos. I always learn something new, even when I don't think I will. TIL parenthesis make generator comprehension. :)
@spaghettiking653
@spaghettiking653 2 жыл бұрын
4:44 what is this _make method you call? I wasn't expecting this code to run, but it still did, even though it doesn't appear you defined the method? Is this something generated by dataclass?
@NostraDavid2
@NostraDavid2 2 жыл бұрын
Note that MyDataPoint class is the type NamedTuple. I googled for "_make NamedTuple Python" and got some info on the function. Turns out that _make is part of the NamedTuple class.
@NostraDavid2
@NostraDavid2 2 жыл бұрын
Also note that's its a NamedTuple and not a data class. This just means that you have different functions that you can use. So no comparison, like you do with data classes. You CAN do it, but you'll have to implement it yourself.
@spaghettiking653
@spaghettiking653 2 жыл бұрын
@@NostraDavid2 I see, thanks for pointing it out. So, I'm not sure why I thought it was a dataclass, but I guess I just saw the type annotations and didn't read the top part. That makes a lott more sense, big thanks :)
@CrYpt001
@CrYpt001 2 жыл бұрын
Finally something about generators. Never quite got them
@krimsonsun10
@krimsonsun10 2 жыл бұрын
ALL hail @mcoding High Priest of the Python temple!! Your explanations are succinct, and the usage examples help me so much to grasp the concepts. Thank you.
@yxh
@yxh Жыл бұрын
I would def like a video going deeper into "yield from" generator uses, with examples
@blacky7801
@blacky7801 2 жыл бұрын
You can pass things into a generator? Wow, incredible video!
@modest_supreme
@modest_supreme 2 жыл бұрын
Thank you for this! Excellent deep dives. Your videos are a great source for understanding the why/how behind functionality.
@Jakub1989YTb
@Jakub1989YTb 10 ай бұрын
9:40 - is there a nicer way to chain the generators like this? To not repeat the "nums" etc?
@trag1czny
@trag1czny 2 жыл бұрын
craving for the async vid 🤤
@redon638
@redon638 2 жыл бұрын
Thanks! I always learn something new and really impressive watching your videos. Wish you twice as much subs
@vidlb
@vidlb 2 жыл бұрын
Thank you ! And yes for the async video
@yxh
@yxh Жыл бұрын
So at 9:31, the `nums` on line 15 is really a 5th-level nested generator? And whenever sum() wants the next value in the top-level generator, the inner nested generators wind all the way down to the lowest level to read the file and parse it one row at a time?
@mCoding
@mCoding Жыл бұрын
Yep, that's right!
@aditya95sriram
@aditya95sriram 2 жыл бұрын
That generator pipeline was eye-opening! I've worked with generators plenty, but still this pipeline idea escaped me
@ren200758
@ren200758 2 жыл бұрын
didn't know about the generator pipeline and was blown away by it. big hype for the async video!
@dixonstroi
@dixonstroi Жыл бұрын
I think this will really help me with lazy execution via generators. This would have really helped if . Also, 'yield from' was hard for me to get from just the docs, so thanks for summing that up tidily.
@MoritzWallis
@MoritzWallis 2 жыл бұрын
Super informative video, thank you, very interesting
@PikalaxALT
@PikalaxALT 2 жыл бұрын
4:25 - Pycharm thinks your text file is a requirement.txt.
@andrewstribling4385
@andrewstribling4385 11 ай бұрын
Nice work! I cannot wait to start using these generator techniques!
@thyllmuller4524
@thyllmuller4524 2 жыл бұрын
9:42 I tried that exact code and I got the "sum is inf" - any idea what might be happening? I triple checked the code, it is identical aside from the function name being different.
@LuckySeer2
@LuckySeer2 2 жыл бұрын
when i started python, i was so proud of my prime number generator function
@PetrSUsername
@PetrSUsername 2 жыл бұрын
I watched quite few videos about async but I would still like your take as well. It is one of the more complicated/complex? parts of the language to grasp.
@AByteofCode
@AByteofCode 2 жыл бұрын
Very nice CTA at the end, hadn't noticed it before, clever :) Otherwise great video and can't wait to know how async uses generators, I've been curious about that ever since you mentioned it a while ago.
@adamstrejcovsky8257
@adamstrejcovsky8257 2 жыл бұрын
when I feel on top of the world with my python knowledge, I go here to restore humility :D
@BRICKSINSILK
@BRICKSINSILK 2 ай бұрын
I have imagination...I just lack drive, passion, purpose, stamina, motivation, IQ, and I essentially am unable understand social ques...
@martinc.7424
@martinc.7424 2 жыл бұрын
I was just told that list comprehension where faster than for loops... now it makes sense.
@MichalPlichta
@MichalPlichta 2 жыл бұрын
Wow I thought I know almost everything about generators, but i was wrong! Well done!
@mCoding
@mCoding 2 жыл бұрын
There's always more to learn!
@Plann
@Plann 2 жыл бұрын
Same.
@BR-lx7py
@BR-lx7py 2 жыл бұрын
This video is definitely eye-opening. I thought that generators were just regular functions that you could re-enter, but I never comprehended how python calls them totally differently. Adding something like "if False: yield" to a regular function totally changes how it is called, despite those 2 lines of code being a noop.
@knut-olaihelgesen3608
@knut-olaihelgesen3608 2 жыл бұрын
Thank you so much. This was a very confusing topic.
@klaasvaak2575
@klaasvaak2575 2 жыл бұрын
not just imagination or/and downloadspeed but also your pride deciding what you appropriate from the interwebs or what you painstakingly program yourself in most likely a less efficient methode.
@BR-lx7py
@BR-lx7py 2 жыл бұрын
@9:08 It's crazy that you can even have the same name for the generator comprehensions, so all of them called "nums".
@StephenBuergler
@StephenBuergler Жыл бұрын
Do you think it would be a good idea for the else clause in the for/in/else loop to be the place where the generator's returned value is made available?
@lawrencedoliveiro9104
@lawrencedoliveiro9104 2 жыл бұрын
14:44 Yield-from does nothing that cannot be done by the outer generator doing its own send calls on the inner generator.
@mCoding
@mCoding 2 жыл бұрын
True, but take a look at what yield from is defined as in the spec. Due to all the edge cases, it would be an absolute nightmare to implement it yourself every time you wanted to use it. peps.python.org/pep-0380/#formal-semantics
@darske1
@darske1 2 жыл бұрын
Plot twist: The 1 Gb file you're trying to read, has only 1 line xD Awesome video. The send method confuses me a lot every time I see it, I don't know why lol
@macedmondson782
@macedmondson782 2 жыл бұрын
I'm really liking your videos! I would love to see an async video from you!
@anibaldk
@anibaldk 2 жыл бұрын
You cannot see this (hopefully) but I had to give a standing ovation to this one.
@mCoding
@mCoding 2 жыл бұрын
I saw it!
@anibaldk
@anibaldk 2 жыл бұрын
@@mCoding 😱😱😱😱😱😱😱
@Blubberland
@Blubberland 2 жыл бұрын
Thumbs up. 4:27 This might be a rather be a problem of spell-checking settings than on syntax.
@AIRLZ6
@AIRLZ6 9 ай бұрын
damn i need to watch this video over and over again :D. Love it!
@ErikS-
@ErikS- 2 жыл бұрын
4:26 - Maybe the parser may think this may be a line-number?
@vt2788
@vt2788 2 жыл бұрын
Really advanced high value stuff! Thanks!
@arthurfelipe544
@arthurfelipe544 2 жыл бұрын
what is the use of the line x: float? i tried to search the use of the : in that line but didn't find anything
@greasedweasel8087
@greasedweasel8087 2 жыл бұрын
Can you do a video on making __iter__ a generator vs defining a __next__? I’ve seen both in the wild, you seem preferential to the __iter__ generator, and I’ve googled to no avail. I’m mostly interested in which one is more pythonic, but I’d also like to know what you think the use cases for each are. Great videos!
@ciCCapROSTi
@ciCCapROSTi 2 жыл бұрын
Good stuff, I didn't know most of the advanced stuff here.
@endoflevelboss
@endoflevelboss 2 жыл бұрын
Looking forward to the async video 👍🥳
@mattlau04
@mattlau04 2 жыл бұрын
Great video as always! And would love to see a video on async
@taylorbreutzmann6034
@taylorbreutzmann6034 2 жыл бұрын
It’s crazy, being able to tell how intelligent another human is just behind a screen/video.
@lohphat
@lohphat 2 жыл бұрын
What's confusing here is that in the first example you have to use next() to get the next yield value but in the loop the next() is inferred. To me, that just destroys readability as the syntax has changed for no good reason.
@reef2005
@reef2005 9 ай бұрын
It would help if you say something about the arg NamedTuple in the MyDataPoint class and also the '_make' method that does not exist.
@chinmayk8004
@chinmayk8004 Жыл бұрын
3:12 and to "reiterate"... unforced?
@gormster
@gormster 2 жыл бұрын
No, I don’t want a video on asyncio. I want a whole goddamn *series.*
@mCoding
@mCoding 2 жыл бұрын
Take note of how I'm methodically making videos on all the prerequisites for a deep async discussion 🙃
@khalilhadji6489
@khalilhadji6489 2 жыл бұрын
@@mCoding can't wait, i am facing some issue in one of my projects and understanding how asyncio works inside out would be very helpfull
@morrispearl9981
@morrispearl9981 2 жыл бұрын
I am a fan of generators. I use "yield from" when the function that generates the next thing happens to be recursive. For example, I have a generator that takes a list of N things, and yields each permutation of those N things, and the underlying algorithm is recursive.
@mayank8387
@mayank8387 2 жыл бұрын
Dude you're amazing! Thanks for sharing.
@mytalala
@mytalala 2 жыл бұрын
looking forward for async video, great job
@stacymaimoon4189
@stacymaimoon4189 2 жыл бұрын
I was impressed. Just one observation: if you try to debug an example (I did it with example_composable()) and you want to see what happens with the data in the generator after each step, transforming the generator to a list (I am using PyCharm), you will be surprised to discover that the list (and the generator itself) is empty after the very first conversion (and all subsequent generator steps as well). The example runs in execution mode, but delivers wrong results if you debug. I have not found a suitable means to overcome this limitation, so far. Do you have any presentable idea?
@miQize
@miQize 2 жыл бұрын
What's the difference between yield from and just a yield followed by a send to the wrapped generator?
@mCoding
@mCoding 2 жыл бұрын
See peps.python.org/pep-0380/#formal-semantics for the formal semantics. As you can see, there are a huge number of edge cases and exceptions that it takes into account.
@AngryArmadillo
@AngryArmadillo 2 жыл бұрын
Looking forward to the asyncio video!
@HonsHon
@HonsHon Жыл бұрын
Helped me refactor some code and provide some memory savings! Thank you!
@ObsessiveClarity
@ObsessiveClarity Жыл бұрын
I find myself routinely wanting to iterate backwards through a list with access to the correct array indices: for i,x in reversed(enumerate(myList)): # enumerate(myList) is not reversible so this sadly doesn't work. ... for i,x in reversed(list(enumerate(myList))): #works but is moving memory around for no good reason. ... for i,x in zip(range(len(myList)-1, -1, -1), reversed(myList)): # works but is less readable / higher cognitive load ... for i,x in zip(reversed(range(len(myList))), reversed(myList)): # OK but makes me very slightly sad ... Anyone have any good tips for this?
@StrangeArrow
@StrangeArrow 2 жыл бұрын
Super Content. Thanks for sharing with us 🥰🥰
@con-f-use
@con-f-use 2 жыл бұрын
> "Do you think, they would really introduce a whole new set of keywords just to have a shortened for-loop?" Yes. The answer is "Yes, I can see the Python devs do that."
@RoamingAdhocrat
@RoamingAdhocrat 2 жыл бұрын
What typeface is your editor using? I covet those flat-top 3s
@mCoding
@mCoding 2 жыл бұрын
That's the default font in PyCharm, JetBrains Mono.
@fyellin
@fyellin 2 жыл бұрын
Just out of curiosity. Python allows min, max, sum, etc on both sequences and iterators. Any idea why len() can’t be called on a generator. I’ve been trying to decide what was going through the heads of the Python developers. You really shouldn’t have to write sum(1 for _ in generator).
@ccgarciab
@ccgarciab 2 жыл бұрын
Probably because generators aren't guaranteed to end and having a common function as len() being potentially non terminating on a common type of object might have been considered bad design
@fyellin
@fyellin 2 жыл бұрын
@@ccgarciab But is len() really that different than min() or max()?
@lawrencedoliveiro9104
@lawrencedoliveiro9104 2 жыл бұрын
len() requires you to evaluate the entire sequence. Addendum: you should be able to define a ‗‗len‗‗ method for your iterator object, if you want. Then len() should work on it.
@fyellin
@fyellin 2 жыл бұрын
@@lawrencedoliveiro9104 Again, how is this different than min(j, max(), sum()? Unless you’ve got infinities, all require knowing the entire sequence.
@lawrencedoliveiro9104
@lawrencedoliveiro9104 2 жыл бұрын
@@fyellin Probably because of the ‗‗len‗‗ issue.
@tytywuu
@tytywuu 2 жыл бұрын
I can't wait for the async video!
@bigmacbeta
@bigmacbeta 4 ай бұрын
Thank you. Amazing explanation.
@zacharyreilly7307
@zacharyreilly7307 2 жыл бұрын
Please make an async video!! I would love it if you included best practices for incorporating async into sync func/programs I find most times i need something async i want to do sync stuff too or dont know how to convert it all to async
@dmitrylitovchenko2365
@dmitrylitovchenko2365 2 жыл бұрын
In collatz generator checking for n == 1 should be at the top otherwise it behaves funny with input of 1: [4, 2, 1] :)
@mCoding
@mCoding 2 жыл бұрын
Good catch! Yes you are absolutely right.
@alexsere3061
@alexsere3061 11 ай бұрын
"Did you really think they would add a new keyword just to abbreviate a for loop?" This is python so yes, yes I did
@mjdevlog
@mjdevlog Жыл бұрын
So useful, even yhough i don't understand the last half of the video. Maybe one day i will😅
@briban65
@briban65 9 ай бұрын
what is NamedTuple in the file processing one??
@mCoding
@mCoding 9 ай бұрын
I talk about it in this video! kzbin.info/www/bejne/rHSvlqeandiqg8k
@jullien191
@jullien191 2 жыл бұрын
Muy buen video. Gracias amigo
@oshavlfarms7239
@oshavlfarms7239 2 жыл бұрын
It's like a clockwork function. Every time it's called the cog moves a tick.
@didotb01
@didotb01 2 жыл бұрын
RegEx expects a digit or a whitespace after the dot .. might be the reason for the red squiggle
7 Outside The Box Puzzles
12:16
MindYourDecisions
Рет қаралды 248 М.
Python Generators Explained
28:37
Tech With Tim
Рет қаралды 163 М.
Маусымашар-2023 / Гала-концерт / АТУ қоштасу
1:27:35
Jaidarman OFFICIAL / JCI
Рет қаралды 390 М.
Comprehending Proc Macros
47:41
Logan Smith
Рет қаралды 23 М.
Please Master These 10 Python Functions…
22:17
Tech With Tim
Рет қаралды 255 М.
21 MORE nooby Python habits
9:55
mCoding
Рет қаралды 122 М.
25 nooby Python habits you need to ditch
9:12
mCoding
Рет қаралды 1,8 МЛН
Modern Python logging
21:32
mCoding
Рет қаралды 214 М.
Andrew Kelley   Practical Data Oriented Design (DoD)
46:40
ChimiChanga
Рет қаралды 156 М.
Unlocking your CPU cores in Python (multiprocessing)
12:16
mCoding
Рет қаралды 313 М.
Async for loops in Python
16:36
mCoding
Рет қаралды 68 М.
Python itertools - The key to mastering iteration
20:03
mCoding
Рет қаралды 36 М.
Python Decorators in 15 Minutes
15:14
Kite
Рет қаралды 458 М.