Tame the Power of the Iterator Pattern and yield return in C#

Enlist my course Beginning Object-Oriented Programming with C# ► codinghelmet.com/go/beginning...
Download source code from Patreon ► / zoranhorvat
The Iterator design pattern has always had a special place in .NET. It was the level of abstraction imposed by IEnumerable transcended concrete collections, giving rise to the magic of 'yield return' in the early days of C#.
If you are thinking this is merely about traversing a set of data points, think again. .NET took the Iterator concept and introduced deferred execution, lazy evaluation of sequences, and, eventually, LINQ: the heart and soul of iterators in .NET. It diminishes the manual iterations over sequences and collections and makes coding efficient and fluid.
However, with power comes responsibility. Using the IEnumerable interface in .NET has its intricacies. Iterating through the same IEnumerable more than once can be dangerous. There are instances when the data from a first iteration might vanish, especially when coming from a source like a network socket. Other implementations might cause a performance penalty, such as reiterating the result of a database query, essentially causing the query to run twice on the underlying database. Such intricacies emphasize the need for understanding the Iterator Pattern to leverage its potential fully.
In essence, .NET has magnificently integrated and enhanced the Iterator design pattern. By striking a balance between high-level constructs like LINQ and low-level, detailed access via the IEnumerator, it offers developers a broad spectrum of tools.
Thank you so much for watching! Please like, comment & share this video as it helps me a ton!! Don't forget to subscribe to my channel for more amazing videos and make sure to hit the bell icon to never miss any updates.🔥❤️
✅🔔 Become a patron ► / zoranhorvat
✅🔔 Subscribe ► / @zoran-horvat
⭐ Learn more from video courses:
Beginning Object-oriented Programming with C# ► codinghelmet.com/go/beginning...
⭐ Collections and Generics in C# ► codinghelmet.com/go/collectio...
⭐ Making Your C# Code More Object-oriented ► codinghelmet.com/go/making-yo...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⭐ CONNECT WITH ME 📱👨
🌐Become a patron ► / zoranhorvat
🌐Buy me a Coffee ► ko-fi.com/zoranhorvat
🗳 Pluralsight Courses ► codinghelmet.com/go/pluralsight
📸 Udemy Courses ► codinghelmet.com/go/udemy
📸 Join me on Twitter ► / zoranh75
🌐 Read my Articles ► codinghelmet.com/articles
📸 Join me on LinkedIn ► / zoran-horvat
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👨 About Me 👨
Hi, I’m Zoran, I have more than 20 years of experience as a software developer, architect, team lead, and more. I have been programming in C# since its inception in the early 2000s. Since 2017 I have started publishing professional video courses at Pluralsight and Udemy and by this point, there are over 100 hours of the highest-quality videos you can watch on those platforms. On my KZbin channel, you can find shorter video forms focused on clarifying practical issues in coding, design, and architecture of .NET applications.❤️
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⚡️RIGHT NOTICE:
The Copyright Laws of the United States recognize a “fair use” of copyrighted content. Section 107 of the U.S. Copyright Act states: “Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phono records or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright." This video and our youtube channel, in general, may contain certain copyrighted works that were not specifically authorized to be used by the copyright holder(s), but which we believe in good faith are protected by federal law and the Fair use doctrine for one or more of the reasons noted above.
⭐For copyright or any inquiries, please contact us at [e-mail]
#csharp #dotnet #designpatterns

[@zoran-horvat]

Learn more from the video course *Beginning Object-Oriented Programming with C#* ► codinghelmet.com/go/beginning-oop-with-csharp Download the source code from Patreon: www.patreon.com/posts/source-code-for-88935843 Learn from related videos: Master the Power of Composite Pattern in Rich Domain Modeling ► kzbin.info/www/bejne/Z53CmXulgsyHjJI Clean Code Tip: Favor Method Chaining Over Nested Calls ► kzbin.info/www/bejne/sIjRYYJmrd-Kg6M Master the Builder Pattern: The King of Creational Design Patterns in C# ► kzbin.info/www/bejne/aaGcZIRsntuVkJY

[@conbag5736]

I've just found your channel, and I'm a huge fan already. You're a great teacher.

[@davidghydeable]

I have really struggled with the "multiple iterations over IEnumerable" issue without satisfactorily answering it. So often you want to test a sequence before iterating it, but that breaks the rules. Passing an IEnumerable around an application can be dangerous too because you don't know how other functions will use it, so maybe bringing it into memory and passing as an IList is better for these occasions. The trouble is that IEnumerable is just seen as a de facto standard and so many developers default to it without understanding what it actually implies. Direct use of the iterator seems to be a partial solution (it allows you to safely check whether a sequence is empty), so that's nice. Maybe a video that touches upon some of these issues might be good. Anyway, thanks for posting- as always i enjoyed it.

[@zoran-horvat]

What you have mentioned here is a known problem in .NET - sharing a reference to IEnumerable. If you don't know what that IEnumerable is, and wish to iterate it yourself, then you mustn't share that reference with others. There are two general approaches to solving this issue. One is to know what IEnumerable is by materializing it (e g. into a List). I did this in the ParallelDiscounts class, so to be sure it is safe to iterate it multiple times. The other approach is to return IEnumerable without iterating it, effectively transferring ownership to the caller. Any LINQ expression is doing this.

[@BoteeQ]

Great video. If it comes to the topic of "yield" would be good to hear your explanation of why, when, and how to use "await foreach/IAsyncEnumerable" as it is more advanced and I cannot find good resources about this

[@nkazimulojudgement3583]

mee too i get confused as to when to use it

[@matsedv]

Great video, i subscribed 3 minuites into the video, first time checking out this channel. :)

[@tarsala1995]

It's crazy how many times during Pull Requests I see people initializing new List just to put items to it and return the List. I always comment on that, it's a waste of memory on the spot.

[@zoran-horvat]

Learn more from video course *Beginning Object-Oriented Programming with C#* ► codinghelmet.com/go/beginning-oop-with-csharp Download the source code from Patreon: www.patreon.com/zoranhorvat Learn from related videos: Master the Power of Composite Pattern in Rich Domain Modeling ► kzbin.info/www/bejne/Z53CmXulgsyHjJI Use the Decorator Pattern To Reduce Code Duplication in Complex Models ► kzbin.info/www/bejne/bX-TfaN9dq5oipZ3lean Code Tip: Favor Method Chaining Over Nested Calls ► kzbin.info/www/bejne/sIjRYYJmrd-Kg6M

[@ivandrofly]

Thank you sir this again replies to my question in the previous video 😂. Big thank

[@zoran-horvat]

Yes, that's the one you asked about.

[@ar_xiv]

yield confused me for a long time because I didn't know if it was a noun or a verb (Yield as in stop or Yield as in a yield of crops or something). return has this ambiguity as well because it can mean "return out of the function" or "return this value from the function."

[@AhmadElkhouly]

Thanks

[@RichardJohn86]

Thanks as always But I dont quite understand why I would use the GetEnumerator example, in what way is this a better solution than foreach with yield? Is this so you can avoid the previous .Any call to avoid an additional enumeration? Or are there any other reasons to get the enumerator over a foreach? Thanks for the consistent uploads

[@PlerbyMcFlerb]

Compared to foreaching the ienumerable, It allows you to lower your level of abstraction, giving you more hands on control and flexibility but at the cost of more complexity and more risk of a programming mistake if you’re not careful and know what you’re doing.

[@RootAndroidAndEthicalHacker]

A question here : will using count() also cause an iteration ? if so whats the best way for it like if it variable exists, and count is bigger than 0 then go and do the iteration whats the better way there? do same in this vid ?

[@davidghydeable]

Yes, Count(), Any() and any other function that has to "look" inside the collection will iterate it. If you *have* to get the count first before iterating yourself, then the safe thing to do is do a ToList() on the data. However, that is not good because the function that has called you doesn't know that you've brought the collection into memory, so I think the correct thing to do is make your function's parameter an IList. That way, you are telling the caller that you don't want one of those dangerous IEnumerables so give me an in memory list instead.

[@friedcrumpets]

Hi Zoran, found your videos the other week and ended up buying one of your Udemy courses. Absolutely brilliant. I work as a Game Dev and it's been stated within the team that we are to avoid linq as it can create and dispose of a lot of objects in a short time frame, which causes a spike in the Garbage collector and can tank performance. I quite like linq as it's quick and simple, but I'm no longer allowed to use it. Where performance is concerned should I be worried about using linq in other projects?

[@zoran-horvat]

I agree with that - game developmwnt has other priorities. LINQ is better adapted to streaming uses, like network or database, where its lazy evaluation actually saves resources, rather than wasting them.

[@friedcrumpets]

@@zoran-horvat Thank you for the response, that was really helpful

[@henrik3098]

thanks brotherrrrrrrr

[@imhassane]

Hold on, if I use Any() in a condition a then loop on my data, am I fetching the data twice with Entity for instance ? That's horrible if it's the case

[@zoran-horvat]

The answer is both yes and no. Yes in the sense that you will fetch the first element in Any and then again all elements during the iteration. Therefore, the performance of Any will be reasonably good - it is hard to make it any better without prior knowledge of the concrete collection, if there is a collection in the first place. The horrible part is that there can be a high cost on fetching the first element. Imagine a database query - you need to do it twice just to figure if there is any data. And the worst part is yet to come. There could be a race condition which would cause the first and the second execution of the same query to return different results - Any could evaluate to true, and then the iteration fetches nothing because another caller has just deleted the data in between the two executions. Even worse, some data sources cannot even be consumed twice, such as some streams. An attempt to iterate after Any would cause an exception. There are two ways out. Either don't use IEnumerable, but some other type, such as a collection, or use GetEnumerator and MoveNext to ensure both the existence test and the transformation are performed in one pass through the sequence. Remember that IEnumerable is an abstract sequence. Whatever might come to your mind about its content - it does not promise that.

[@Curlack]

To return items from primary of alternate if primary contains nothing. Couldn't we also use two foreach loops where first will set found boolean and afterwards return if found?

[@zoran-horvat]

Some sequences throw when you try to iterate them twice. For example, streams do not support multiple iterations in general, and so a sequence deserialized from a stream would throw.

[@Curlack]

@@zoran-horvat was thinking more in the lines of the following code (see found variable): public IEnumerable GetDiscountAmounts(Money price, DiscountContext context) { var found = false; foreach (var discount in _primary.GetDiscountAmounts(price, context)) { found = true; yield return discount; } if (found) yield break; foreach (var discount in _alternate.GetDiscountAmounts(price, context)) { yield return discount; } }

[@user-we6wp1ky7f]

Hello Zoran. Let me express my impression about the video. There is one point in it about enumeration a sequence twice, which would be more clear with an concrete example. Why can't we use the Any method? Will 'Any' enumerate whole collection, I think not. It would be interesting if you created an example of a collection that cannot be enumerated twice and showed that using a condition with the 'Any' method will distort the result. It is just my opinion cause it is not so easy topic and concrete example can explain double enumeration trap better. Also as I know resharper suggests in this case to transform enumerable into list if collection could be enumerated many times. Your opinion about this also would be very interesting. I hope my thoughts can influence you to make an additional video about iterators) But anyway this video is very useful.

[@zoran-horvat]

I have explained all the aspects of that problem in my course Collections and Generics in C# at Pluralsight. The short version is as I said it in this video - Any will trigger generating the sequence (e.g. by executing the database query) and attempt to fetch the first item, therefore changing the second iteration if iterating is destructive.

[@dmitrykim3096]

Hate those one lone conditions and loops

[@slowjocrow6451]

Is it really bad to just use List mostly ? Makes my life so much easier because i don't really understand when I'm using IEnumerable wrong

[@zoran-horvat]

The only wrong use of IEnumerable that comes to my mind is to iterate it twice. It is really straightforward.

[@slowjocrow6451]

@@zoran-horvat Thanks. My work just bought me a pluralsight subscription, so looking forward to studying your course there 👍

[@josebarria3233]

I didn't know the compiler creates a whole state mschine class for us when using the yield instruction! I wonder what other things can be achieved through yielding

[@protox4]

As a game developer, we avoid Linq because of performance concerns.

[@redcrafterlppa303]

I heard that a few times, but why is linq slow and what is the fast alternative you are using. I cannot imagine that a hand written for loop is much faster than a linq one. Since linq is basically only a set of generic helper functions. The only reason I can think of is the overhead of the lambdas used in linq.

[@zoran-horvat]

Game development has different nonfunctional goals, such as minimizing garbage collection and putting emphasis on in-place operations.

[@protox4]

If Linq was free (same speed as manual loops), I would absolutely use it, because it is way easier to read and write!

[@protox4]

Also, f*ck YT deleting my comments explaining why Linq is slow!

[@winchester2581]

@@protox4 it also depends on what code you're writing. Because you can write simple for-loop which can be slower than LINQ code. Scenario means, but I know that game devs avoid LINQ in overall (and such structures like List too)