How IEnumerable can kill your performance in C#

The first 100 of you can use code SCHOOL2022 for 20% off courses and bundles at dometrain.com
Become a Patreon and get source code access: / nickchapsas
Hello everybody I'm Nick and in this video I will show you how IEnumerable can harm your application's performance. I will explain why it happens, what you can do about it and how to deal with it in future scenarios.
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasG...
Follow me on Twitter: bit.ly/ChapsasT...
Connect on LinkedIn: bit.ly/ChapsasL...
Keep coding merch: keepcoding.shop
#csharp #dotnet

[@TheBreaded]

I swear this is one of those things resharper has taught me with it's warnings, I rarely see it now because I know better. Great explanation of multiple enumerations.

[@MicrosoftLifecam1]

Agreed - Before using Rider I had no idea lol

[@ivaniliev93]

Me too

[@TonoNamnum]

Visual studio also gives you the same warning.

[@tonyschoborg]

@@TonoNamnum there must be a setting for that. We recently just upgraded to Rider from VS and have caught a few times we missed it. I have never seen the warning until we upgraded.

[@crack8160]

haha same, when I installed resharper I got aware of this situation.

[@rafaelm.2056]

I been programming with C# for about 15 years and there are parts about it that still mystify me. Your example of obtaining a count via an IEnumerable reminded me of how I learned on my own a similar situation with your example. In my case I was loading over 100k records. EF was new to me and I couldn't understand why my app was taking a performance hit until I discovered the difference between IEnumerable and IQueryable. From then on it forced me to take into consideration the overall purpose of the program and how to use IEnumerable properly. You are very well versed in the programming language, more than me after working with C# for so long. On a side note, back when I was learning programming in 1991 I asked a senior developer of our mainframe why people are sloppy with their code. He told me that it will only get worse because as computers get faster it will compensate for bad coding practices and the end result will be lazy programmers. I came from learning to program on a mainframe environment where every byte counted. We ran accounts payable and payroll for 300 employees. All of it was done on a 72 megabyte hard drive.

[@rmcgraw7943]

I started with assembly code. U and I are in the same boat. 😅

[@marcotroster8247]

This enumeration style is called co-routine for those who didn't know. You basically have a function on hold that can give you the next element right when you need it 😄 Actually this is a crazy efficient way to represent e.g. endless streams like indices from 1 to n, e.g. for n=int.MaxValue this is 2^31-1 * 4 byte. Your PC would simply explode if you'd call ToList() on it because it's 8GB of data. But a co-routine like Enumerable.Range() could do that with just 2 int variables and 8 byte. It really makes a huge difference as you can keep this little chunk of 8 byte in faster cache levels of your CPU and crank on it like crazy. A ToList() too less or too much can make your program run 2 hours instead of 1ms 😅😅😅

[@fred.flintstone4099]

I don't think that is called a "co-routine", I think it is called an "iterator". So it is like a class that implements an interface that has a next() method, so the foreach loop calls the next method every time it loops.

[@marcotroster8247]

@@fred.flintstone4099 Historically speaking, when coroutines were invented, all machines were single-core. Real hardware multiprocessing wasn't even a thing until mid 2000s. So, the resulting programs back then were fetching from something really similar to what we call an iterator nowadays. It's all about the illusion of concurrency by decoupling consumer from producer. And honestly, most time in compute is still spent waiting. Waiting for operations to write results back, waiting for registers to be loaded from cache, waiting for jumps because of control flow, waiting for cache synchronization between processor cores, etc. Our code in C# is really just an illusion of what's actually happening. But feel free to tell me where I'm wrong. Maybe I can learn something profound.

[@fred.flintstone4099]

@@marcotroster8247 No, I think you're right. I think an iterator that waits might also be called a "generator". In C# you can use the "yield" keyword for iterators. There is also IAsyncEnumerable.

[@shahzaibhassan2777]

Wow! 👏 This man has some serious knowledge.

[@asteinerd]

Great illustration of how/why this happens. Something I can send to my peers that get confused as to why their code is hitting an API twice when running around with IEnumerable or IQueryable.

[@asdasddas100]

I feel like you can explain this in 1 minute if they're already experienced programmers, but if they're new this would be helpful

[@mariorobben794]

My personal choice is to return an I…Collection, so that the consumer knows that the “inner” code isn’t deferred. Of course, there are situations where an IEnumerable is better, for instance when implementing repositories. But such repositories are mostly consumed from other application specific services.

[@megaFINZ]

It's fine if result is supposed to be mutable because ICollection exposes things like Add and Remove. Otherwise you'd want to return something read-only: IReadOnlyCollection or ImmutableList or similar.

[@Arekadiusz]

Whoa, for the past one year I was getting sometimes warnings "Possible multiple enumerations" and never knew what does it mean :V Thank you!

[@nocgod]

it really is quite clean, there is even a warning (at least in visual studio) CA1851: Possible multiple enumerations of IEnumerable collection it just requires the developer to read the warning and handle it. (or the senior developers elevate this from warning to a compilation error)

[@stefanvestergaard]

You could also adress how yield return's are dangerous in that the source list can be changed between enumerations, e.g. items removed between the .Count() and the output.

[@michaellombardi3638]

You have no idea how much this helped me today! I was looking at a problem where counting an IEnumerable with zero elements in it resulted in a significant delay and I thought I was going crazy! I had no idea that IEnumerable would be lazily evaluated. Thanks for the help! :)

[@Max-mx5yc]

IEnumerable - The fast-food restaurant of programming

[@andytroo]

the IEnumerable approach is the only sensible one in some situations, if there are too many items to fit in memory. I find myself using a 'batchBy(int n)' approach: it turns IEnumerable to IEnumerable so that you can work on a smaller list, but if things are too big, you can take them in byte sized chunks. It does mean something like 'count' (or other things that require global knowledge) can only be accumulated and discovered at the end of the list.

[@LordXaosa]

Materialization is not good option too. What if you file is 200GB size? Or what if there is pseudo infinite enumerable like network data or reading database cursor? So you can't always cast to list because of memory. So yes, watch you code and do what you understand. yield return is not bad if you know what you are doing.

[@henrikfarn5590]

I agree! Understanding your code is the mantra - at one point in time IEnumerable was THE way to do it in my company. For large payloads IEnumerable is great but applying it everywhere is an antipattern

[@battarro]

Then treat it as a stream. On the scenario he gives, if the file is 200GB ReadAllLines will create a 200GB memory array of strings, so ReadAllLines is not the appropriate method to read such a large file, you have to stream it in.

[@billy65bob]

@@battarro it would actually be well to 400GB, the file is likely ascii/utf-8, whereas the in memory representation is UCS-2, which is 16-bit (and similar to utf-16).

[@DanielLiuzzi]

@@battarro ReadAllLines won't do streaming but _ReadLines_ will

[@OwenShartle]

A key phrase, which is maybe more of a LINQ term, that I was also hoping to hear was "deferred execution". Great topic to be privy to, and great video, Nick!

[@MiroslavFrank]

IReadOnlyCollection

[@superior5129]

A bigger problem with methods that return IEnumerable is when they take parameters like a Stream or any IDisposable.

[@krftsman]

I was just giving my developers a lesson on this exact topic last week. I wish I could have just pointed them at this video! Thanks so much!

[@the_wilferine]

Awesome video as always! It’s worth noting however that the implementation of GetCustomers using Select behaves subtly differently to yield return. The call to GetCustomers itself is deferred until enumeration when using yield return whereas it’s called only once when using the Select, when it is assigned to the customers variable. Still absolutely a performance issue as the iteration over the lines still happens twice but the file is only loaded into memory once in the Select example.

[@emmanueladebiyi2109]

Great stuff Nick. Your impact on my programming had been tremendous!

[@SmoothSkySailin]

Great video! I always feel good about myself when I know exactly what the problem is and what your solution is going to be at the start of the video... It doesn't happen often, but when it does, I give myself a gold star :-) Thanks for posting such good content!

[@stephajn]

This is something I knew about and have been working to pass on to others as well. Thanks for making this video. I will share this with them in the future!

[@tonyschoborg]

Funny you should come out with this video. We recently upgraded to using Rider and have caught a few times we missed this when using Visual Studio. Thanks for the content as usual!

[@epiphaner]

This was exactly the solution I was hoping to see because that is what I have been using for years :) As for the return type, I always return as specific as possible while accepting as generic as possible. Worked well for me so far!

[@KingOfBlades27]

This multiple enumerations text occured to me as well Resharper which is the sole reason I am aware of this behavior. Really good thing to teach to people.

[@figloalds]

I heavily use co-routines on my applications, specially for large operations, I can read 28k lines from the database, make them as objects, turn them into JSON and send them over to clients without loading 28k things in memory, then making 28k objects, then making a 28k items json array, saves a lot of RAM and avoids high-gen GC.

[@FunWithBits]

yay - I was able to find the issue before Nick pointed it out. =) Though, I only looked for it because Nick pointed out there was a problem though...probably would not have cought it in real life.

[@harag9]

Thanks for sharing this, didn't know about it, but I personally never use IEnumerable. However looking at colleagues code during code review I can now point this issue out to them when I spot it. Cheers.

[@Victor_Marius]

Possible issue. If they're not using the count method on IEnumerable there's no problem. But should point out the multiple resource access if they intentionally iterate multiple times.

[@DanStpTech]

yes I knew and it caused me a lot of trouble. Thank you for your explanation, always appreciated.

[@carducci000]

I do actually know of this, and typically do take this into account; I'd be lying if I said I catch myself [or others] every single time :). It's one of those things you miss if you're working fast

[@Marfig]

The general advice for any caller of iterators that return IEnumerable, is to not mix cursor calls like ForEach, with aggregate functions like Count if both results are in scope of each other, unless the first call casts the result to a collection or array and the second call uses that cast result instead. Not doing that is not just a matter of performance; that's even potentially the least of our worries. The problem is instead that most likely we just introduced a potentially hard-to-find bug if the source data can be changed by a third party between both calls. But if both calls are not related and they are out of scope of each other, do not cast. That's a potentially expensive operation in itself.

[@andreast.1373]

I've seen that warning before and had no idea what it meant. That was a great explanation, thanks for the video!

[@harag9]

Is that warning just a Jetbrains warning, or does it appear in VS2022 now?

[@andreast.1373]

@@harag9, to be honest, I'm not sure but I believe it's only a JetBrains warning.

[@harag9]

@@andreast.1373 Thought it might be, never seen the warning in VS. Cheers.

[@abugsbunny]

Great explanation of multiple enumerations

[@chazshrawder8151]

I learned this the hard way playing with Entity Framework when it first came out years and years ago. It was not a fun or quick learning experience! Unlike this video, which was both fun and quick 👍

[@za290]

Thanks for this video. I don't use IEnumerable. After that video i'll still so :) but i learn why i'm not.

[@Aweklin]

I didn't know about this before and will be more careful about the usage going forward. Thanks Nick

[@Spartan322]

Makes me think it be nice to have an enumerable type that when called statically constructs the IEnumerable once with minimal overhead when the function is called, while using ToList is clear, it would be nice to designate from the function definition without requiring to produce a list or other container explicitly. Lazy loading and enumerable reconstruction is deceptive when you're used to how containers work especially when this behavior is built into the language.

[@user-tk2jy8xr8b]

Strangely there's no OOB class or ext method to do this better. ToList would create a list that reallocates as it grows. What would be cool is a linked list of exponentially growing blocks - takes O(n) memory and time to build, but more efficient than just a linked list. And a common rule is: "iterate multiple times - use IReadOnlyCollection"

[@aaron4th2001]

I questioned the system when I used a Where clause on a list and then when I modified a value on that list it suddenly got added/removed based on the where clause. When I never updated the IEnumerable collection to reflect the changes. After trial and error I debugged it and write a before and after count and my value change somehow magically reflected in the collection, I got an inkling of how this enumeration worked after watching this video I've now discovered that Count and iterating through it, reexecutes the Where clause everytime.

[@masonwheeler6536]

8:45: "Know that the warning might be there but there might not be multiple enumeration in every single one of those occasions." Enumeration is the process of _going over the elements of the enumerable,_ not of creating it. When you have a List, LINQ's Count() can call the Count property directly and not have to enumerate the list to count it. But if you had multiple foreach loops or LINQ queries, that would indeed be multiple enumeration even if it's of a List or an array. As the video says, the warning is confusing. The problem isn't multiple enumeration; it's multiple _generation,_ which can be a problem for more reasons than just the performance hit. If the generation of the enumerable is non-deterministic for whatever reason, (maybe you have a call to a random number generator in there, or you're querying a database twice and someone else INSERTs something into it in between your two calls,) you can end up enumerating two different sequences of values when you intuitively thought you'd be enumerating the same values twice, which can cause bugs in your code.

[@levkirichuk]

Great and very important point Nick

[@coolkama]

To be fair, this is why I hate using Var. It can hide these issues. Where specifying type makes sure you know from the outset that it has still yet to be resolved.

[@Null-byte]

I recently switched to Rider and have build a library from scratch, and Rider gave me the warning. First time I ever stumbled upon this. And now you made a video about it :D

[@TkrZ]

loving the Barking joke at the start 😂

[@nickchapsas]

At least one person got it 🥲

[@HazeTupac]

Thank you for tip, quite interesting. One question.. Does your courses come with certificate at conclusion?

[@xavier.xiques]

Very useful video Nick, thanks again

[@robwalker4653]

If you are creating a list internally, why not just return a list instead of IEnumerable? What are the advantages of returning IEnumerable?

[@blazjerebic8097]

Great video. Thank you for the information.

[@vertxxyz]

I feel like you should also show the rider-specific "why are you showing me this warning" link they usually build into the alt-enter menu for warnings like these (when they exist)

[@nickchapsas]

This is such a good idea for a video or a short actually

[@nanvlad]

Wow, that's amazing! The more subscribers you have - the more basic stuff you'll have to explain :( The next thing, I suppose, will be the difference between ++i and i++ and how Interlocked.Increment(ref i) works with async/await code :)

[@nickchapsas]

You’d be surprised how many “senior” developers get this one wrong

[@Crozz22]

> how Interlocked.Increment(ref i) works with async/await code Can you elaborate on this one @kalistrat ?

[@jackkendall6420]

I would be down for Nick just giving his take on everything Eric Lippert's blogged about.

[@freddyflares2757]

That could've been phrased the other way round.

[@rvladkent]

I usually IReadonlyCollection or IReadOnlyList, unless really need lazy evaluation

[@sigtermnull]

I know about this feature and always take it into account. But I know too many developers who don't know about it. It's easier to list those who know Most often I see the multiple enumeration warning on projects where people use Visual Studio. I think it's partly Microsoft's fault that they still don't warn people about the multiple enumeration possibility, so people don't care

[@hero1v1]

Resharper taught me, now i understand it

[@EvaldasNaujikas]

Great video, but I think it is also important to mention that calling ToList() should be done only if underlying implementation is not enumerated. For example, in your example when your IEnumerable was returning List (instead of yield), a call to ToList() would copy the same list, which increases memory usage. And for new developers, they could start thinking after the video that ToList() should always be done if they are using a method that returns IEnumerable.

[@nickchapsas]

There are checks in ToList to prevent the extra allocation so you won’t increase the memory

[@EvaldasNaujikas]

@@nickchapsas but why then rider shows additional allocation of System.Int32[] and a new array in memory? And that additional +1 only happens AFTER ToList(). See the image here: snipboard.io/XkNj4A.jpg

[@EvaldasNaujikas]

And it even does the same if I use List as return type for GetNumbers. After ToList - a new array is allocated in memory.

[@EvaldasNaujikas]

And just for fun, I added four ToList calls one after another. dotMemory still sees the allocation snipboard.io/e2NrTl.jpg

[@stempy100]

@@nickchapsas incorrect. .ToList() will create a new list.

[@codefoxtrot]

Yes, I knew about this problem because Rider told me :)

[@abdellatifnafil]

thanks man u r the best!

[@corinnarust]

I really want a Nick Chapsas for Rust!

[@DoctorMGL]

does ObservableCollection have the same behavior since it implements IEnumerable in its base source ?

[@howardgod]

I don’t have resharper and don’t know about this until now…damn

[@zdavzdav86]

I could be wrong but I get the impression that most people tend to materialize `IEnumerable` instantly and that there's a lack of understanding on how `IEnumerable` works. Also, that was me some years ago...

[@quantum_net219]

This video made me subscribe 😁

[@FireDragon91245]

my first work day as developer / programmer today 🎉

[@nickchapsas]

Congratulations! You gonna smash it 🎉🎉

[@JavidLeo]

today lesson. thanks :)

[@doneckreddahl]

Can anybody telll me what Nick is using to show stuff like "x: "Nick Chapsas, 29"" and "splitline: string[2]" when he debugs? It seems to show the count as he debugs as well.

[@nickchapsas]

It’s just part of the Rider debugger

[@ClearSight2022]

I really like your videos. This one was good too but the title was not really correct. Your presentation makes it look like the problem is IEnumerable. Its the correct way to return something when you don't want to limit the implementation of the method to a specific collection type. In fact, the problem is the Select() and knowing when to use Select().ToList() . In fact, you didn't even use ToList() instead you used a clunky list.Add() method. Also Count() is to be avoided in some cases. Length and Count are preferred (but hopefully Count() uses Length and Count behind the scenes, but it still requires a method call and an if statement). Another thing to realize is that by returning a LinQ QueryCollection type instead of IEnumerable you can avoid the double iteration? In that case, your Title would have been more appropriate. Whatever, I enjoyed the video anyway. Your presentations are always top notch in quality !

[@MirrorBoySkr]

What is the better to use in such cases? ToList() or ToArray()?

[@nickchapsas]

It depends on what you wanna do with the result

[@MirrorBoySkr]

@@nickchapsas I just want to enumerate. So, it seems to me, ToArray() is more suit. But, I see that most of people around me use ToList().

[@tussle2k]

Yay, new video 👍

[@wertone]

How do you gain all this knowledge regarding perfomance?

[@nickchapsas]

I read a lot of the .NET codebase and try things out

[@ps-pu5dd]

lol, never used yield, always did materialization to list intuitively

[@qm3ster]

Why does no one have this problem with `Iterator`/`IntoIterator` in Rust? 🤔

[@stefanalecu9532]

Because nobody uses Rust so there aren't any people who would complain to begin with

[@BryanFlores-ox5fe]

one interviewer asked me to compare IQueryable and IEnumerable :)

[@adamstrejcovsky8257]

why is it that he can write a method without any enclosing class?

[@TheNorthRemember]

7 years c# programmer, just knew about it

[@jam560]

why would you ever need to return ienumerable anyway

[@TrickyCat]

"Hello Worldish" topic, and every jumior knows it (at least every juior I've worked with)

[@hmixa]

It is very bad practice to save IEnumerable at variable. If you need to save IEnumerable need to save it as array. By my opinion array is most basically type instead of list.

[@slygamer01]

I can't say I have ever seen a yield return of an IEnumerable in 15 years of C#. Return of an IEnumerator in coroutines, yes, but not this. Must be a business software thing.

[@bramburn]

Hello Nick, from everybody

[@RobinHood70]

This is another example of Linq hiding bad coding. If you don't use Linq here, it's a lot more obvious that you have to iterate the enumerable to get a count, since you don't have a Count property/method.

[@siposz]

IEnumerable can kill your performance in C#, but if you want to be 100% sure, combine IEnumerables with databases and Lazy loading features. Devastating.

[@Jhonnyoliv]

If you have 1M rows in your file, is better the first approach.

[@doctor9101]

Nick the great, in C we were warned calling length of array in a for loop, it will end up with O(n square) instead of order N

[@Victor_Marius]

It's hard to think that in C the length of an array would be evaluated by enumerating the array and not by returning an internal stored value. Of course it depends on the type of array. The standard array has no length stored or even possible to evaluate (of course you can iterate from the starting pointer until hit a null value or something like that but it's not recommended and pretty stupid). I see only for linked lists to have a length method that enumerates the list but that is not an array.

[@gostan2718]

I like your mustache

[@co2boi]

Good stuff. Curious, at the end of the video you said "I probably wouldn't return IEnumerable, I would probably return the Type". Can you expand on that? Also, in your example you are getting a count. Wouldn't it be better to use ICollection instead?

[@borisw1166]

I think what he means is an API designing question and what your intend is how to use your API. Ienumerables look like your you can filter and actually late execute the code behind that. While arrays or read-only lists express that there is no sense in filtering, because the "heavy" code is always execute, whether you filter or not. In his example filtering makes no "sense" since the file reading all the lines, creating and returning the object anyways. But ienumerable let's you think that you could filter in that and it actually makes a difference. Having an array or read-only list makes it very clear: the file is read anyways.

[@siposz]

If a function actually return with a List then the function return type should List, not IEnumerable. In this way the caller exactly know, what they get back, and could consume it optimal way. If I you see an IEnumerable return type, you don't know, what happens if you call a Count() on it. Anything could happen, for example a 5 second long lasting database call. Or it could throw FileNotFound exception. But a List is easier to deal with. If it's not null, Count() will be ok.

[@efrenb5]

Oh LINQ and IEnumerables, such a blessing and a curse. Very easy to abuse and misuse.

[@7th_CAV_Trooper]

Calling ToList or ToArray is forcing a multiple enumeration. You enumerate to create the array and then you'll enumerate it again to process it. Try to avoid using ToList/ToArray when you can. You only want to iterate any given list once. The count could have been computed by the GetCustomer and returned as an out param.

[@MrMatthewLayton]

I might LINQ my ting from Barking! 🤣🤣

[@Coding-jm2uy]

Example too simple, can not apply to production code. I don't understand.

[@RemX405]

TLDR: Most C# programmers use APIs and functionality they don't understand.

[@TkrZ]

TLDR: Most -C#- programmers use APIs and functionality they don't understand. 🤣

[@czajla]

As an old programmer I must say I don't like those modern ideas... gross

[@scproinc]

This is why I hate programming. Everything you do is bad. If you want to make anything, you have to spend 10 years coding overhead because it's more optimal than being productive.

[@WestCoastSwinger]

It doesn't necessarily take longer to write good code. It takes knowledge. And yes it can take a long time to learn that knowledge. (I'm not sure what you mean by "overhead".) There are other jobs that require a lot less learning.

[@Igor-xm1jy]

"var" is hidding valuable type info from dev. IMHO it is the worst feature added to c#.

[@harag9]

Not really, most of the time you know based on the text to the right - e.g. var x = new Customer() -- x is a Customer type. So it's easier to read when you put var. However when you don't know what might be coming back, or it's not obvious, then it's recommended to not use var.

[@i_fuk_religion]

After watching your videos, it feels like all the programming I have been doing is just wrong..

[@scanix100]

Simply call .ToList() and problem solved.

[@mastermati773]

When I started to think about it more deeply, this system is actually very very good: If we have some enumerable thing A given for a consumer B, how could B assume that it has enough memory to hold all elements of A? Ans: It can not, and thus it protects itself with this solution of multiple enumerations: If file read in this video was gigantic (let's assume milions of lines) then multiple enumeration IS desired! The solution is just to use IReadOnlyList, which has enough space saved prior to the enumeration.

[@marcusmajarra]

A common recurring problem among programmers is not knowing how the code they're using works. At the very least, they should understand what the API commits to doing. Deferred enumeration of IEnumerable is a great feature in C#, but if you're using any API that exposes an IEnumerable object, you should always assume that you need to enumerate at some point, unless your objective is to merely chain subsequent operations to perform on the object. In fact, if you never actually enumerate the sequence, it will never actually execute, and this is also an easy trap to fall into. So my best advice would be to write your API according to what client code should expect. If you're returning a finite object, rather than returning IEnumerable, you should return IReadOnlyCollection or IReadOnlyList (or any read-only interface). That way, client code knows that enumeration has already been performed. If you return IEnumerable, client code should assume that enumeration will be required, and even the implementation should probably avoid enumerating to a terminal operation. JetBrains.Annotations also has the [NoEnumeration] attribute that you can assign to an IEnumerable method parameter to indicate that your method isn't performing a terminal operation over the parameter.

[@youcantsee3867]

What do you mean by 'you should always assume that you need to enumerate at some point, unless your objective is to merely chain subsequent operations to perform on the object' ? I don't understand the part ' you need to enumerate at some point'. Could you explain it for me?

[@marcusmajarra]

@@youcantsee3867 it means that if you're dealing with an API that provides an IEnumerable object to you, you should assume that no actual query operation has yet to happen behind the scenes. It is only when you enumerate that the query is actually executed. For example, if the API operation is fronting a database call, no query is run against the database until you first enumerate over the results. This is different from working with a list or an array, which has already been materialized with contents. The enumerable object has no contents until you enumerate.

[@youcantsee3867]

@@marcusmajarra Thanks for your reply. I have one more question, so the word 'enumerate' is means calling some method like 'count()' 'toList()' or calling for each. Am I right?

[@marcusmajarra]

​@@youcantsee3867 essentially. If you're not digging into the results, you're not enumerating.

[@Tekner436]

@@youcantsee3867 A good example would be for instance in C# you call a function that returns an IEnumerable - IEnumerable list = GetList(); - you would think that doing a foreach (var item in list) twice would use the same results from GetList(); but the IEnumerable interface doesn't actually grab any data until it is 'enumerated' in a foreach. That means each foreach of list will execute all the actions GetList() did to build the IEnumerable result. You could something like IEnumerable query = customers.Where(c => c.Active); List result1 = query.ToList(); List result2 = query.ToList(); It's possible that result1 differs from result2. if customers data is backed by a database, each call of ToList (or any foreach statements) on the IEnumerable will build the query, execute it, and return a new result.

[@kenbrady119]

I seem to remember the LINQ documentation explicitly stating that Enumerables are lazy-evaluated. It is a feature, one that all developers should be cognizant of so that they can force one-time evaluation when appropriate.

[@jamesmussett]

The biggest problem I have with Linq in general rather then IEnumerables is the heap allocation that takes place when evaluating queries with ToList() and the like in memory-sensitive hot paths. In almost every other scenario it's absolutely fine, but it makes my life hell when I have to do rate calculations on 100-500 messages/s. Would be good to see you cover MemoryPool and ArrayPool at some point, those types have truly saved my bacon!

[@nickchapsas]

I have a video about object pooling coming, probably around October or November. It's a really interesting topic

[@jamesmussett]

@@nickchapsas Perfect, I look forward to it! =)

[@sealsharp]

@@nickchapsas Sweet!

[@urbanelemental3308]

BTW, there's a CSV competition article that covers all the CSV parsers for .NET and since the last time I looked the Sylvan.Data.Csv library was the winner and shockingly fast even when using types.

[@billy65bob]

Oh, that's neat. I'm still using TextFieldParser from the VisualBasic namespace, since it's the best one (and the only one) built into .NET itself.

[@crifox16]

so basically yield returning an IEnumerable is great when you work with transient data that doesn't get reused? just to know if i understood right

[@flybyw]

When you switch to .Select(), the file is only read once while each line is selected twice; and then you could just append .ToList() to the .Select() to return a list of Customer's without splitting each line twice.

[@smwnl9072]

The beauty of IEnumerables is lazy/deferred execution. A trap (per this video's message) if you don't have a grasp of what it is. Lazy/deferred execution I believe was borrowed from the Functional paradigm. The idea is that you have a set of logic/algorithm which wont be executed/evaluated unless with explicit intention. In C# LINQ, you express the 'intention' by calling operators like .First() .ToList() .Count() .Any() etc. Examples of lazy LINQ operators, .Where() .Select() .OrderBy() etc. These return an IEnumerable of . Lazy/deferred execution shines when composing/chaining functions and when you intend to use your functions in between a "pipeline". Hence the above 3 are often used in a query chain/pipe. Pertaining to collections, lazy evaluation passes only 1 item to each node/operator in the chain/pipe at a time. But for eager evaluation, the whole collection is evaluated and passed down. If there were conditions of 'early breaks', the latter won't benefit as the collection has been prematurely evaluated. E.g. a lazy pipe/chain products .Where(p=> p.InStock()) // each product 'in stock', will flow down.. .Where(p=> p.Price < 3.14) // but only 1 at a time and not the full list because 'where' is lazy. .Select(p=> p.ToShippable()) // Concatenated lazy chains act and behave as one (select is also lazy). // I often combine multiple individual lazy operators to solve complex problems with very little concern for performance penalty. // Shifting the order of the operators around is also quite easy as they are somewhat stand alone..