Achieving compile-time performance with Reflection in C#

Become a Patreon and get source code access: / nickchapsas
Check out my courses: dometrain.com
Hello everybody I'm Nick and in this video I am going to show you how you can optimize reflection in C# and .NET and try to achieve compile-time performance with it. This might not be applicable to every single reflection usecase but it will definatley be applicable to a lot of them and I think there are several gotchas in this video that you can use in a variery of cases.
Timestamps
Intro - 0:00
Traditional Reflection - 1:58
Optimizing Traditional Reflection - 5:05
Super fast but less flexible Reflection - 6:39
The problem with this approach - 11:00
Super fast and flexible Reflection - 13:07
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasGitHub
Follow me on Twitter: bit.ly/ChapsasTwitter
Connect on LinkedIn: bit.ly/ChapsasLinkedIn
Keep coding merch: keepcoding.shop
#reflection #csharp #dotnet

[@MaximGorbatyk]

Really useful video. Thank you for the demo of how the reflection works

[@sps014]

This video is pure gold, thanks , never thought reflections can be made this much faster.

[@SuperTerracraft]

Nick, I absolutely love your videos, and the way you go in depth about the topic at hand. Also, I'm a big fan of reflection and C#, so this video pleases me profoundly. Keep up the work, you're doing fantastic! 😄

[@dotanoob466]

Dude. You have the best videos. On my way to sr dev because of youre taking my game to the next level. Thank you!!

[@hoisinholdup]

Would love a vid on Expression Trees my guy

[@sigma_z]

I would second that. Especially on performance tuning Expression Trees.

[@nickchapsas]

Expression Trees are such an interesting topic to me but it's so hard to talk about a tree of any type without having a way to visualizing it as you're going

[@X39]

They are also a way to get that stuff done too But much more complex to setup

[@davidrogers8352]

You would need a series on Expression Trees. The fundamentals themselves are not briefly conveyed. There's quite a bit to them.

[@chriscardoz]

@@nickchapsas I've recently implemented Expression trees (compiled expression) in a custom json serializer to mask or remove any secure data from the logs in my gRPC interceptor. Probably a topic as such could help create a video.

[@tobyjacobs1310]

War story: I once had to address an issue in a library I had no control of. I worked out that I could use ILSpy to get at the internal class and inject the logic I needed, but then I had to make sure it was actually used instead of the actual internal class. Expression trees + a lot of static constructors and extension methods later I had a customised way of using the other library. With basically zero performance penalty... So fun!

[@rafaelocariz1384]

Your videos are pure gold! Besides that, I must admit: I see this stuff as some kind of sorcery hahaha. Thanks a lot for the magnificent content you provide us

[@matthewkupriyanov6504]

Mind blowing!

[@user-pu4qu9my5j]

You also just can use Expressions. Look: you can get methodInfo from property getter and then create compiled function from Exprression.Call with method info in parameters. At the end you need to create Expression.Lambda and cast it to Func. Then it expression must be compiled and saved into static field as delegate. It can shows similar performance with simple delegate creating.

[@JobertoDiniz]

He could extend in another video to benchmark using expressions

[@user-pu4qu9my5j]

@@JobertoDiniz honestly it's perf is like IL emit. It's not really something new :)

[@SebGruch]

Interesting... Do you have any sample gist fur such syntax?

[@user-pu4qu9my5j]

@@SebGruch me not really. But you can inspect open source project Automapper. They have the space on github. This module have mapping logic where result of reflection using in expressions to compile mapping delegates and store in the concurrent dictionary.

[@SebGruch]

@@user-pu4qu9my5j Thanks. Usually I used to cache Get/SetMethod info for frequently serialized objects, but I thought about improving the performance. And actually, I've been playing with this until succeeded ;-) Yet gonna look into AM code, for sure. static void TestCode() { var instance = new SearchParameters() { FileNamePattern = "adsf asdf" }; var refType = typeof(SearchParameters); var refProp = refType.GetProperty(nameof(SearchParameters.FileNamePattern), BindingFlags.Instance | BindingFlags.Public); var objParameterExpr = Expression.Parameter(typeof(SearchParameters), "instance"); var valueExpression = Expression.Parameter(typeof(string)); var getterCall = Expression.Call(objParameterExpr, refProp.GetMethod); var setterCall = Expression.Call(objParameterExpr, refProp.SetMethod, valueExpression); var getter = Expression.Lambda(getterCall, objParameterExpr).Compile(); var setter = Expression.Lambda(setterCall, objParameterExpr, valueExpression).Compile(); setter.Invoke(instance, getter.Invoke(instance) + " 111111"); Debug.WriteLine(instance.FileNamePattern); }

[@msafari1964]

🎉🎉very good 💯

[@KoScosss]

Thanks, never heard of that.

[@ibnfpv]

Great content!

[@AFE-GmdG]

That's a WOW!

[@addkim7889]

Absolutely love the video you made.Since my company has code using reflection, I have been looking for a way to speed up the existing code base and found your video. One thing to note though, and had been pointed out by other comment(s), is that the Expression API do the same thing as Sigil, with less confusing syntax(IMO) and according to my bench marking, also faster than Sigil, this is a small sample and I have not tested it against pure IL Emitter code but just something to consider. | Method | Mean | Error | StdDev | Gen 0 | Gen 1 | Allocated | | IlGenerator | 69.39 us | 0.233 us | 0.218 us | 6.1035 | 0.1221 | 38 KB | | ExpressionGenerate | 42.82 us | 0.848 us | 1.103 us | 0.7324 | 0.3662 | 5 KB |

[@johnathankrausrig9237]

I ised to have a issue on my Rest library where i could solve the problem by caching the propinfo. But just getting the getter is genius. I need to implement it! Thanks for sharing it with us.

[@TheMonk72]

You can use LINQ Expressions to build a function that gets the property value, then cache that.

[@johnathankrausrig9237]

@@TheMonk72 the reflection is required because i use attributes to Mark properties which are used in the request. I dont know how this could be achiveable by using linq expressions. How you mean it?

[@utubekade]

I had to hook to an event which was internal to another library. I ended up using IL emiting approach, only wrote the IL manually

[@aughey]

Dynamic assemblies at run-time are life-changing. I wrote a program that drove dynamic execution through various traditional techniques such as data-driven calls through virtual methods. Even pre-caching using Action/Func was still 15 times slower than native. Using techniques from this video, the differences between compiled vs run-time compiled are imperceivable because it's literally the same code.

[@renauddanniau676]

Can you elaborate ? It seems interesting to me :)

[@aughey]

@@renauddanniau676 The applicaiton was essentially unrolling a graph of logically connected nodes that perform computation. Like the visual node graphs that you'll see in applications like Unreal Engine or Houdini. Through reflection or a lot action indirection, this graph can be executed in an interpreted sort of way. Applications like simulink will generic code that is compiled, however using C# and building IL code on the fly, I can unroll the node graph at runtime and generate native-speed type-safe code without extra boxing or runtime interpretation.

[@renauddanniau676]

@@aughey You mean that your users generate some codes, then precompiled some assemblies and finally inject them in your application ? And thanks to the IL-emitted code you are able to run their codes in similar native performance that is ? I thought that you could use the activator class to get the desired performance by reading via reflection the name of the classes inside the assemblies injected. And when you have found the classes that could potentially derived from your interfaces, you just use the activator class, create the instances and cast them to your interfaces. And you should have native performance with this trick. Am I wrong ? Is your code visible anywhere ?

[@aughey]

​@@renauddanniau676 The generic application is a node graph architecture Wikipedia(Node_graph_architecture) where nodes are precompiled static methods or method instances of an object. The "user" can manipulate the graph to create extensive interconnected networks of nodes that represent a computation. The DAG is then unrolled into a linear execution where nodes consume and generate data. Through reflection I can create an executor by inspecting the MethodInfo to extract parameters and ultimately call the function using MethodInfo.Invoke. Depending on the types, IDisposable overhead is incurred as well as boxing and unboxing of valuetypes and managing the storage. This is what happens through roughly minute 7 of Nick's video. The IL generated version of the executor still uses reflection to inspect the types, but then generates the straightline code (IL) to manage data and execute methods without runtime reflection overhead. I've benchmarked the IL generated runroll vs unrolling it by hand and they are equivalent.

[@phizc]

@@aughey this might be similar to something I did a couple of years ago.. I wanted to parse the data files for Daz Studio's 3d models, which are json files that are several megabytes in size. I found that the json deserializer in dotnet was too slow.. can't remember if it was dotnet's built-in or Newtonsoft's or both, but anyway I decided to write my own. It analyzes the C# classes and generates IL for the deserialization. My best version (out of 31) was more than 3 times faster. than the other methods. It was originally in .net framework (4.7 or so), but I managed to get it to work in dotnet core too.. Just can't export the generated assembly to a dll, so it's a little slower on first run.

[@berylliosis5250]

I don't know .NET IL and calling conventions well; are there any specific dangers to watch out for? Presumably you can wreck the stack and mess with the wrong registers (?); anything else? Also, in this case, since you've already written the code, it's probably pretty safe to copy (unless the requirements for safe IL change frequentlyish).

[@X39]

Did not know about that Delegate utility method. Would have loved to see linq expression too (instead of emit eg. As that really is dangerous territory and can be F up horribly easily)

[@js6pak]

you could create the instance with IL aswell

[@kamyker]

Dno if this is a bug in net standard 2.0 but I had to use Func delegate to make it bind. 8:30

[@bluedev6304]

I wanna access all the members with a pirticular attribute and i'm using reflection for it which take 5 sec. will this emitted version improve it? btw i want to be able to find the attribute even if it's on another assembly

[@WarrenLeggatt]

I have a love/hate with reflection. The way I see is if you are using to get to private data in an object then that is a bad code smell. If you are accessing public then lambda, expression and also something like a Haskel Lens system are better and more performant. The lenses also compose so give a great means to get/set data fast

[@smikkelbeert]

Hi Nick, you are saying that you used this on a production level. Could you explain to me some situations where you might need reflection?

[@JustArion]

Hey, intermediate level here and shower thought: What would be the performance impact of static vs instance? ofc Rider suggests static and my assumption would be that static is faster. I've looked into the IL Code and there's not much difference and BenchmarkDotNet throws "Benchmarked method `something` is static. Benchmarks MUST be instance methods, static methods are not supported. " As I would like to know exactly how much of a performance impact both has.

[@nickchapsas]

Static doesn't really affect performance. You shouldn't worry about it.

[@Marko-the-Beast-Master]

Hello, And what if I don't want to call method name from some instance but inside a specific class. Dont want to call new VeryPublicClass, just call it from the VeryPublicClass in some method.

[@shadowkras]

My Voodoo skill just leveled up.

[@TheAceInfinity]

You can do the same with Expressions in C#

[@TankersonWoT]

I have a question - rider constantly prompts making methods that are not dependent on object's fields static. It results in having bunch of private methods that just process some strings or values being static. Is this recommended or not? From my point of view it can cause bottleneck and issues with parallel execution when many objects refer to the same method. What is your opinion?

[@nickchapsas]

It’s basically the same. You can safely make them static. This depends on what the method is doing but the use case you describe makes sense

[@TankersonWoT]

@@nickchapsas Thanks for the reply

[@agsystems8220]

In a sense all methods are static, with instance methods being shorthand for methods defined in a type where the first argument is of that type. Extension methods do this explicitly. When the compiler sees ObjectInstance.Method(argument) it first treats it like Object'sType.Method(ObjectInstance, argument). By making it an instance method you are effectively just giving it a redundant argument. As for why it doesn't cause problems, methods are immutable (and this is a also good lesson on why immutability is awesome for parallel programming). That means that the method doesn't need to be synced up. You can have as many copies of a method as you like without caring, and as many readers as you like. Instances of objects don't actually carry the methods around with them, instead just a reference to their type, which is where all the methods live (usually, exceptions apply). The only implementation difference between static and instance methods is whether it takes an instance as an argument or not. When you see that static variables live in the type, while instance variables live in the object, it is natural to assume that static methods live in the type, while instance methods live in the object. Natural, but wrong. They actually both live in the type.

[@cedricsmit8794]

Would like to see a video about quartz with .net core 5 from you.

[@FromBeaverton]

Thank you, amazing video! However, if you need to access private properties, you need refactoring (if it is your library) or a new vendor (if it is third-party), not reflection. There is a valid use-case for reflection, though - enumerating (public) properties or fields of a class and passing them to some type-specific properties or methods in your class. This is handy in serialization. Another case when it is handy is when a generic base class is getting a generic type from a derived class and needs to do something with each field of a class of template type. I wonder whether this delegate approach is applicable.

[@ya4eburashka]

Generated private stuff still need reflection to access them.

[@T___Brown]

I fall into the wow'd group.

[@user-tk2jy8xr8b]

Wow, used IL-generated delegates, but didn't know there was GetGetMethod+CreateDelegate approach

[@DamianWalczak]

hey qucik question, are you sponsored by JetBrains ?

[@nickchapsas]

I am not. If I was I would have to disclose it. That being said I am a Microsoft MVP which means I get JetBrains products for free but I was paying for them before the award and I would pay for them after the award.

[@mosth8ed]

@@nickchapsas +100 on that. I had a student version of the Jetbrains suite for a while, I pay for the personal suite now and for the last few years. Worth every. single. penny.

[@yehudamakarov]

Maybe soon you’ll burn out from c# and do some higher level stuff? Or maybe some rust or go? 😁

[@oldfish3059]

We hope to increase Alipay by payment

[@neralem]

I know this makes no difference but you actually get the VeryPrivateProperty in your CachedInternalProperty. Copy paste failure.

[@nickchapsas]

That's what I wanted to get. Internal refers to the class on this one not the property. Should have given a more descriptive name tho

[@neralem]

@@nickchapsas Oh of course...excuse my brainfart. Haven't slept very well^^

[@mathboy8188]

His Benchmark video: "Benchmarking C# code using BenchmarkDotNet" kzbin.info/www/bejne/e4jQppmYi6tld5Y