One trick I came up with in the process of fighting borrow checker in Rust is short pointers. My arena is at most 4 GB, so I don't need full-sized 8 byte pointers. So I use offsets from the base of the allocator instead and I overloaded indexing operator to make it ergonomic. Since I pass allocator everywhere anyway, there is little additional effort to use it. There are several benefits: graph-like structures become more cache efficient, I can potentially import compile-time data structures using include_bytes macro and even cooler stuff like building quad-trees on CPU to later traverse them in compute shaders on GPU without any additional preprocessing.
@cribalik4 ай бұрын
I agree with the last point. From experience on a larger project with implicitly-passed arenas, it quickly becomes extremely hard to keep track of to which arenas what data belongs. Arena parameters are annoying, but worth it in the end
@salim4445 ай бұрын
20:21 The problem of refactoring is mostly a tooling issue and from what I understand creating an arena is meant to solve a design problem, namely to specify which context is responsible for the memory, and the second easiest design is that which the same context is both responsible for creating and destroying it. As for tooling, one possible solution, currently impractical, is to inspect code while compiling and to modify it by a program. I know the idea of messing with the compiler is terrifying but if we develop tools to inspect the changes it might be feasible
@jack-d2e6i4 ай бұрын
Odin uses an implicit context system which works great. It's much more convenient than Zig (imo). However I've never ran into the refactoring issue described as I've never dealt with a large Odin code base. I disagree with the statement that the "whole" point of allocators as parameters is about the function signature. In my view the core point of custom allocators is to control how third party library code allocates. The same with logging. Odin has a logger on it's context, that way calling code can always control how library code logs are handled.
@cribalik4 ай бұрын
I had pretty big issues with it as the codebase got large. Knowing and remembering when and what arenas to switch to between function calls got really difficult quickly. You would accidentally have module A put data in in module B's arena, and waay later when you use module B, causing a clear on arena B (if you're lucky), it causes a crash in module A. That lack of locality of behaviour made the bugs really hard to find.
@TheSandvichTrials4 ай бұрын
@@cribalik Yeah, I'm not sold at all on using the context allocator in your own code. The idea that it is to allow you to control the allocation of libraries to me is also extremely limited in usefulness, because if the library is not written to consider allocation properly then realistically the only meaingful thing you can do by overriding the context allocator is make the whole library allocate from some allocator with free-all functionality so that you can bulk free the whole allocator after you're done with the library. I guess there is some value to that but it feels like a shallow victory, and it doesn't work for any library that has to "stick around" for the duration of the program. It also implies that the library has to actually use the context system, which is not a given that it will, especially when the bulk of the libraries out there are going to be written in C/C++.
@ivythegreat24084 ай бұрын
16:50 "When we're reading the code that defines an API we can immediately determine which functions return allocated memory because those are the ones that take in an arena parameter." What if a function which does not take in an arena parameter, allocates into some global arena and then returns that allocation? In that case you would not know that an allocation from a global arena was being returned simply by looking at the function signature.
@Mr4thProgramming4 ай бұрын
That is perfectly possible, and a fair point. In my own style I would never do that. I may use a global allocator, but I wouldn't return what is allocated. Returned memory is always tied to the explicitly passed allocator. It's just a convention not enforced, but it's a convention that I think does a lot of nice things.
@mgostIH4 ай бұрын
In the case we can construct allocators out of thin air (which imo is reasonable), you can apply the same reasoning on async functions, where a function b could just instantiate its own async executor, for example one that just calls the functions and waits for them to finish. This would also mean that the entire concept of function coloring is a wrong assumption.
@Mr4thProgramming4 ай бұрын
Super interesting point. Taking your thought further, I think both cases involve breaking out of the "function coloring" by doing a call wrapped in a constructed local context. Where those wrappers exist is significant structure in the program, it's not just a hack around the coloring. So the choice of when and where to place them matters. In the case of arena parameters, I think it's very common that this happens. It is even common in my code to construct an arena for temporary memory when there's already an arena passed to you too. Just because you are passed an arena doesn't mean that's the arena you pass to everything you call. I wonder if there is ever a case for an async function to begin it's own async executor?
@theevilcottonball5 ай бұрын
You have made some good points here. What do you think of Zig's approach of having always a "generic" allocator parameter vs a specific allocator parameter (general purpose, temporary, arena, pool, buddy, etc.) It allows the use to swap out the allocator, but I kind of don't like that the function's implementation cannot take much advantage of the specific advantages of the allocator, i. e. even if you pass an arena in Zig you still have to free, becuase it might not be an arena. Of course you can just accept an arena, even in Zig, but that what I saw from afar. I still write in C and have dabbled very little with Zig. Zig's even seems to recommend passing an allocator for a temporary allocation, whose pointer is not returned to the user in any way. Do you prefer having an allocator parameter for cases like that or do you not, since the allocation does not matter much to the user. Sam mention's implicit context parameters as a "solution", what do you think about them? Are they sort of equivalent to thread local storage - or globals for singly-threaded code? I know some newer languages like ODIN and I guess JAI have them. Do you think they are useful, when you seem to advocate against emulating them in C (via always passing a context struct, or always passing an arena)? I kind of like the approach of Chris Wellons of passing temporary arena's by value so they automatically reset themselves, but I still need something to try this out with. This means that you would pass two arena's one for scratch allocations and one for permanent allocations returned to the user. Arena's usually reserve a lot of memory (either by allocating a bigger than what you need static array, or by reserving a large portion of virtual address space). If I want to call an arena-function from an arena-less function at many places throughout a codebase, would I get into problems by reserving a lot of memory by creating temporary arenas to call arena-functions (unless I make them more appropriately sized, but sometimes it can be hard to guess how much memory is needed? (In general, how do you size your arenas?)
@oliverfrank81245 ай бұрын
I will just comment on your first point, I don't have much experience with Odin or Jai. Zig doesn't require that you only pass a generic allocator implementing the Allocator interface. I think the reason you're probably seeing that as a common idiom within zig code is that you're probably looking at library code (correct me if I'm wrong!). With library code, you often are trying to please a larger subset of users, and you (may) want to make your library flexible wrt allocation strategy. You're totally right that this may incur some performance costs, which is the reason some of the Zig std libraries provide "leaky" versions of certain functions, which take advantage of not needing to meticulously match every malloc with a free.
@_lod4 ай бұрын
@@oliverfrank8124was just going to mention leaky functions, but you beat me to it haha
@Mr4thProgramming4 ай бұрын
1. I don't know much about Zig's generic allocator. What I will say is that there's no such thing as a "generic" allocator. Here are several APIs for different perfectly valid allocators: malloc/free style: { alloc(Size)->Ptr; free(Ptr); } memory pool style: { alloc()->Ptr; free(Ptr); } arena style: { alloc(Size)->Ptr; get_pos()->Pos; pop_to(Pos); } There is no known way to abstract the differences between these and still control the allocator. Because of that they require different code. Whatever interface choice is made for the allocator is a real choice - it rules out the others for good. 2. I never pass down allocators for temporary allocation. Whether a function performs temporary allocations is an internal matter for the implementation of the function, it is not encoded in the interface. 3. Contexts in newer languages are a little strange to me. They do basically offer the same thing as thread local storage. The only difference is they are somehow language level. Maybe the language defines what's in the context. Maybe the language defines how the user defines what's in the context. IDK. I just use thread locals when I need it to be implicit, and explicit contexts for larger families of functions like a parser. 4. Don't care for passing temporary arenas by value. It's cute, but my arenas do things like chained blocks for arena growth, so I need explicit cleanup points in the code. 5. This is why I don't actually construct a new arena whenever I need temporary memory, I use a pair of scratch arenas on each thread. Whenever I need a locally unique arena I have a single call arena_get_scratch that retrieves one. A little care must be taken to avoid arena aliasing, but otherwise you just get a temporary arena as if you just constructed it, but without the overhead of constructing it. See my videos on scratch arenas more details. Thanks for your questions!
@tychepi65764 ай бұрын
Thank you for introducing me to Sam's channel!
@10e9995 ай бұрын
Fascinating discussion. I can relate with the feeling that Allocators are contaminating (or coloring) my codebases; especially the "base" layer (fundamental utilities). That said, at some point, I think you need to bite the bullet. I had the same realization when I started to use string slice (or view) instead of C-string: this improved my codebases drastically, but it required a compelete re-write of everything touching strings. As a follow-up video, I think it would be interesting to explore the type of arena (not allocator) that can be use full in a codebase. I can imagine the usefulness of a "frame" arena (or main loop arena) that is used as a temporary arena for inputs. Or maybe a UI-Widget arena, that lifetime is bounded to a UI element.
@TrystanBrock5 ай бұрын
Completely agree with this take. When I watched the original video I had a similar reaction, but as always you vocalized it better than I had in my head. Much love to Sam, maybe he'll see this and do a reaction-reaction video. lol
@samhsmith4 ай бұрын
The reaction-reaction video is planned.
@michaellee27865 ай бұрын
Wasn't "function coloring" coined to criticize async/await? The semantics there are a lot different than allocation. The argument here is about API design. Loads of devs are stuck using libraries and can't just fork all of their dependencies or write them all. I think that popular perspective is the friction. Industry devs want the ability to control the allocation of library code, and there's no 1 correct answer on how to achieve that.
@ruroruro4 ай бұрын
While I somewhat agree that explicit allocators don't *exactly* behave like async/await, I still think that they **do** result in something similar to function coloring. Keep in mind that in the case of async/await you also actually **can** call colored functions from non-colored ones by submitting the task to a global event loop and then suspending the thread or by creating a temporary event loop and blocking until the async task is completed. Notice how these methods of working around colored async/await functions are extremely similar to the workarounds proposed at 8:08 in the video. The malloc/free global allocator is an extremely flexible allocation method, but it has a significant overhead. Similarly, free threads are an extremely flexible concurrency mechanism with a significant overhead. Async/await and explicit allocators are both mechanisms that allow for cheaper solutions to these problems, but introduce additional constraints on code organisation. So I still think that in practice explicit allocators introduce something extremely similar to function coloring.
@j-wenning4 ай бұрын
Upon rewarching the video, I'm even more bothered by this idea. JS has a history of `this` contexts being absolutely miserable to work with, and "modern" style has come to learn the benefit of a more explicit control flow, avoiding of re-binding methods, and purity. Bringing this into the world of systems programming is just stuffing more "magic" into a space that it doesn't belong, for the sake of saving a few keystrokes.