Optimizing The Evolution Simulator To Be FAST

New vid new problems, time to make these AI creature evolve way faster by making the whole evolution simulator (Vitanova) faster and just optimizing it as much as possible, multi threading is painful man
GitHub repo for the project github.com/4t-2/EvolutionSimu...
The Bibites Project, an inspiration for this / @thebibitesdigitallife
The Life Engine, an inspiration for this thelifeengine.net/
Discord - / discord
Music used for this channel
Defense Matrix - • 'Defense Matrix' by Vy...
Sharp Edges - • Sharp Edges
Will 2 Pwr - • Will 2 Pwr - half.cool...
Skystrike - • [Non Copyrighted Music...

[@ShadowDrakken]

If you're not already, when you do the collision checks you can mark both entities being checked so that you don't recheck collisions from the other entity XD

[@peppidesu]

also add quadtrees

[@imlemonth]

can't wait for the next video :3

[@MyDude199]

Large map sizes are great for speciation

[@Lawtro]

Smart optimisations I might younk some of those for my own program

[@knoblauchbrot1667]

I just found your channel randomly it was one of the first recommended videos that I saw after firing up KZbin. And I love these kinds of videos where there is a problem that you are facing and how the creator solved this is explained into detail what they have done, why and how it helps. I also love the little bits of uniqueness sure it may also be laziness but it also makes it unique and funny. Like when I first saw a text on the bottom left saying "No Animation" and only then I realized that the Fire is not animated. And I actually grinned. But yeah keep it up! I like this style

[@timtrussnersims]

Very cool to see someone go through all the same thought processes and struggle with the same problems. You write everything from scratch 100%? Thats insanely cool Make the grids much bigger and use a worker pool that you assign jobs. It will make multithreading worth it. Also there is another fun trick you can do to do this without any locks at all. Also just use arrays. When removing an element just replace the removed element with the last element in the array. No need to copy the whole rest of the array. THough im sure even copying will be faster than a linked list.

[@Sander-Brilman]

looking forward for your next video, great job so far!

[@peppidesu]

Hi, the thing i typed sounds low key rude, idk did not intend that, but i see a lot of ways this can be improved further: ecs is better performance wise than your solution, since it stores components of the same type in one big list, making multithreading a breeze to implement. not only that but the data-only components are idealy stored by-value in the array, which gives you optimal cache utilization when working on them. In your version, you lose these two arguably biggest performance perks of ECS, so i wouldn't call it "better". having the systems detached from the components still allows multithreading to be easy to implement, although suboptimally (due to some entities not having the component the system looks for, therefore causing the thread to block until the others are finished, unless you use a concurrent queue or something to make work available to finished threads asap which you should do anyway). The linked list also hurts performance due to suboptimal cache utilization. If you want to quickly remove from a list, you could've instead noticed that update order of the creatures does not matter, so you can grab the last element in the array (which you can find with a counter variable alongside the array) and swap that into the empty slot. thats a O(1) remove operation with contiguous memory, so you get the best of both worlds. If you really want to improve performance, you might want to consider outsourcing calculations to the GPU using compute shaders.

[@scasz]

My junk shack implementation actually does look through data sequentially as every kind of object (food creature etc) is stored in a seperate list and if i wanted to access the physics component it would look sequentially through kind of object that has that component. If an object doesnt have a component it is never even considered and the list its in is never touched. My OO ECS should be as fast as any other regular ECS, multithreading in this is just as hard as any other ECS

[@scasz]

My junk shack implementation actually does look through data sequentially as every kind of object (food creature etc) is stored in a seperate list and if i wanted to access the physics component it would look sequentially through kind of object that has that component. If an object doesnt have a component it is never even considered and the list its in is never touched. My OO ECS should be as fast as any other regular ECS, multithreading in this is just as hard as any other ECS

[@scasz]

I can implement the linked list alternative though the bug issue is that most of big updates happen through me doing updates to objects near each other in the spatial hash grid so there are technically about a hundred ish lists with there being one master list where the data is managed and each grid square just holding pointers to specific elements, if I make the master list contiguous and not a linked list then the performance increase would be minimal at best as I rarely loop through everything and do mostly local checks and if I instead remove the master list and make every grid store the data continuously for every entity inside it (which I may end up doing but later as Im not sure on it) then Im not sure if the performance would increase that much either as every time an entity would move out of its grid then memory would need to be shuffled. And either way it would mean extra code will need to be added to get a pointer to an entity (because it may be shuffled around in memory so ill need to do some extra stuff to get around it). I have actually been looking into trying to get some of the stuff done on the GPU or maybe using SIMD but I think as of now its fast enough (but ill just have to see how badly evolving complex bodies will destroy the optimisations that I have currently)

[@mollthecoder]

Let's say a thread has 2 jobs to complete: Job A and Job B. If Job A requires Resource A and Resource A is locked, why can't the thread just move on to Job B while it waits for Resource A to become unlocked?

[@scasz]

That would help stop threads from sleeping but then the thread will need to keep track of which jobs it has done and which it tried and failed to do, and I just got around the problem with the “mega jobs” idea as giving the thread an entire contiguous strip of grids helped make it so two threads wont be accessing data close to eachother (similar idea to random grid access but works better) and ive also made it so the the threads only check the grids that they need to check (as before I had it so that to update the physics for example it always checked the grid an object is in and the surrounding ones but now it only checks the surrounding grids if its close enough to the edge if the grid its in) so less grids being checked means less chances for a thread to be locked.

[@UnChiller]

0:18 I can't really, I'm trying *to* die

[@ShadowDrakken]

why not use a thread queue instead of locks? Threads can move through the queue and skip over any grids that are busy, allowing the threads to always being doing something. When a thread is done it goes back to the top of the queue looking for the next thing to do.

[@scasz]

I did consider that but doing that might cause it to slow down as it will also need to manage and keep track of what grids it has already done and also it skipping over grids could cause the entire program to freeze as if the threads could skip over grids it can get into a situation where it freezes along with another thread which means the entire simulation is stuck and cant continue, maybe i could try and rewrite it but the main slow down isnt locking in the simulation code but locking in the job managing code so the speed up would be minimal

[@germanminer1276]

how would i run this on my pc, i dont understand the github instructions

[@scasz]

On my discord server there is a download link to a zip with the program and all you do is unzip it and run the program and you can start evolving life yourself. The link the discord server is in the description