Is this safe? Creating a custom profiler

#gamedev #gameengine #cpp #indiedev
When making a game with my custom engine I noticed some lag, so I decided to build a custom profiler from scratch to see what was going on.
The engine is written in C++ and supports windows (DirectX and OpenGL), Linux (OpenGL) and macOS (metal).
I use some questionable OS functions to create a profiler.
💭 All views are my own 💭
Link to iris: github.com/irisengine/iris
Models from Quaternius: quaternius.com/

[@MAPster7000]

Me, learning OpenGL and game engine internals *woah so hardcore* This guy "Oh yes, I wrote profilers for 3 different OS's" o_o awesome =D

[@wilykary]

Game dev is really hard, especially if you build your own custom 3D engine that supports multiple backends 😂

[@RandomGuyyy]

Sometimes I feel like UE5 makes it too easy. Contemplating rolling my own engine, I look up channels such as Nathan Baggs. Then I feel a shock that sends me limping back to my safe sandbox like an animal who dared to touch the electric fence.

[@nathanbaggs]

I’d recommend rolling your own engine as a side project, you can make it as simple or as complex as you want

[@wokeupinthefuture]

great vid as always but i'd like to see more in-depth on your problem solving, like stepping thru source and showing what functions are actually slow, what did you do to increase perf and so on, anyway keep up with this!

[@boneyboi]

I found ur channel today in recomended and i love ur videos

[@nathanbaggs]

Thanks so much :)

[@LukeAvedon]

You are my freaking hero. This channel is amazing. Gets me so jazzed about being a programmer like nothing else.

[@Supakills101]

If this is just a hobby what the heck do you do as job?! Very impressive.

[@nathanbaggs]

Thanks, glad you enjoyed it.

[@penguthepenguinj]

Intresting I remember bukkit (Minecraft plugin server) used to have a timing system similar to the one you described, each action is timed in milliseconds and the highest time can be viewed on its Web interface

[@Andrii-zc4dp]

Linux to suspend use: kill -s SIGSTOP and to resume: kill -s SIGCONT and strace command gets stack trace

[@nathanbaggs]

That's definitely a way to do it externally, but I wanted to profile from within the process itself

[@gustavcoetzee5018]

Luv it. ❤️

[@nathanbaggs]

Thanks!

[@AraSec-hj5mx]

Hey, I liked your content. One day I wanna be as good as you in programming. I want to learn assembly and c and be very good at them. What books do you recommend?

[@nathanbaggs]

Hey thanks! I don’t have any recommendations for books I’m afraid. But sites like CodeWars and Project Euler have some good exercises for practising

[@unLinuxeroMas]

witch software do you use for your animations in your vidios

[@nathanbaggs]

I use OBS for screen recording and edit the videos with Shotcut

[@az-kalaak6215]

I was wondering wouldn't it be better to parse the code, inject some clock work, then compile it? taking a snaphot might be rigged because your cpu is waiting for it's pipeline to flush no?

[@nathanbaggs]

When you're doing something as low-level as profiling there'll be intricacies whichever approach you pick. Your suggestion is totally valid but one of the reasons I went with the runtime approach is that I'm producing a library and I didn't want to impose any undue restrictions on the user. With this approach someone just needs to link against the library, call start and they get profiling information. This also means I can integrate it into a UI in the future

[@az-kalaak6215]

@@nathanbaggs true, I did not consider this

[@itzblinkzy1728]

hello nice video i was wondering what them is ur vscode

[@nathanbaggs]

Glad you enjoyed it. It’s Dracula

[@TheSensationalMr.Science]

3:30 wait.... so you couldn't use 'SIGST' (Ctrl-Z for terminal) or 'kill -STOP PID' & 'kill -CONT PID' for linux? or strace as a way to get stack traces? (also for linux) Hope you have a great day & Safe Travels!

[@nathanbaggs]

Sure that’s possible, but I wanted a way for the process itself to capture stack traces. So with this approach I can just run a game and when it exits I can see where it spent it’s time

[@charactername263]

@@nathanbaggs yes going to shell is heretical, if shell can do it, kernel can do it, code can call kernel, code can do it, shell is a SHELL OUT. WEAK. Very principled to not cop out to shell. Spit on shell abusers.

[@guilhem7171]

I didn't got how you get from "I have a working tool that can stop my program and give me the stack trace" to "I have timing information about how my game run". Did you just get the stack very frequently to have statistical information ? BTW that's great content, want more :-P

[@sadlan7228]

I'm wondering about the same thing

[@nathanbaggs]

Yea I set up an extra thread (which doesn’t get profiled) and repeatedly pause and sample all other threads. The windows implementation is here and you can see the loop github.com/irisengine/iris/blob/2fe26e936abdf0d9eed73e100bd7d3590622ee46/src/core/win32/profiler.cpp#L182 Glad you enjoyed it 😊

[@KiritoDev]

Do you think i could port your engine to a platform that doesnt support any profiler?

[@nathanbaggs]

Sure just don’t create the Profiler object in the start() function e.g. github.com/irisengine/iris/blob/2fe26e936abdf0d9eed73e100bd7d3590622ee46/src/core/win32/start.cpp#L90

[@phizc]

Ah, yes, shader compilation stutter. A seemingly all encompasses problem. 😅

[@user-dh8oi2mk4f]

Can you explain the problem more in depth?

[@nathanbaggs]

The problem lies in how I’m building and managing “materials” which is just a wrapper around graphics specific shaders. I want to be able to reuse materials where possible but internally that needs improving. So at the moment I recreate the same material (which is slow) several times.

[@gustavcoetzee5018]

Teach me master

[@charactername263]

If you are willing to torture yourself with the worst API ever written, you can get hardware counter data using ETW to microbenchmark things like branch mispredicts in usermode. Might make for a nice follow up video.

[@nathanbaggs]

Thanks for the suggestion!

[@charactername263]

@@nathanbaggs as a followon from my last comment, I think Linux has some equivalent functionality to communicate from usermode to a kernel driver and query performance counters, and is much easier to use.

[@charactername263]

@@nathanbaggs Having done a bit more research, you can even get InstructionPointers from the ProfSample event, and I think if you did that with like an InstructionRetired Counter, you could have a quite nice sampling profiler for problematic instructions.

[@gustavcoetzee5018]

Argh. Damn. Harsh

[@darthtesla301]

It's very cathartic when a dev vlog starts with "game dev is hard" 🥲

[@nathanbaggs]

It’s something we can all relate to!