Multithreading the Entire Destiny Engine

In this 2015 GDC Talk, Bungie's Barry Genova explains how Bungie turned almost every part of Destiny's engine into a a job graph, with only limited use of thread-based pre-emption. This talk should prove useful to programmers working on demanding game engines of their own.
GDC talks cover a range of developmental topics including game design, programming, audio, visual arts, business management, production, online games, and much more. We post a fresh GDC video every weekday. Subscribe to the channel to stay on top of regular updates, and check out GDC Vault for thousands of more in-depth talks from our archives.
Follow us on Twitter
/ official_gdc
Check out our Facebook page for GDC exclusives
/ gamedevelopersconference
Visit our site: ubm.io/2ctNvqZ

[@dandymcgee]

I knew multi-threading was difficult.. but goddamn. Someone give this guy a hug.

[@alengm]

Hey I'm subscribed to you :D

[@TripleMoonPanda]

Man Destiny's community needs to hug this guy and also hug all the rest of the Devs that keep this game going season to season. This games community gives Bungie so much shit like calling them such a greedy developer with bad monetization practices because prices on content are slowly increasing due to inflation and also because they're now one of the only AAA level independent game studios while apparently not getting any founding from outside, at least no there no public information about any recent outside founding. A game company with ~1000 employees isn't a cheap thing to keep running especially when that game company is doing their own publishing. Or the community say things about how lazy Bungie is about making new content and at the same time they bitch about how bungie have to vault old content to be able to add new content because they're trying to keep the games files size at a realistic amount for console gamers and also keeping the core game at a manageable size to help keep it stable. Plus over the past year Bungie has been working with basically a new and most likely very different game engine because of all the major under the hood changes they made last year with the engine updates they launched with beyond light. Before those updates it took 4-6 months for them to be able to push out any major updates while over the past year they've been able to push out major updates in less then 2 months. Its' like almost no one ever takes the time to learn about any aspect of how game development works. No one seems to even bothering to get a base line basic level of learning about how insanely difficult and complicated game development can be. From the coders to marketing and PR all the way up to the management CEO there's so much going on inside a company and it's like most people are incapable of realizing that everything going on at all levels creates so many variables that effect even the smallest aspects related to a game. Hell I'm not even a dev, the most experience I have with game developing was some really shitty mods I made for oblivion, skyrim, and fallout games that I thought were so bad I was to embarrassed to even share them with anyone. That's not a lot of experience but working with those modding tools that bethesda released to the community gave me a very good idea about how hard this shit is. Learning to make even a few very basic mods for a game was a very humbling experience.

[@austinfritzke4326]

Destiny has destroyed this man's soul.

[@iucidium1982]

Austin Fritzke it destroyed every Bungie Dev lol, 8+hours to implement changes... I'd have no soul too

[@ZKtheMAN]

Y'know, at first I was like, "Damn, this guy's got _no_ presentation ability," but after a while it got really soothing. Not sure about information retention, but hey.

[@JuanCamiloAcostaArango]

This man makes me feel depressed... I'm going to cry

[@bcuz8998]

He wants to

[@buffetize]

haha, comments are funny. I kind of like his talking voice for some weird reason. Kind of soothing

[@Luorax]

Thanks for uploading this video, I wanted to watch this so much ever since I first read about it!

[@Cerekay]

Great video! Good multi-threading techniques I haven't used. I'm not in the game industry, but this is still useful where I work as general C++ techniques to go try.

[@daarekcabrera8247]

Very interesting and insightful. I'm wondering how the engine has changed over the past five years. Especially since there were talks about the engine overhaul with beyond light and vaulting content. The fiber system makes a lot of sense... I wonder if it is industry standard at this point? Now that every other game is a live service.

[@NocturnalJin]

I'm sold. Jobs with permissions is superior to thread syncing. Reminds me of the design philosophy behind ZeroMQ (the best cluster comms system available IMO).

[@foobar2285]

Unfortunately it's only as good as your policies.

[@andmefikri7555]

Thank you for this talk!

[@capability-snob]

Region-parametric effect typing in C++ : check everything dynamically. I hope at some point we have tools that do this for us and the freedom to use them.

[@adud6764]

Wow, never listened to a less enthusiastic speaker.

[@MyLittleMagneton]

Very interesting talk! A shame he sounds so depressed : /

[@mikerathbun5449]

I would imagine an hour of high energy multithreaded engine talk would have been exhausting for everyone. A great talk of course!

[@rodericasmoana6937]

not everyone can bring the showmanship of the nickelodeon kids choice awards, you so crave.

[@thedankestdungeon8091]

I would be too if i worked on Destiny.

[@amfern]

multi-theaded game engines makes you depressed.

[@westonford6317]

Yeah, but the alternative isn't to sound like a depressed high schooler who's only doing it because his parents are making him.

[@anton6650]

Does this mean that they create the entire execution tree and all markup for all those jobs before execution?

[@nutritionalyeast7978]

what is the software he used to create the graph at 15:30? he said "tuner" i think, but i cant find anything by that name besides radio tuners EDIT: its SN Tuner from the PS3 devkit

[@kidkool27]

I think it was part of the PS3 SDK

[@randomfootages9120]

I wish Epic also bring Full Multi-threaded-rendering in UE5

[@ENZO-xu4sn]

PREACH! 🙏

[@randomfootages9120]

@@ENZO-xu4sn should I or not

[@YRBYD]

UE5 rendering is single threaded?!

[@SirMo]

How does an i3 dual core beat a Ryzen 7 8core if this thing is so well threaded?

[@rixzin5046]

It isn't

[@raingloom5334]

Soo I'm not an industry person, but this sounds like something Rust (the language) would have been good for. Of course back in 2015 it wasn't where it is now and it is still not as mature and the middleware Bungee uses probably wouldn't support it, but I imagine the multithreading would have been easier.

[@Raattis]

The development on Destiny begun in 2010 and it's built on an even older codebase. Even if that wasn't the case it is not trivial to transition to a new language when the programmers aren't used to it and its benefits to game development have yet to be proven in practice.

[@raingloom5334]

True, but needing that huge farm of machines for validating your threading model... that kind of thing is only feasible for a huge studio. Of course a smaller one wouldn't be making a huge mmo anyway.

[@GladerDev]

​@@raingloom5334 Many millions of dollars at stake. Worth the money.

[@alexkfridges]

it's not common practice to just rewrite an entire ecosystem in a new language for every project. They probably had >1 million lines of existing c++ code in their engine that is well tested and shipped games when the project began.

[@YRBYD]

Multithreading is always complex, no matter the language

[@MentalJihadist]

LeRuse

[@voxelamateur]

why is he wearing a red hat? is his head freezing or something?

[@pcuimac]

Multithreading the terribly stupid one dimensional storylines would have been of greater importance.

[@vast634]

Why is he wearing his grandma's swimming hat?

[@KlutchKwonDo]

Sounds like someone forced this guy to do this talk. So monotone

[@greyfoxzero]

Bullshit, they could never get the multi-threaded render path to work properly.

[@GnoClip]

Is this guy for real with that hat?

[@ddostesting]

Why is he wearing such a warm wool hat indoors? Seems bad form and uncomfortable.

[@Obmats]

The same reason every man wears a hat... Balding

[@ryanscottwright]

weak