How to make C++ run FASTER (with std::async)

Go to www.hostinger.com/cherno and use code "cherno" to get up to 91% OFF yearly web hosting plans. Succeed faster!
Patreon ► / thecherno
Instagram ► / thecherno
Twitter ► / thecherno
Discord ► thecherno.com/discord
Series Playlist ► thecherno.com/cpp
This video is sponsored by Hostinger

[@TheCherno]

Thanks for watching, hope you enjoyed the video! Don't forget to check out hostinger.com/cherno and use code "cherno" to get up to 91% OFF yearly web hosting plans! ❤

[@CatDaWinchi]

Hello, What do I need to create a std::mutex object? I`m compiling project with mingw-gcc (using cmake, under windows) and I cannot include a mutex lib (#include and then use it). Only solution I`v found is to use WinAPI wrappers around those calls, like HANDLE mtx = CreateMutex(...) and others that declared in header. Is there a way to use std::mutex?

[@FergusGriggs]

91%?

[@artyquantum7283]

They are already giving 90 % sale of Black Friday, I dont think they do any regard of the link you provided.

[@267praveen]

Hey Cherno Can you please elaborate a bit on using the std::future ...

[@bulentgercek]

​@@artyquantum7283 But they will give a share to Cherno if you click that link. Because it passes that cherno keyword in the url. That's how sponsorship works with the links.

[@jovanrakocevic9360]

Someone possibly mentioned it already, but you can pass by reference to threads, you just need to use std::ref()

[@1495978707]

What is the difference between this and a reference?

[@jovanrakocevic8263]

@@1495978707 Pointers and references have slightly different semantical meaning, namely a reference can never be refrencing nothing like nullptr does, therefore it's guaranteed to be "valid". Also you can't change what it points to once it's made. It's nothing you can't do with a pointer but it enforces it. If you mean why can't you just push as a reference instead of std::ref... I'm not sure. I think it has to do with references being uncopyable ( I guess this can be also worked around with std::move() if you are okay with the reference being possibly invalid). But this is fairly low level c++ arcane stuff and I'm not claiming I know what it is, it's just a guess.

[@w.mcnamara]

@@1495978707 std::thread requiring you to use std::ref instead of & is a deliberate design decision from the C++ committee to prevent you from shooting yourself in the foot when creating a thread. If you (for example) pass a local variable to a thread by reference, the variable may go out of scope before the thread uses it (because the thread executes in parallel), and then when that memory is accessed by reference in the newly created thread, it will result in undefined behaviour. Requiring people to use std::ref forces them to think about why they're passing by reference, (and if they're making an error, possibly realize it) and displays deliberate intent within the code.

[@user-qy2le2fo7r]

​@@w.mcnamara Perfect explanation !

[@mostafatajic6457]

​@@w.mcnamara Thank you for such a clear explanation!

[@igorswies5913]

Nobody: The Cherno: I need 19 Soviet trucks NOW

[@TopConductor]

6 seconds is to long to wait!

[@grdev3066]

Lol)

[@oleksiimoisieienko1430]

if you preallocate a vector with number of "meshes", you can pass index to the async function, and you don't need to have locks. Mutex and memory allocation are one of the top CPU eater, if you kill them performance will be increased a lot. Without preallocation of vector it will allocate more space few times during push_back. And with lock your algo will wait sometime until other thread will finish pushing back to vector. if you provide index to vector you can just run load_mesh without lock, and it willl be safe, because every thread will access unique index. Result should be much better.

[@maxmustermann3938]

In this specific example, this will barely gain you anything (even though I agree with you). What's taking time here is definitely not some small reallocation of a vector that ultimate just contains 19 References to Meshes, and those Mutexes are not gonna be noticeable when the actual tasks running in parallel are this "big" comparatively. If you have a lot of small tasks, then the points you mentioned become very important though.

[@TheOnlyJura]

You could increase the performance even more by increasing the vector's size and passing just one reference to a cell as an argument to the function. That way you don't need to reallocate all the time + you get rid of the mutex.

[@minneelyyyy8923]

that could work at first however if you end up extending this code and suddenly something somewhere needs to push to the vector you've just created ub. it's best to just never have a pointer to an element inside of a dynamic array. you could get rid of the reallocations by just setting the capacity and get rid of the mutex by returning the loaded mesh and then sort of collecting all of the results at the end

[@malgailany]

Parallel stacks view blew my mind. Didn't know such a feature exists! Thanks.

[@Ariccio123]

Trust me, it's made a huge difference for me in the past. Always have it open in a tab!

[@schoebey]

Right? I've been using VS for a long time (15+ years), but still discover new things about it. Thanks for that!

[@gonzalosanchez9191]

By far this is one of the best channel on youtube, thanks a lot for all this series.

[@vladuchoo]

Dude, that's a great stuff, please keep it up! Another suggestion would be multithreading + immutability, lock-free constructs.

[@op3003]

This How to finally showed me how to use async inside a loop. Did not find an answer on any of the 10 sites I've been on trying to find the answer. You rock Cherno!

[@Albert-lr7ky]

Great video! You have always been such a nice teacher. One thing to add, possibly someone added it already: if we want to wait to get the returned value of all async(), we need to call get() of the future objects, for example, in your code -> for (const auto& file : meshFilepaths) { m_Futures.emplace_back(std::async(std::launch::async, LoadMesh, &m_Meshes, file)); } /* * if we wish to get the result value and keep processing * we need to use get() of every future object */ for (auto& futureObject : m_Futures) { futureObject.get(); }

[@xxXSashaBardXxx]

perfect, as always! Thank you, Cherno.

[@Light-wz6tq]

Best C++ channel ever!

[@TheMR-777]

No doubt

[@Quancept]

100% right.

[@kingsnowy3037]

*ChiliTomatoNoodle* wants to know your location

[@abdosoliman]

dude, you literally saved my life I was about to multithreading c++98 style before finding this by coincidence. *Thank you* could make something about multiprocessor so far I didn't find anything to it in clean way in the standard library

[@bumbarabun]

Only problem is - this is naive and most probably not effective way to do multithreading.

[@HamzaDev]

Thank you so much, Cherno. I was trying to solve a problem with multithreading from the last 3 days, and your explanation resolved my problem. ❤

[@SriNiVi]

Amazingly precise presentation. Great job. Will be looking for your videos more.

[@guywithknife]

I just want to take a moment to contemplate the way he pronounced “cached”

[@StianF]

Ohh, I thought he said "caged"

[@ryanj.s9340]

Thank you for making this video I have wanted you to dive into multithreading a little deeper hope we can see some of this implemented in your game engine series rock on Cherno

[@commanderguyonyt0123]

The reason the why the reference doesn't work is, that the std::async function takes the arguments for the function by copying them. When calling the function, std::async has has an own copy of the meshes, so LoadMesh function takes the variable of the std::async function. TLDR: the std::async passes the arguments by copying them (in lambda [=])

[@connorhorman]

However, it will unwrap std::reference_wrapper into T&.

[@NomoregoodnamesD8]

call std::ref(x) and std::cref(x) to pass by reference in thread/async invocations

[@AvivCMusic]

I'm confused. std::async takes a "forwarding reference", aka Args&&. We pass into it an lvalue (the meshes), which turns the forwarding reference to an lvalue reference &. std::async will later take that reference, and pass it into the function passed in, which also takes a reference. Where does the copy happen?

[@NomoregoodnamesD8]

@@AvivCMusic since the function given to std::async may be run on a different thread, arguments are copied so that once you've called async, you may dispose of those arguments (assuming you pass by value rather than calling std::ref or std::cref, which instead basically passes by pointer)

[@AvivCMusic]

@@NomoregoodnamesD8 Oh, do you mean std::async explicitly makes a copy of the arguments, inside its body?

[@eddyecko94]

Thanks for this video! You’ve helped a lot. I’ve never considered financially supporting anyone but you might be the 1st.

[@ccflan]

Each time i see one of your videos i learn something new, thank you

[@XxxGuitarMadnessxxX]

I realize this is a two year old vid by now, but at ~18:30, you could also make the async function take an rvalue reference with '&&'. I'm not familiar with your 'Ref' there but I think it would then make that vector take a reference to those rvalue references passed in (maybe something similar like using std::forward(...)? I honestly have no idea, just guessing at this point lmao

[@sunthlower4812]

This was really cool! Thank you man :D

[@Ariccio123]

You missed out on one huge problem: std::async launches a new thread on *every* invocation. This burned me back in 2015 when msvc switched from the (better) threadpooling implementation to the standards compliant implementation. My program (altWinDirStat) can invoke upto millions of std::async tasks, so it hangs almost immediately at about 700 threads.

[@michmich6645]

Bruh

[@IExSet]

TBB looks better, and now we can use latches and barriers from C++20 ???

[@Energy0124HK]

Where are the alternatives of std::async then? Or how can we limit the amount of threads it creates?

[@JamesGriffinT]

@@Energy0124HK The alternative is to build your own thread pool tailored to your individual needs.

[@jhon3991]

@Alexander Riccio `std::async launches a new thread on every invocation`, how about g++?

[@coolumar335]

Mr. Chernikov, you are one of the best software/ game developers as well as teachers in existence right now. Thank you and kudos!

[@AntoniGawlikowski]

Best episode I've watched so far - real world example and not some artificial sandbox makes much more sense for me :)

[@davidm.johnston8994]

Great video, man, liked the practical exemple, even though I'm new.

[@jeffwonk2024]

Excellent explanation! Top quality.

[@azamjandali867]

Seriously great job man !

[@arthopacini]

Nice video, as always! One question though, instead of LoadMesh receives a pointer to meshes, could the async call pass a std::ref(LoadMesh) ? I know this need to be done when using std::thread and passing by reference, is this the case? I've never used std::async to be honest

[@dhikshith_reddy]

nice to see how the cherno grew during this series now vs in first video : )

[@MsAskop]

You talk a lot about performance and I really like your videos. Can you please make a video talking about data oriented design? Some people say that is the best approach to write faster code. I love you so much. Great job with this channel, cherno! ♥️

[@bhaskart488]

I hope you reach 500K by the end of this year... maybe million:)

[@pc5207]

That was cool,keep going on !!!

[@Sebanisu]

I use async for loading a lot of textures. For saving some files I use a detached thread. I'm creating a map editor and I had some batch ops that I didn't want to wait or lock waiting on the writing to the hard drive. This video was very helpful.

[@kylefillingim9658]

I like the video. I usually use c# myself, and have utilized the parallel for and parallel foreach loops numerous times as well as the thread safe data structures. Is there any chance you could do a video on utilizing the gpu to speed up math operations, say matrix multiplication? You are awesome. Keep making these videos

[@twhartonable]

Great video. Very clear and informative.

[@MelroyvandenBerg]

7:40 why do you first load it into a string vector, and later do a for-loop again :\? You can just put the Mesh::Load() in the while loop directly.

[@erwinschrodinger2320]

Great as always!

[@user-jd1zx]

this channel deserve much more subs

[@dimts2701]

Thank you for the video Cherno! Could you please make a video explaining all the different types of parallel computation (openmp, mpi, std::async...)?

[@karemegypt25]

Been waiting for that for so long..

[@ShivamJha00]

A 23min C++ video from Cherno. Now that's what I call a hell of a day

[@maltr2447]

Never really understood asynchronous programming, but that really helped, thank you!

[@changthunderwang7543]

You are such a good teacher dude

[@furball_vixie]

1st, wow i just woke up to this. perfect to watch while eating breakfast i guess

[@sudani1993]

hhhhhhhhhh Yes, and I also

[@akshaymathur2225]

Yeah me too..

[@acatisfinetoo3018]

Yes gonna need some sauce with my breakfast...

[@adamodimattia]

Same here :)

[@furball_vixie]

@GigaramsYT holy shit when did this get 109 likes. nice

[@hufor1996]

Thank you very much Man , That was super helpful 👍👍👍

[@secrus]

2 questions. 1. Have you tried CLion by JetBrains? Any opinion? 2. Is there any sense in using .h files instead of .hpp? With modern C++ we can’t really import it to C programs (cause only reason for using .h, that I can see, is to share C and C++ code) so whats the point?

[@filosofisto]

Very good explanation and sample

[@skrya1248]

Hostinger is the best! No. 1 support :D

[@lukassoo]

2 videos in less than 24 hours? Impossible... Cherno found a way to make videos faster too!

[@brakenthemole2377]

multithreaded video editing

[@guestimator121]

Even his hair looks like he was riding a bike 200miles per hour

[@sky_beast5129]

*Cherno.brain started using multithreading*

[@Mulla_killer]

Sponsors make people to achieve the impossible

[@suntzu1409]

Cherno overclocked himself

[@chris_burrows]

Best video yet. Thank you.

[@GrantSchulte]

Is your Mesh::Load(...) function loading the meshes to the GPU or just storing vertex data in RAM? How are you able to load the all the models at the same time without messing up the OpenGL states?

[@EfficientJava]

I'm having this exact issue

[@chris_burrows]

Does this work out the box for loading textures? or do you have to do some locking and juggling to prevent one texture calling glBindTexture() while the previous is still bound and loading?

[@TheCherno]

OpenGL can't really* be multi-threaded, so no.

[@MarkusLobedann]

And what about when using direct state access?

[@Amipotsophspond]

really good video, Thank you!

[@Energy0124HK]

What is that Ref class that he used in the video? At 11:57, std::vector. Did he explain it elsewhere?

[@alimehrvarz8391]

"Real-world" examples are great, thanks.

[@willinton06]

Well this just makes me appreciate the simplicity of C#’s concurrency model, not even looking at Go’s model which is somehow even simpler, all this could be done with a simple foreach, a Task list and and Task.AwaitAll

[@ColinBroderickMaths]

I have a long running process that I would like to do in this async style. How would you recommend managing the vector of futures you introduce? I guess that list needs to be emptied of completed calls?

[@therealgunny]

a resource system is probably the easiest part, what i'm really interested in is to see how you can multi-thread a rendering pipeline. btw, references are not real objects, so they cannot be copied nor assigned and they are constant by default so they have to be initialized at declaration, while pointers are real objects and can be copied, assigned and are not constant by default so you can rebind them during runtime and they can be nullable!

[@leixun]

*My takeaways:* 1. One of the difficulties of making things run in parallel is to figure out the dependencies 2:38 2. Check parallel stack graph 19:54

[@shifbar]

Hi Cherno, I love your videos about C++. They are very clear, helpfull and nice ;-) The only question that I have is to turn down the volume of the music in the background, becasue it's hard for me to get focused with the music :-(

[@DeathAtYourDoorStep]

what music bro? are you high

[@NinjaGhostWarrior123]

Cool video. So mutex and semaphore are used for the same principple right? Is one better than the other?

[@oozecandy]

Wow- I just googled the term "std::async" and got an answer by one of my favorite cpp explainers which is only a few days old :)

[@chrisparker9672]

Note that MSVC has a longstanding bug where `.valid()` on a future may erroneously return true even after calling `.get()`

[@bassbatterer]

at 15:00 shouldn't you then use a try catch statement aroung the pushback where the catch statement will unlock the mutex? So if the thread crashes before completing the resource doesn't stay locked.

[@salunkhesagar1386]

That's a masterpiece!!

[@fennecfox2366]

Great video. Just curious as to why the models aren't cached? Is this just for the sake of the example? Thanks.

[@TsArun-qw6xn]

What is 'Ref' at 12:04 std::vector is it cherno's internal class for this coding alone?

[@alexhirsch889]

Have you considered using (or talking about in your upcoming video) a singly-linked list? Herb Sutter has a genius talk ("Lock-Free Programming (or, Juggling Razor Blades)") in which he explains the multithreaded slist. Really cool stuff and as an slist would be very powerfull in the mesh example (as no lock would ever be required) you should give it a thought.

[@ArlenKeshabyan]

You don't need any mutex at all to do that. Just add a local std::vector "futures" variable. Iterate through file paths and add futures.push_back(std::async(std::launch::async, [](auto &&filepath){ return Mesh::Load(filepath); }), current_filepath); Right after that, just iterate through all futures you've just added and do m_Mesh.push_back(current_future.get()); That's it.

[@AlienFreak69]

Will you do a video on how to use the GPU for processing stuff? I'm talking about using the GPU for moving thousands of objects on the screen with each one having its own collision data. That kinda stuff is usually too taxing for the CPU.

[@DeathAtYourDoorStep]

we need dis

[@ayushprakash8343]

Your videos are really helpful !! thanks. Btw why don't you make videos related to data structures and algorithms?

[@DrGreenGiant]

If you're doing this embedded and using freeRTOS, for example, does this still work multicore or would you have to spawn a load of tasks?

[@broccoloodle]

Is it more readable to use lambda to pass in `std::async` instead of writing a whole new `static void` function?

[@TheCherno]

It might be more convenient to use a lambda, but I definitely wouldn't call it more ~readable~. The end result is the same but with an actual function we get to name it (hence more readability), as well as keep the scope of our std::async function less polluted. In practice, I personally use both; it depends.

[@achihabhalib7435]

Async approach of loading files is only benificial on an SSD right? Doing this on an HDD would make the magnetic head bounce around to perform seeks at different locations, slowing down the reading process.

[@enjinia8307]

Finally, we are getting into multithreading... Exciting!

[@urielcohen9251]

At 18:28 wouldn't wrapping the m_Meshes in std::ref() supply it as a reference?

[@xmfcx]

Yes, that's the right way of doing it.

[@MountainKing123]

Wouldn't you have to have another push_back function for your std::vector member and lock that one? Else multiple async LoadMesh calls could still try to push back returned std::future(s) concurrently. Or am I missing something?

[@Zorgosto]

Can't you use the std::for_each method in c++ together with the parallel_policy in std::execution to get a parallel for loop?

[@xxdeadmonkxx]

I believe you have to use std::ref to pass reference arguments to std::async

[@ccfrodo]

is there a particular reason that you roll your own transformation loop over using std::transform with a parallel execution policy? std::vector output; std::transform(std::execution::parallel_unsequenced_policy , begin(filenames), end(filenames), begin(output), [](std::string path)-> ref { ...; return ref;} );

[@aldrinaldrin4618]

The Great Cherno.. are you going to make one video for move semantics?

[@littlebearandchicken2077]

Can't wait for a The Cherno Rust channel

[@dirtInfestor]

If you want to use a reference instead of a pointer at 18:20 you have to use std::ref. Also is there any reason to use async instead of std::thread here? Besides showing us std::async of course

[@lengors1674]

I think it's because with async you can easily convert between multithreaded and singlethreaded. With async you can pass a function with a return value where with thread that is not possible, which means that if you have a function with a return value that you want to execute in parallel, with thread you'll need to convert it to a function with no return value and use some method to share data between that thread and the calling thread.

[@dirtInfestor]

@@lengors1674 Im aware of that, but since the futures here are not used, there seems to be no reason to use async over thread

[@lengors1674]

@@dirtInfestor yeah, you're right I misread your question

[@foxvulpes8245]

@@dirtInfestor I believe the way he is showing it makes it easier if you want to have return values. The real question is, whats the performance hit of thread vs async? For a journeyman coder like myself, asyc method was more useful to me.

[@shadowassasinsniper]

std::async is typically pooled and will be faster to kick off than std::thread (which forces a new thread to be created)

[@lucasgasparino6141]

OpenACC is also an option, as it allows use of GPUs without losing execution generality (if no GPU is detected it behaves like OpenMP). Or just go ballistic and use CUDA/OpenCL :)

[@sebastiangudino9377]

Why tho? GPU parallelism is not a magic solution to make everything faster just because. There is nothing wrong with loading models from the CPU. Actually, it's preferible. Since in a game or other RT Rendering situation the GPU is likely already busy actually rendering frames as it is supposed to (And using parallelism to do so anyways). Using CUDA to load some models in real time is beyond overkill

[@gvcallen]

Would using emplace_back() change the speed noticeably in this scenario? Just curious if the vector can also be seen as a bottlekneck since copying data of mesh structures might be time consuming?

[@TheBestNameEverMade]

Reading off a disk is 10s to 100s of thousands times slower than cache and it's a pointer copy so I won't see much of a difference.

[@450aday]

one real thumbs up for you sir, as apposed to all those fake ones I'm always handing out. good job!

[@ChrisM541]

13:29 The current thread being utilised will have one/more unique ID's - is it possible to have a unique ID sequentially allocated and use that as an index into your own custom vector table? If yes, that will do away with any requirement for mutex/locking.

[@TopConductor]

isn't it better not to copy mesh into mashes by doing push_back, but rather do push_back(move(mash))? Or am I wrong and copy elision will rull this out and automatically call move version of push_back?

[@greob]

I have enjoyed this episode.

[@rs4artha]

Your channel is awesome and you are awesome too...... Keep posting videos.........

[@nutinmyass]

Great video

[@maxmustermann3938]

I wonder what's happening inside of your Mesh class - are you creating the GPU buffers outside of it at a later point? Since when you're doing that multithreaded, you would need to do some extra work in whatever graphics API you are using, which might get complicated when you aren't really in control of the threads as they are part of some threadpool created by the standard library.

[@antonfernando8409]

Does the ::async work in Linux? sorry, I thought you need pthread_mutex lock and unlock for mutual exclusion in Linux, or is that OS level lock APIs wrapped inside the std::async.

[@toast_on_toast1270]

Love your videos and this one in particular, music's a tad loud at the start though :)

[@shushens]

C++ threads by design only accept objects by value. Trying to pass things by reference will also pass them by value. So, either pointer or std::ref is the way to go :)

[@alexanderpopescu5539]

Could I use multithreading to read a fasta file (DNA sequence) character by character ?

[@abbedidriss5189]

Are these threads being run on different cores or is a single core switching between each process?

[@LoSte26]

Starting from C++17, the standard library actually has a built-in parallel for loop, it's just simple as this: #include // needed for std::execution::par which enables parallel execution #include // needed to use the std::for_each algorithm ... std::for_each(std::execution::par, std::begin(meshFilepaths), std::end(meshFilepaths), [&m_Meshes](auto const& file) { std::lock_guard lock{s_MeshesMutex}; m_Meshes.push_back(Mesh::Load(file)); }); that's it; no need to sore manually the std::futures anymore and the algorithm itself tries to correctly balance the number of spawned threads... and it's almost identical to a sequential for_each loop, except the use of std::execution::par (that makes run the algo in parallel) and the use of the lock_guard/mutex. The lock_guard template parameter does not need to be specified (it's deduced in C++17) thanks to CTAD.

[@ycombinator765]

I am a noob but I don't know why I think Bocarra(fluentC++) author is right that C++ should not have deceptive anstractions. this one is exactly an example. What Cherno did is perfect verbosity mixed with elaborate simplicity. I am not arguing just meaning that some complexity should be in there. anyways thanks for this

[@attcs]

This is a good presentation about "How NOT to do". There are at least two major issues. 1, Usage itself. C++17: execution policies. This code can be written much simpler and with better performance without any mutex and the overhead of the countless thread: m_Meshes.resize(meshFilepaths.size()); transform(execution::par, meshFilepaths.begin(), meshFilepaths.end(), m_Meshes.begin(), [](auto const& file) { return Mesh::Load(file); }); // or probably par_unseq std::async can eat a function with return value, you should just call the future::get in a second for loop. The video solution bypasses the obvious way, hardly readable/maintainable, etc. There are a lot of non-standard solutions to parallelize the calculation, OpenMP 2.0 and Parallel Patterns Library (PPL) are built-in in Visual Studio. 2, Parallel File I/O could have major performance problems, especially if the files are not fit in the same hardrive cache (large files, fragmented data). In your example you read the same file from the same cache, obviously did not encounter this issue.

[@Kabodanki]

Would love to have a response to that. Having things popping up isn't great, to paraphrase people from DF Foundry, it's distracting. Mutex should be avoided when not necessary. Having lots of thread is not great (MHW on PC is a great exemple of that). I/O is a great problem as well (Just look at what Mark Cerny had to say about game sizes and issues they got with it, like copying multiple time the same asset (increasing in the process game sizes) so they can be loaded quickly -> "there are some pieces of data duplicated 400 times on the hard drive." )).

[@mohammedalyousef5072]

The compiler I’m using doesn’t support execution policies, it does support std::async. As far as I can tell, gcc has only recently added full support for it.