Path Tracing // Ray Tracing series

[@TheCherno]

Really getting exciting now! Who's keen for the next episode?? In the meantime, keep exploring at brilliant.org/TheCherno/ Get started for free, and hurry-the first 200 people get 20% off an annual premium subscription. AMAZING place to learn all the math you'll need for this series! ❤

Hi Cherno! So I am now 39 years old, started to code when I was 37. Thank you for all the stuff you have made, taught me a lot. I am now looking at this game engine project and it is a great way to learn how to make maintainable code, working sw that has any level of complexity. Thanks a lot man, great job.

[@varhar3296]

Id love to see if we can reuse past image when moving the camera. Maybe use the roughness=0 image, temporarily 'bake' to textures, compare changes and map last frames rendered textures? although it might be too hard to do.

[@nathans_codes]

yeah but can you push the code to the github repo?

[@Alkanen]

This series is brilliant (my god I'm so funny), can't wait for next episode! Do you have a release schedule planned for this series or is it more organically released?

Path Tracing at 7:33. You're welcome.

[@arsenbabaev1022]

making it faster would be fun, also might be a good practical way to teach multi threading.

[@torcher5023]

What multi-threading? Just use your GPU with such a graphics API like OpenGL or Vulkan

[@dickheadrecs]

gpu has a *lot* of “threads”

[@DeGuerre]

The terminology that I tend to use: - If you are using rays as a hidden surface technique (i.e. first-hit), that's "ray casting". - Direct-lighting shadows, specular reflections, and refraction is "Whitted-style ray tracing". - Motion blur, depth of field, glossy reflection, and in particular, any technique where adding more camera samples fixes convergence issues in the limit, is "distribution ray tracing". - Any technique that can ray trace multiple diffuse bounces, is "path tracing". - Anything that involves tracing ray adjoints from light sources is "photon tracing". There are obviously fuzzy boundaries here. Mathematically, you are trying to solve an integral equation. Distribution ray tracing is the point where randomness becomes important: you solve the equation by constructing a random variable, the expected value of which is the solution to the equation. In general, any technique that does this is "Monte Carlo ray tracing". Path tracing is, in a sense, distribution ray tracing where the equation you are solving is Kajiya's rendering equation.

[@CreativeOven]

This kind of video structure is better than other ones that you have made, An overview and then code......... It is easy to learn like this!

[@LeroyMustang]

As an illustrator and tattoo artist I have learned far more and far more clearly about light and reflections from the history of CG lighting in games specifically than any other source. Of course we get to exaggerate and simplify but it’s fantastic how much even a simple application of this sort of methodology makes a digital painting/canvas painting more interesting. People really don’t know why they can’t render things in a more realistic way because apart from specialized courses this theory or rough rules aren’t taught.

[@fischi9129]

hands down best course on ray tracing and/of rendering on youtube. It's noce going step by step incrementing complexitly slowly and just add a simple thing after an other :) Then 10 episodes in and you like... damn, that sounded much more difficult than it was to do it like this :)

[@suri4Musiq]

I watched this video a year back and now after a year - i am still learning path tracing vs ray tracing.. but you did a really good job, just the knowledge is really deep!

[@andrewpolychronakis849]

Excellent episode, Cherno. You explain pretty simply what path-tracing is without introducing global illumination, and I can wait for the next episode. I would propose for the future of the series that you teach about acceleration structures before moving to multithreading or the GPU due to current GPU architectures implementing acceleration structures as a black box by giving you some access due to the intersection shaders. 😃

[@MrFirelord]

I would also prefer to know more about ray tracing theory before rewriting app to GPU. Acceleration structures is a great candidate for the next video

[@sh_4016]

In ray casting you have a ray that will take one step each time and then will check if it hits something. In ray tracing you have a formula that will calculate the intersections for you. Also in ray casting you can implement shadows, soft shadows, reflections, global illumination and all the things ray tracing has, but it is slower than just some simple intersection formula

[@Happyhenzo]

No that’s ray marching

[@computersdontcompost]

These ray tracing series are super cool. Every next topic is exciting. If optimizations at this moment will make all furtherly added features run smoothly then I vote for optimizations.

[@davidcfrogley]

Shadows, soft shadows, area lights, triangles, transparency, maybe parsing an entire scene file. For scene files, especially the ability to embed mesh descriptions inside another scene file, but translated/rotated/scaled from the point of view of the containing scene (so we can use a single mesh definition in many different scenes but change the size, position, orientation, and material with data in the scene description).

[@Zyhorn]

Best video series, cant wait till the next epidode. Multithreading, shaders, optimizations and eventually move to the gpu. Amazing stuff and so fun to watch. Thank you

[@per-axelskogsberg3861]

Next episode seems like it's going to be awesome!

[@أسامةالعبدالله-ه5س]

I was trying to find multithreading videos from you and I still want it + thank you for everything

[@Carolina-mw4po]

That random path accumulation method is what Chaos Vantage actually does. Every time I move the camera or I perform any change on materials or whatever, it goes back to noisy and rapidly accumulates the random outputs, progressively cleaning the image. So Vantage is a true pathtracer (different from UE)

[@arthurmightybear6119]

Important to mention that im learning ray tracing for now only from your videos (im planning to read, yeah). But i'm interested how this gets enhanced so you don't loose image quality when moving stuff ? My best guess would be - brute force rays when moving for a few frames then chill)) It actually explains why some games get super laggy when you move camera 😅

[@Hadrhune0]

As far as i know, Raycasting is the primitive operation of shooting a ray from A to B for different purposes (for example player interaction, evaluate distances etc), and not for compute illumination. Ray tracing uses a lot of Raycasting but Raycasting is a general purpose technique.

[@naturallyinterested7569]

Are you doing gamma correction? Because I'm guessing ImGui::Image takes sRGB values and skimming the code I can't see any gamma code to turn your linear RGB to sRGB. Or is it maybe handled in Walnut::Image?

[@Howtheheckarehandleswit]

Honestly, I'd love either of those options! They both sound super fun.

[@gameguy301]

at the edge of my seat the whole time, its like a horror movie I keep looking through my fingers "is he going to scratch that table with that screwdriver"?

[@LinDahai88]

Fun idea: what if changee the random depending on the reflection angle? For example the bigger angle between normal and a ray the less random is. In theory it should make objects edges more precise but still have more blurry picture when the ray is almost perpendicular to surface.

[@Alkanen]

That wouldn't be physically plausible though. The randomness is simply small imperfections in the surface. Those imperfections aren't increased by the reflection angle. The randomness should be changed to being within a unit sphere rather than a unit cube though.

[@TheNerd]

If I remember correctly Nvidia has a couple Videos about the differences between raytracing and pathtracing. An Nvidia Engineer explained it, but I saw it like 3 years ago when I started using Blender Edit: It's called "Ray Tracing Essentials" its a multipart series. he explain the differences in the first video towards the end

[@Carolina-mw4po]

Thank you so much for this!!! I finally understood pathtracing, and its coding implementation. Just awesome!

[@TheGubble]

Quick question: as you continually add to m_AccumulationData and then only divide by the frame index when finding accumulatedColor, do you run the risk of losing some accuracy as the float grows (limited precision for large numbers) and eventually hitting inf/overflow (given enough time)? Would a rolling average be a better approach (clamping m_FrameIndex to a maximum number, and storing the average back into m_AccumulationData each time)?

[@Alkanen]

Eventually, yes, but since he's adding values between 0 and 1 and doubles can handle something like 17 digits of precision and we care about precision to about 0.004 (1/256) that means we can continue to a sum of about 1e14 before we lose information (unless I've calculated like an idiot, a fairly high possibility). That means he should be able to accumulate data for some 52 000 years at 60 fps.

[@Alkanen]

If he's using single precision floats to accumulate values (sorry, I didn't pay enough attention when I watched the video, will have to rewatch later), that's about 7 digits of precision which is only enough for a couple of minutes and then you're absolutely right that it might be a problem.

[@Alkanen]

I did some tests to see if it's actually a problem and with a loop up to 10,000,000 frames with addition of 1/255.0 and divided by frame number, there was some jittering to be sure, but it was less than 1 % of the expected value. It ended up as 0.003907, and should have been 0.0039215[...]

[@calabrais]

This is such great info and you are an amazing teacher! I don't know why this doesn't have more views.

[@Jurkox26]

Hey Cherno. I've been waiting for path tracing to appear here. Finally! BTW regarding maths and Brilliant. Do you sometimes forget, or maybe don't get familiar with topics right away, so you have to learn them using that app?

[@mr.mirror1213]

these series are too good , would like to see optimization!

[@ricgreen1]

Thank you for this informative video! Hope you keep at it!

[@goshisanniichi]

Right now you're treating each pixel as a point and shooting each initial ray through the same point every frame. If instead, you were to treat each pixel as a square and shoot the initial rays through a random point within each pixel thereby using rays that still go through the pixel but in a very slightly different direction you basically get 'easy' anti-aliasing. It doesn't have to be random though, I usually split the pixel up evenly and shoot at least four rays per pixel to a nice effect. That said, if the images you're rendering are high-resolution (I'd say 1080p or greater) anti-aliasing isn't really necessary.

[@erikrl2]

This is so exciting! Thank you for sharing your knowledge!

[@AliceErishech]

How would it look if you took the roughness as a weighted average between fully reflective and not reflective? I've never messed with programming 3d graphics myself so I'm not sure whether there'd be a flaw with this approach or not but it seems like it'd be a fairly simple and straightforward one.

[@Zombieanfuehrer]

Hello @The Cherno, I am a big fan of your C++ videos, which have helped me many times - I am now a professional software developer for C/C++ systems (QNX). You have made several videos about multithreading a long time ago. What do you think about making a big video or a whole "small" series about this topic. In this you could introduce the whole different and classes? Maybe also the whole control and synchronization mechanisms via cond_var and mutex ? I think it's great that you can always learn something new in C++ or a new/different way to realize something, recently I came across packages_task and so I also come to the request

[@Kazyek]

Wouldn't you want for the "random" ray to use some kind of uniform distribution which never return the same value (well, until it loop eventually), and keep the same seed until the accumulation is reseted? Like, the current random is alrealy some function which for any X seed return a sequence of values respecting a normal distribution. We'd just want to A: have a sequence of non-repeating values (which I don't think the current random function do, although duplicates are statistically unlikely regardless) and B: keep the same sequence over the accumulation. Then the seed can be reset when resetting the accumulation. You'd also ideally want a distribution that quickly cover the range of possible vector output; not necessarly the value from the distribution, but rather the end vector within the cone, you want to "cover" the most area within the cone at any given time, so, not having a bunch of them on one side but rather that for any subsequent vector, it is picked within the largest uncovered sub-area of the cone. Not sure if I wrote it in an understandable way, I suck at math and don't have the terminology, just speaking based on observations 😅

[@Alkanen]

You probably want to do some kind of stratified sampling eventually, where you make sure the samples are spread out evenly in the pixel (which uniform distribution isn't in any way guaranteed to do, it tends to cluster a lot).

[@anonymoussloth6687]

I think you should address all the complex topics, but also start shifting towards doing them on the GPU instead of cpu

[@splitupgamesofficial]

Good idea

[@ZeroUm_]

I'm a bit torn between going smart implementing higher quality algorithms, or brute forcing with CPU multithreading and eventually GPU. I feel the latter could mask bad algorithms, but with the former you might get too clever for newbies to understand what's going on. It's quite a balancing act when you're teaching stuff.

[@taojiang2735]

how could say brilliant without smile, loll, I cannot help laughing

[@jamestuppenney8235]

Could you use a seeded random number or something to get a correct image without needing to send as many rays 🤔

[@Morphexe]

A seed you pick might work great for a specific scene, point, ray, but wont absolulety work for all the scenarios . Thats why you need to go clever with your sampling/rays :)

[@ThatsMySkill]

(just talking about ray/path tracing in games here) i was always interested in video game graphics so when i first learned about ray tracing, it was always described to me as being another word for path tracing so i thought those terms were used interchangably. now that path tracing seems to be another form of ray tracing that combines everything into one render (shadows, reflections and global illumination) i was very confused cause i thought "if we had ray tracing before and its supposed to be path tracing, then what the hell is path tracing really?". admittedly im not deep into the matter so i was really confused. so just as ray tracing was the buzz word for all the rtx implementations so far, path tracing is gonna be the new buzz word for doing all those things in one render i guess. instead of splitting all those things up its gonna be mimicking real lighting and for example what would have been a sole global illumination path can now bounce from lets say gi to reflection. i might be understanding something wrong. like i know how ray/path tracing works but is this just seems like its another marketing thing to make it sound different from ray tracing. otherwise they would have to call it "real ray tracing" or something like that.

[@bishboria]

Looking forward to the gpu performance boost!

[@Alkanen]

Me too! I just hope it doesn't require an RTX card, but that any Nvidia card will work (or any card at all, would be even better)

[@farhadaa]

We call it raytracing because the camera is tracing the ray of light back towards the lightsource to calculate whether or not an object is lit by that lightsource. Pathtracing however seeks to trace a path of light from the light source that bounces many times until it reaches the camera, this is more expensive in computation, which is why ray tracing was created to reach desirable results with less computational costs, albeit losing the great effect of global illumination in the process.

[@powerclan1910]

i would like to see both your suggestions. Meaning you implement features untill it gets too slow (like now) and then we do an optimization episode.

[@UAVXP]

Waiting for the next episode!

[@lmoose7341]

When will you explore UE source code?😃😃

[@zennmyst5347]

I think optimizing is the way to go first. Then multi thread after.

[@squelchedotter]

If you jiggled the camera by less than a pixel every frame, you'd also get "free" anti-aliasing, right? Might be neat to add.

[@3DProgramming]

Amazing video as always.

[@icesentry]

@The Cherno by the way, the code hasn't been uploaded to github

[@idk7-c7s]

You should proberbly transfer the raytracing code to a compute shader so that the gpu will do everythinh instead!

[@ricbattaglia6976]

Is better Twinmotion pathtracing or D5 D5GI engine render? And faster? Is it enough a rtx 4090 or is necessary a multicores cpu too? Thanks!

[@vectoralphaSec]

The Cherno do you have any idea about how many episodes this series might end up being??

[@sshawarma]

Great video! Could someone explain how this would work with dynamic objects in the scene though? I can't imagine you would want to throw out the accumulation buffer if any object moved.

[@terryrussell8527]

The jagged edges of the spheres can be anti-aliased by applying a little jitter to the initial ray at the start of the PerPixel call. glm::vec4 Renderer::PerPixel(uint32_t x, uint32_t y) { Ray ray; ray.Origin = m_ActiveCamera->GetPosition(); ray.Direction = m_ActiveCamera->GetRayDirections()[x + y * m_FinalImage->GetWidth()]; ray.Direction += Walnut::Random::Vec3(-0.001f, 0.001f); ...

[@andreanderson626]

What would would it take to optimize the code enough to get 60 fps on your laptop? Is it only possible with the GPU?

[@dmitrimozgovenko810]

Do you know if Brilliant offer courses with video or only in text (so people have to read)?

[@Kley96]

I don't know anything about this. But I'm stuck in this Accumulation thing. Is that the same thing what Ray Reconstruction by nVidia is doing as a Denoiser?

[@MLBB_CHILL]

You did not push the episode on github

[@igorlevdansky4545]

Cherno, what kinda monitor do you got? Dell?

[@Basel-ll8fj]

clean and understandable 👌👌

[@codingman313]

How about refraction, with reflection and albedo it will be kind-of-a wholesome.

[@Morphexe]

refraction is just a matter of sending the ray, and when you hit a transparent refractive surface, you just "shift" the ray based on a factor and keep tracing it, so intead of going through the object it gets displaced. There is "nothing else" on that :)

[@badgraphix9659]

For me it already looked smooth bc of youtube compression lol

[@hsavietto]

I got a Brilliant ad right of the start of the Brilliant sponsor segment... 🤣

[@maraisr]

You know its a Cherno video when 90% of the explanation is what a "path" is, and 10% of what the code is doing.

[@NathanHedglin]

Great thanks!

[@11superjump]

[@echo000]

Insightful!

[@godofpro5767]

can you push the code to github ?

[@ModerateDev]

Hey cherno, did you see that Nvidia open sourced PhysX?

[@mikk5428]

You should really optimize it before doing anything else. Kinda takes away the fun when doing some fancy graphics with that fps.

[@TopCurls]

i just posted a question in your discord about distributed ray tracing, i hope youll take the time to check it out

[@SXpitbull]

Can't follow along if our systems doesn't support vulkan ?

[@codingman313]

Use stb_image_write.h It is a very simple to use single file header-only library that lets you save a image from a buffer. Or just convert the image data into a texture and use it on a full screen quad.

[@anon_y_mousse]

Definitely needs more optimization. You've already kind of started by moving that one-time-useful test out of the loop. It's never good for performance to have tests that are only useful one time inside of a loop that runs thousands or millions of times.

[@Jkauppa]

recursive raster global illumination

[@Jkauppa]

render multiple parts of a full screen with multiple apis/gpu (any number of gpus) inside a game/application, SLI without SLI, even without using multiple displays, think 4x 1080p to 4K and above

[@Jkauppa]

one (game) application hooking up to multiple gpu's in a system at once NOT using SLI/Crossfire, just manual split the display resolution, such as 8K to 4x 4K for each GPU

[@Jkauppa]

360 degree fisheye camera raster maf

[@Jkauppa]

try isometric whack-a-mole pseudo-3d game in the 2d hazel engine

[@Jkauppa]

6x 90 degrees pyramid camera cube mapping lighting, kinda 360 raster from a point

[@naylinnfrautschmied9438]

GPUs can super boost rasterization, but ray tracing, not so much.

[@maxi-g]

nah, let's get into more graphics features fist

[@s187v]

realyl cool

[@ujox3720]

so path tracing = accumulation?

[@RogyDev]

:D

[@sh0kushu-kun946]

ඞ

[@aleksanderaksenov1363]

If to be honest Cherno's videos are rather messy from the point of view of explaining the basics and especially mathematics

[@terryrussell8527]

That accumulation buffer trick is pretty nice, though it doesn't like animated scenery. (My balls are bouncing.) Instead of accumulating color across all frames, I take a ratio of the previous frame's color with the current. Like so: vec3 color = renderer.PerPixel(x, y); vec3 accumColor = renderer.FetchAccumColor(x, y); if (frameCount > 0) color = 0.125f * color + 0.875f * accumColor; // basically an 8 frame blend renderer.SetAccumColor(x, y, color); // save for next pass