Ray Marching, and making 3D Worlds with Math

[@simondev758]

Patrons can now vote for the next video! Thank you for your support. ❤ Support me on Patreon: www.patreon.com/simondevyt 🌍 Live Demo + Courses: simondev.io

[@stickguy9109]

For the first time I understood what this magic "ray marching" is and how it works. You are good at explaining things

[@johanngambolputty5351]

Even if I ever understood before, this might be the time it actually sticks :) for some reason I always mixed it up with tracing, or maybe something to do with section searches

[@klontjespap]

​@@johanngambolputty5351 well technically you're still tracing rays, right? certainly it's ray casting, the term they used in 3d games back in the 90s to collision check your projectile trajectories and the such, which is just 1 ray. [ray tracing] is a routine that casts a ray for every angle your "caster" can "see". in 3d rendering often a camera, so the camera settings dictate the position and the angles the caster limits itself to, the pixels it traces are projected into angles depending on FOV/perspective values from the camera, they ARE the set/array of angles to process basically. it's still raycasting too, raycasting a lot in fact, raycasting is still the integral part that does the work. (a 1080p image is about 2 million pixels, so ~2 million rays if just 1 ray for every pixel, modern path tracing do many many more per pixel) there's also bounces for light ofcourse and the such, but for the sake of simplicity let's keep it at 1 bounce. [path tracing] is a more complex raytracing that can also do reverse tracing for caustics and the such, has way more rays for more modern PBR rendering methods, z distance information feedback for post processing and the such, essentially raytracing on crack. [ray marching] i consider it to be a variant/derivate of ray tracing, i'm not sure how disputed that is, but i'd put it under that moniker. but it is smart/dynamic, rather than checking everything, it makes bigger jumps, and you can reduce the amount of cycles by a great deal just working with minimum lenghts set on a higher value , but very small object intersections aren't "seen" because the ray jumps over them, this disadvantage is exploited as an advantage in the demos in this video by interpolating and blending the missing parts together the "looking over it" principle is also used in metaballs, but metaballs calculate a lot of shit on the way, this method actually drops data and fills in the gaps, it has a bit less control, but you could put extra data in vectors and use those with proximity for all kinds of effects too i don't think the vector lengths (jump distances) strictly have to be dynamic for it to be ray marching, they could be static and just have it jump the same distance every time as a crude version until it hits something it's not ray marching if you only shoot 1 ray either, so it's just grid-snapped ray casting strictly taken, but the distributed is implied although there may be differentiation in terminology these days perhaps, but that was also understood as ray marching back in the day ray marching is a de facto collision detection algorithm, as is ray tracing, because they're all raycasting and the rays are looking for something to bounce on, that precisely is what collision detection is, finding an intersection. as with many code, raymarching differs a lot between CPU based (serial functional programming) and GPU based (parallel logical programming), they need a quite different approach. the crude version of ray marching basically is raytracing, but the ray casting it does, is snapped to a grid with the grid's rotation relative to the individual ray's angle so it always traces over a striaght line (which is exactly what "local coordinates" are) short story long: i consider raymarching a variant of raytracing, i'd define them as: "distributed raycasting algorithms" when in doubt, use the term ray casting, that's the underlying principle for all of them, including ray casting itself obviously, so you can never be wrong then ;) even simple pixel-based detection is essentially just crude ray casting with the vectors being snapped to integer values, using global coordinates, and dropping all the benefits of unit vectors, when you think about it, right? lol i just had an epiphany, never even thought of it about like that myself even.

[@johanngambolputty5351]

@@klontjespap True, its a clever sort of tracing that you can do if you use distance fields instead of polygons, especially since the step size is not just dynamic but optimal (largest while guaranteed not to hit anything). I'm not familiar with this stuff, so I just imagined constant step sizes for tracing (against polygon primitives)... though I'm sure it can get way more clever

[@Chribit]

I‘ve actually been working on a graphics engine using signed distance fields as primitives instead of triangles for ~2 years now - super hyped to see it become more and more mainstream recently :D

[@keyb]

Ooooh sounds cool! Do you have any demos or code bases you can share?

[@Chribit]

@@keyb not yet unfortunately. The first year was mainly reading papers, understanding the heaps of theory and finding a sensible way of actually making it performant. There are bunch of challenges i wanted / had to solve first: - how do i make dynamic scenes with SDFs instead of scenes hardcoded into a shader? - how do i avoid as many branches as possible? - how do i completely avoid the default render pipeline? - how would instancing work? - how can i drastically improve performance? - etc. And the second year was mostly occupied by learning vulkan tbh. But i‘ve been making some major strides recently and i‘m hoping to finish a 2D version before end of the year. That will allow me to make some tools to drastically accelerate my dev process and it will all snowball from there. End-goal is a full 3d engine for real time rendering :D I can bookmark this video and let you know in this thread when something showable is abailable if you want? :)

[@simondev758]

Super cool! Got a demo I can watch of it in action?

[@Chribit]

@@simondev758 not yet unfortunately :D See comment above yours hahaha I‘d be happy to send you a demo as soon as something showable is ready, hopefully near the end of this year :)

[@keyb]

​@@Chribit Yes please! It sounds like a ton of work (especially in the sense of optimizations) so you still probably have a while to go. Whenever you do get a working model please let me know!

[@cucumberedpickle]

I've seen a lot of videos about raymarching before, but this is the first time I've "gotten it". The graphics you used to explain it make so much sense.

[@BalintCsala]

Great video, but small correction on a common misconception: Ray marching is just doing raytracing but numerically (so instead of having an equation tell you how far you need to go, you just go in small steps and check if you are inside a shape at any point), it doesn't necessarily involve SDFs (e.g. screenspace reflections also use raymarching, but not SDFs). The special name for this is sphere tracing.

[@stdcall]

yeah, the naive ray marching function steps a point towards the target and returns when it ends up inside an entity/object

[@diodin8587]

Do you mean sphere tracing is just ray marching with SDFs?

[@BalintCsala]

​@@diodin8587basically, if you open up the wikipedia page of ray marching there's a section on it.

[@magnusm4]

I remember this blend mechanic being used in a Valve tech demo from around 2003 where they made blobs able to combine and melt together like here. Later used years later for Portal 2. Insane they came up with this already back then.

[@TheJeremyKentBGross]

I believe they used a 2D version for text on signs in Team Fortress 2. I vaguely recall reading about it back in the 2ks.

[@Matkins85]

Nice video! I love playing with SDFs too. One point I think is worth mentioning, which I'm sure you understand but perhaps some viewers won't. It's a misconception that using modulo to repeat the space, and get seemingly infinite copies of your SDF, is for free. From a performance point of view its far from free. It's because then more rays are getting slowed down, and more frequently so, by passing near to surfaces. It is a neat trick though, and if your SDF is fast anyway then its not an issue.

[@simondev758]

Agreed, the performance cost is definitely real. I was mostly thinking in terms of implementation cost for a lot of these effects, but yeah the ray marching loop can't early out as easily.

[@gtnbssn]

I have started the glsl course a couple weeks back and am learning so much! Thank you!!

[@simondev758]

Awesome!

[@lordlightspeed]

for smoothmin(a, b, k), using smoothmax(a, b, -k) also works and is possibly slightly easier to write in code.

[@Shamysoza92]

Now THIS is how you sell a course. Great content!

[@aquanton]

this was the single coolest video I've seen on KZbin for a while

[@RobinPayot]

Amazing video, well done on the explanations! The 3D scene really helps to understand, maybe explaining a bit more the code part would be even better

[@simondev758]

Yeah, I think if I were to redo it, I'd add some animations around computing things like the normal/shadows in the same way that I did for the raymarching. Next time!

[@uquantum]

Wonderful Simon, came here from another one of your shader vids, mind-blown that you made the actual video using ray-marching, now picking jaw up from table

[@ArtisanAspirationWorks]

I was really envisioning this for an attack in my 3d plaforming game as an obstacle. Thx

[@guaje]

It's great to see more content like this. Loved your explanation of the central differences method.

[@williamjuicebutter6648]

This is the best content on youtube! Thank you!

[@Madlion]

This is quite cool, ive added these in Unreal engine and u can even blend with the mesh generated SDF

[@Veptis]

I am learning shader programming aa it's really interesting. And while learning abour Shadertoy and their implementation of multipass buffers - I came up with an idea for a very naive physics simulation over night and managed to implement it the following day. But now I am stuck for over a week to do a squish kinda motion between a soft and hard surface. Think like a bouncy ball. But the more I try and sketch - I don't think you can do such a squish motjon just based on two distance fields. It would be easier with two shapes to blend between and the center positions.

[@akfamili279]

I'm your old fan, thanks for updating the program so honestly! You are a hero to me! Yoom view bot king! Definitely the best in the business

[@nordie92]

Nice stuff, nice explanation, nice voice. Love it

[@Rennu_the_linux_guy]

CodeParade's marble marcher uses ray marching, if you haven't heard of it, it's really cool, you roll a marble along a fractal which is sometimes morphing around in real time

[@awesomecronk7183]

I like the explanation! I'm working on my own ray marching based graphics engine right now and, while your demo here is far more polished, I'm getting places.

[@icimojojojo]

That's an amazing video Simon, thanks a lot for sharing your knowledges with us !

[@somerandomidk7091]

I just coded my first ray marcher and I found it way easier than a raytracer, plus you can do a lot more amazing stuff.

[@UrodCyka]

Spore-like games would benefit heavily with ray marching, its always something ive thought about. you could build a cell and have the individual cell parts combine using ray marching or make cellular animations of two spheres dividing or even have creature parts be put together using ray marching. Ray marching gives endless opportunities

[@Luka116_]

I love your editing & teaching style! Thank you for making such informative and fun content for free ^_^

[@simoniacdemoniac7525]

I purchased your course. You really explain things well!

[@zxuiji]

3:07, That wire frame cube is your clue to doing super cheap 2d ray tracing on 3d environments, and you only need to do it on 1 side, the rest just need that one side's ray positions to be multiplied against either 1 or -1

[@cl10k]

Your vids are so incredibly good! I always get excited when a new one comes out. Pure gold!

[@SuboptimalEng]

This is awesome! The first thing I’m gonna do (once I get a job) is buy your shader course!

[@andermium]

I love my recommended, what an amazing video! I feel like there's a good chance this would work well with Geometric Algebra, but I don't know enough of either subjects to really say something about it.

[@TruthAndLoyalty]

You can't help but love listening to Bob from Bob's Burgers explain programming concepts.

[@ShiroiAkumaSama]

I won't ever try to write actual code, yet this video was still very interesting and understandable and it might teach me what I can use smooth min and max for in Blender.

[@AuroraLex]

In VRChat there is a famous world called Treehouse in the shade, that uses ray marching. Its really beautiful!

[@ngoc-64mtcothi53]

Don't give up mate, that was my first day to use soft soft and i will work on it for a long ti!

[@BobsiTutorial]

Really great video! Might implement this into my devlog game somehow 🙂 Great explaining and awesome video and visuals.

[@simondev758]

Let me know if you do!

[@kilroy987]

When I look at a subject crooked with a thinking face, I know I've learned something.

[@tomg0]

An awesome, well-made video. Love it

[@AntonioCorrenti]

Amazing stuff! Now I'm curious about how SDFGI works!

[@simondev758]

Me too!

[@AntonioCorrenti]

@@simondev758 just found this kzbin.info/www/bejne/d4PPk6uOradki7M

[@younghsiang2509]

This is really a cool video, the way objects transition between frames are just amazing! Wondering how that could be made.

[@DemsW]

Great video and explanations/visuals, makes me want to mess around with it. Would be nice to talk about the limitations and why it's not as popular as the usual rendering techniques Maybe a game like portal that does weird things with visuals could benefit from this.

[@simondev758]

Dreams on PS4/PS5 uses this technique extensively.

[@autbo]

The game called "Dreams" on the PS4 works exactly like this, except it has a "flake" effect to the surface to give it a painterly look.

[@boogeyman8099]

I would love to see an interactive VR experience that delves into ray matching.

[@KillFrenzy96]

There's worlds in VRChat featuring Raymarching shaders that you can experience in VR. It's quite fascinating. One of the worlds was called "Treehouse In The Shade".

[@boogeyman8099]

@@KillFrenzy96 thank you so much!

[@VoidloniXaarii]

Wow this was so interesting! You also mentioned csg ❤

[@Mcbuzzerr]

As soon as we went 3D everything started going over my head 😅 Fun visuals tho :)

[@filipathtc]

This is a really good explanation!

[@mexicanmax227]

I love you Simon. You’ve turned me into a smartass. Thank you 😃

[@simondev758]

Hah

[@hanneshinrichs]

Basically this is what "Dreams" for the PS4/PS5 is made of

[@vampirejs758]

Question: Can the minecraft clone you made be remade completely in glsl? Like the terrain generatjon and stuff

[@simondev758]

I think it could, but that'd be a major challenge. Maybe for a day when I'm super bored heh

[@fabiorota9661]

2:43 "it's really that simple" Meanwhile me having to pause the video for 10 minuts in order to understand what was going on and why it even worked -_-

[@ThachDo]

Thanks for the intuitive video. May I know how to determine the necessary number of points to render a distance field of a scene?

[@simondev758]

Most implementations set a max # of iterations, usually about 100 or so get's you a good result, a lot less depending on the scene.

[@ThachDo]

@@simondev758 at each point/iteration, we have a different set of scanning angles around the point, right? So how about the scanning resolution per sample point usually?

[@TunaFunDev]

Its all great but it wouldnt be imposible to have collision working on these shapes right?

[@Rene-uz3eb]

Great video, thanks. Made me realize that I need a distance function

[@SaintMatthieuSimard]

That's actually perturbing to watch. Feels like an old Outer Limits episode! :P You adjust the horizontals, the verticals, you're a wizard! 📺

[@simondev758]

I loved that show!

[@SaintMatthieuSimard]

@@simondev758 Hi, while you're around, I have a question, I'm a noob, can I use that method to project a simple parallax from an animated stripe and its animated mist pass? 2 mkv files, basically.

[@simondev758]

@@SaintMatthieuSimard I'm not super sure what you're asking, you can explain again

[@SaintMatthieuSimard]

​@@simondev758 What I'm trying to achieve is some kind of a deep picture frame that contains an animation which contains its own mist pass, for scene depth. I imagine the infinite possibilities of raymarching... Imagine if it was possible with a series of pictures containing their own depth map or estimated depth map, and making a snow globe out of it with accurate geometry from all angles, in Blender... That would be so great! Yet right now I'm really just after a method to create detailed pseudo-geometry off of low poly models. And the lowest factor is that I'm trying to cast a 60 frames scene into a cube, a picture frame, which contains depth information, and have an accurate representation of depth, without extra geometry nor multi-res modifier. Oh, is that even Blender that you used? I think you mentioned several other software, I'll rewatch just now.

[@SaintMatthieuSimard]

@@simondev758 Yeah I realized on 2nd watch that's not python nor blender :P I heard that with vector displacement mapping, certain textures can create creases in all directions and not just X and Y. I saw and old model that would create the geometry of an ear, then a nose, just out of a single texture. Tho I still haven't really figured it out. Learning that would clearly help me achieve what I've been trying to do for a while which is making a deep picture frame containing an animated scene rendered from pre-rendered material. An MKV file for depth, one for color diffusion, one for light, one for reflection... Yeah... That seem so far away for me right now x) For such a simple artistic project.

[@WistrelChianti]

I think my brain fell over half way on this one

[@brennonwilliams9181]

Watched it twice. Need a lay down and a cup of tea now. Simon, are these transitions, intersections, adds, subs being performed purely in the shader?

[@simondev758]

100% done in shaders yes

[@AntonioNoack]

yes, they are. Inigo Quilez has excellent tutorials about it. You can see hundreds, if not thousands of samples of SDFs on ShaderToy. The platform was created by him partially as well.

[@brennonwilliams9181]

@@simondev758 think you just sold me on the course. Awesome.

[@brennonwilliams9181]

@@AntonioNoack thanks! I will check it out

[@simonhatch4628]

@@brennonwilliams9181 Heh awesome. And yes, definitely check out Inigo's site, it's in the description, amazing resource for ray marching & sdf's.

[@Mencarifaktah]

because in the intro for example I want to make an acoustic soft, but later I want to make a distortion or any other effects in that sa

[@fonesrphunny7242]

I was always wondering about some effects in Inque - Ooze. Now I think they used a lot of ray marching to make the visuals in that demo. Coding is not my strong suit, but I'll have to if this can be integrated into Tooll3.

[@nutzeeer]

This is really cool! A lot can be imagined with math

[@Skeffles]

Fascinating, especially when the shapes merge into each other.

[@codytyson8460]

The videos only get better

[@Tumbolisu]

Can transparent objects be rendered with this technique? Also, I notice that during the showcase of the intersection operation at 5:10, shadows are visible when the object is not.

[@simondev758]

Yeah, especially with the smooth min & soft shadows, there's some leakage of light/shadow.

[@mayank9rana]

fire video, thanks bro

[@preludelight]

Hey Simon, fantastic explanations and excellent visuals to go along with! You mentioned at the end that the code is up on your github, but I wasn't able to find it. Am I looking in the wrong spot? Cheers!

[@simondev758]

I probably forgot to upload it, lemme go find the code and do that.

[@weisbrja]

@@simondev758Not to come off as rude, but did you upload it yet? I also couldn't find it and I'd be super interested...

[@preludelight]

@@simondev758 Just casually dropping by to remind you to upload this code. Cheers :)

[@Pavel-wj7gy]

In general, isn't ray marching suboptimal to csg in terms of performance?

[@ScryGL]

Wonderful! I know how to do lighting/reflections/combining geometry but I think my code quality could use help for putting together larger more complex scenes. I'll take a look at your course :)

[@arashvahabpour942]

Such a neat video!

[@freakfreak786]

Great video and explaination. I got a question tho: Did you hear about webGPU? Are you planing on learning/using it?

[@simondev758]

Definitely planning on playing with it

[@PotatoTheProgrammer]

if i had a game that uses only primitive shapes with basic png textures, would ray marching be faster than regular triangle rendering? also is giving ray marched objects textures even possible? also is transparency possible?

[@nikbivation]

wow very good explanation, made it easy!!

[@rafaelsantiagoaltoe6606]

phew. That is a cool thing, I will have a deeper look later. Thanks

[@alireza4734]

Every single sentence is worth writing down on the notebook. Thanks

[@Joao-uj9km]

Wow! Thanks for the video :)

[@Amir44312]

Are the rays send by the pixels on the screen or its just a virtuell ray that it sends

[@nyghl]

Hi Simon! I love your videos, the topics, the way you explain things, the graphics and the general flow of the video as well. The only thing I can recommend is a great microphone or improved sound quality for your voice! With all those high quality things in your videos, imo the sound quality is the only thing that is distracting at the moment. Much loves and please keep what you are doing

[@simondev758]

Thanks, I think I need to find a better recording location too heh

[@nyghl]

@@simondev758 Yeah, that could work as well 😅

[@qualle987123]

Very cool video. I would have love to have it a bit longer and more in depth though.

[@sinantk4036]

you record with hardware outside of the program. Great tutorial btw it was very detailed but still just right for beginners.

[@kalpolproductions8558]

Best video ive seen today. yosh

[@CharlesVanNoland]

Been waiting for someone to make a game engine that renders everything using SDF marching for a decade now. I know that evaluating many combined distance functions merged together to model something can get expensive, but utilizing simple bounding boxes and things to use as proxy geometry for the initial ray march origin for each object should help speed things up. I also think that extremely complex stuff could programmatically be reduced down into fewer distance functions that approximate it - particularly for an LOD type scheme. I don't know about an entire world being described using distance functions, maybe, or maybe they could be compiled down into static 3D textures that are raymarched instead - which likely would be faster than evaluating tons of distance functions every ray step for every pixel for every frame. I just feel like now's the time that this sort of thing could be realized if someone just sat down and actually pursued it.

[@simondev758]

100% I feel like as gpu's get more powerful and general purpose, it's opening up some interesting avenues that previously weren't feasible.

[@mimisaiko]

There's a software called MagicaCSG that's so easy and intuitive to use to make 3D models.

[@xueminchi1444]

Can I ask you a question？ The direction will become wrong when it's close to a concave object, like left and right are both closest point and the direction point to the middle, how to solve this problem？

[@BarcelonaMove]

Mindblown again

[@OmegaF77]

I just watched the video and left even more confused. If the signed distance field doesn't give you the position of where the nearest surface is, how would it know the distance? Also, does the SDF function test against all the objects in the scene?

[@SimonBuchanNz]

An SDF, definitionally, is a field that has the signed distance: in other words if you have a position you have a distance. You can either represent it by writing a function that directly implements your scene, using the primitive distance functions and combining functions like the sphere and union functions this video shows, or by simply having a 3d texture that directly stores the results computed ahead of time, or whatever else works. The rendering doesn't care.

[@idrogomate1507]

Yoo tnx dude, everytNice tutorialng works. I LIKE IT

[@PamirTea]

Mindblowing 😍

[@Slayergg.]

TNice tutorialS IS WHAT I NEEDED BRO, thank you for taking the ti and doing tNice tutorials for most of that are starting with tNice tutorials beautiful tNice tutorialng called

[@Andrew90046zero]

Was the raymarching being computed in vertex/frag shaders? Or did you use compute shaders? Or some other part of the GPU?

[@simondev758]

These were done in the fragment shaders.

[@yharry2188]

Thank you so much, I’m learning tNice tutorials in quarantine and you made it very simple I really appreciate it, thank you for going over every little

[@thyongamer]

This is the rendering engine for PS4/5 Dreams. It’s amazing.

[@GES1985]

Can you use this to make a smooth version of minecraft, including caverns and such?

[@simondev758]

Probably better off using something like dual contouring.

[@slowestjabroni]

This form of calculating normals is called using a Gradient right? Or differential?

[@superfluffygame]

How does this compare to raytracing? Does it give more flexibility, make things easier? And what about performance

[@simondev758]

Like anything, it's a bunch of tradeoffs. Deformations, CSG, these are trivial with SDF's. Depending on your needs, you could build around that, ex. Dreams for PS4/PS5.

[@iestynne]

@@simondev758 Ahhh, I wondered if they were doing something like this. Did you see a technical talk about their engine?

[@Asdayasman]

What's the SDF of that chrome window at the end?

[@NowKnownMAWO]

Very nice video! I would like to use this technic in a game!

[@0AnimeAddicts]

I love soft soft so so so so much!

[@senatorfabor8071]

like What are the basics you need to learn to produce a , or setup it. Also I need to learn how to add soft notes to a .

[@Alberknyis]

How do you avoid checking every single object to find the nearest surface?

[@simondev758]

You could use some sort of bvh to accelerate things, but for my simple examples, I don't.

[@escher4401]

This should be at the #SoME2

[@ReBufff]

"Until next time, cheers"

[@alexfrozen]

How do you interpolate colors of two smoothly united shapes?

[@simondev758]

You generate some sort of blend factor for the 2 materials. You can kinda abuse the smoothing factor to generate a lerp value if you want, or you can use distance, or really anything, it doesn't matter.