Ray Marching, and making 3D Worlds with Math

Built this entire video on raymarching in shaders using shaders, ray marching, and signed distance functions.
🛒 Recommended books (on Amazon): www.amazon.com/hz/wishlist/ls...
❤️ Support me on Patreon: / simondevyt
🌍 My Gamedev Courses: simondev.teachable.com/
Disclaimer: Commission is earned from qualifying purchases on Amazon links.
Follow me on:
Twitter: / iced_coffee_dev
Instagram: / beer_and_code
In this video, I cover ray marching and some of the cool stuff you can do with it in shaders. We cover basic scene construction, unions/intersections/subtractions, doing smooth minimums, normal generation and soft shadows, and anything else that seems kinda neat about the technique.
Ray Marching: en.wikipedia.org/wiki/Ray_mar...
Great Resource for Ray Marching and SDF's: iquilezles.org/
Smooth Maximum: en.wikipedia.org/wiki/Smooth_...

[@simondev758]

Patrons can now vote for the next video! Thank you for your support. ❤ Support me on Patreon: www.patreon.com/simondevyt 🌍 Live Demo + Courses: simondev.io

[@Chribit]

I‘ve actually been working on a graphics engine using signed distance fields as primitives instead of triangles for ~2 years now - super hyped to see it become more and more mainstream recently :D

[@keyb]

Ooooh sounds cool! Do you have any demos or code bases you can share?

[@Chribit]

@@keyb not yet unfortunately. The first year was mainly reading papers, understanding the heaps of theory and finding a sensible way of actually making it performant. There are bunch of challenges i wanted / had to solve first: - how do i make dynamic scenes with SDFs instead of scenes hardcoded into a shader? - how do i avoid as many branches as possible? - how do i completely avoid the default render pipeline? - how would instancing work? - how can i drastically improve performance? - etc. And the second year was mostly occupied by learning vulkan tbh. But i‘ve been making some major strides recently and i‘m hoping to finish a 2D version before end of the year. That will allow me to make some tools to drastically accelerate my dev process and it will all snowball from there. End-goal is a full 3d engine for real time rendering :D I can bookmark this video and let you know in this thread when something showable is abailable if you want? :)

[@simondev758]

Super cool! Got a demo I can watch of it in action?

[@Chribit]

@@simondev758 not yet unfortunately :D See comment above yours hahaha I‘d be happy to send you a demo as soon as something showable is ready, hopefully near the end of this year :)

[@keyb]

​@@Chribit Yes please! It sounds like a ton of work (especially in the sense of optimizations) so you still probably have a while to go. Whenever you do get a working model please let me know!

[@stickguy9109]

For the first time I understood what this magic "ray marching" is and how it works. You are good at explaining things

[@johanngambolputty5351]

Even if I ever understood before, this might be the time it actually sticks :) for some reason I always mixed it up with tracing, or maybe something to do with section searches

[@dutchdykefinger]

​@@johanngambolputty5351 well technically you're still tracing rays, right? certainly it's ray casting, the term they used in 3d games back in the 90s to collision check your projectile trajectories and the such, which is just 1 ray. [ray tracing] is a routine that casts a ray for every angle your "caster" can "see". in 3d rendering often a camera, so the camera settings dictate the position and the angles the caster limits itself to, the pixels it traces are projected into angles depending on FOV/perspective values from the camera, they ARE the set/array of angles to process basically. it's still raycasting too, raycasting a lot in fact, raycasting is still the integral part that does the work. (a 1080p image is about 2 million pixels, so ~2 million rays if just 1 ray for every pixel, modern path tracing do many many more per pixel) there's also bounces for light ofcourse and the such, but for the sake of simplicity let's keep it at 1 bounce. [path tracing] is a more complex raytracing that can also do reverse tracing for caustics and the such, has way more rays for more modern PBR rendering methods, z distance information feedback for post processing and the such, essentially raytracing on crack. [ray marching] i consider it to be a variant/derivate of ray tracing, i'm not sure how disputed that is, but i'd put it under that moniker. but it is smart/dynamic, rather than checking everything, it makes bigger jumps, and you can reduce the amount of cycles by a great deal just working with minimum lenghts set on a higher value , but very small object intersections aren't "seen" because the ray jumps over them, this disadvantage is exploited as an advantage in the demos in this video by interpolating and blending the missing parts together the "looking over it" principle is also used in metaballs, but metaballs calculate a lot of shit on the way, this method actually drops data and fills in the gaps, it has a bit less control, but you could put extra data in vectors and use those with proximity for all kinds of effects too i don't think the vector lengths (jump distances) strictly have to be dynamic for it to be ray marching, they could be static and just have it jump the same distance every time as a crude version until it hits something it's not ray marching if you only shoot 1 ray either, so it's just grid-snapped ray casting strictly taken, but the distributed is implied although there may be differentiation in terminology these days perhaps, but that was also understood as ray marching back in the day ray marching is a de facto collision detection algorithm, as is ray tracing, because they're all raycasting and the rays are looking for something to bounce on, that precisely is what collision detection is, finding an intersection. as with many code, raymarching differs a lot between CPU based (serial functional programming) and GPU based (parallel logical programming), they need a quite different approach. the crude version of ray marching basically is raytracing, but the ray casting it does, is snapped to a grid with the grid's rotation relative to the individual ray's angle so it always traces over a striaght line (which is exactly what "local coordinates" are) short story long: i consider raymarching a variant of raytracing, i'd define them as: "distributed raycasting algorithms" when in doubt, use the term ray casting, that's the underlying principle for all of them, including ray casting itself obviously, so you can never be wrong then ;) even simple pixel-based detection is essentially just crude ray casting with the vectors being snapped to integer values, using global coordinates, and dropping all the benefits of unit vectors, when you think about it, right? lol i just had an epiphany, never even thought of it about like that myself even.

[@johanngambolputty5351]

@@dutchdykefinger True, its a clever sort of tracing that you can do if you use distance fields instead of polygons, especially since the step size is not just dynamic but optimal (largest while guaranteed not to hit anything). I'm not familiar with this stuff, so I just imagined constant step sizes for tracing (against polygon primitives)... though I'm sure it can get way more clever

[@BalintCsala]

Great video, but small correction on a common misconception: Ray marching is just doing raytracing but numerically (so instead of having an equation tell you how far you need to go, you just go in small steps and check if you are inside a shape at any point), it doesn't necessarily involve SDFs (e.g. screenspace reflections also use raymarching, but not SDFs). The special name for this is sphere tracing.

[@stdcall]

yeah, the naive ray marching function steps a point towards the target and returns when it ends up inside an entity/object

[@diodin8587]

Do you mean sphere tracing is just ray marching with SDFs?

[@BalintCsala]

​@@diodin8587basically, if you open up the wikipedia page of ray marching there's a section on it.

[@magnusm4]

I remember this blend mechanic being used in a Valve tech demo from around 2003 where they made blobs able to combine and melt together like here. Later used years later for Portal 2. Insane they came up with this already back then.

[@TheJeremyKentBGross]

I believe they used a 2D version for text on signs in Team Fortress 2. I vaguely recall reading about it back in the 2ks.

[@cucumberedpickle]

I've seen a lot of videos about raymarching before, but this is the first time I've "gotten it". The graphics you used to explain it make so much sense.

[@guaje]

It's great to see more content like this. Loved your explanation of the central differences method.

[@uquantum]

Wonderful Simon, came here from another one of your shader vids, mind-blown that you made the actual video using ray-marching, now picking jaw up from table

[@RobinPayot]

Amazing video, well done on the explanations! The 3D scene really helps to understand, maybe explaining a bit more the code part would be even better

[@simondev758]

Yeah, I think if I were to redo it, I'd add some animations around computing things like the normal/shadows in the same way that I did for the raymarching. Next time!

[@cl10k]

Your vids are so incredibly good! I always get excited when a new one comes out. Pure gold!

[@gtnbssn]

I have started the glsl course a couple weeks back and am learning so much! Thank you!!

[@simondev758]

Awesome!

[@Matkins85]

Nice video! I love playing with SDFs too. One point I think is worth mentioning, which I'm sure you understand but perhaps some viewers won't. It's a misconception that using modulo to repeat the space, and get seemingly infinite copies of your SDF, is for free. From a performance point of view its far from free. It's because then more rays are getting slowed down, and more frequently so, by passing near to surfaces. It is a neat trick though, and if your SDF is fast anyway then its not an issue.

[@simondev758]

Agreed, the performance cost is definitely real. I was mostly thinking in terms of implementation cost for a lot of these effects, but yeah the ray marching loop can't early out as easily.

[@Shamysoza92]

Now THIS is how you sell a course. Great content!

[@awesomecronk7183]

I like the explanation! I'm working on my own ray marching based graphics engine right now and, while your demo here is far more polished, I'm getting places.

[@aquanton]

this was the single coolest video I've seen on KZbin for a while

[@simoniacdemoniac7525]

I purchased your course. You really explain things well!

[@lordlightspeed]

for smoothmin(a, b, k), using smoothmax(a, b, -k) also works and is possibly slightly easier to write in code.

[@younghsiang2509]

This is really a cool video, the way objects transition between frames are just amazing! Wondering how that could be made.

[@icimojojojo]

That's an amazing video Simon, thanks a lot for sharing your knowledges with us !

[@nordie92]

Nice stuff, nice explanation, nice voice. Love it

[@tomg0]

An awesome, well-made video. Love it

[@Madlion]

This is quite cool, ive added these in Unreal engine and u can even blend with the mesh generated SDF

[@Luka116_]

I love your editing & teaching style! Thank you for making such informative and fun content for free ^_^

[@andermium]

I love my recommended, what an amazing video! I feel like there's a good chance this would work well with Geometric Algebra, but I don't know enough of either subjects to really say something about it.

[@BobsiTutorial]

Really great video! Might implement this into my devlog game somehow 🙂 Great explaining and awesome video and visuals.

[@simondev758]

Let me know if you do!

[@SuboptimalEng]

This is awesome! The first thing I’m gonna do (once I get a job) is buy your shader course!

[@ArtisanAspirationWorks]

I was really envisioning this for an attack in my 3d plaforming game as an obstacle. Thx

[@williamjuicebutter6648]

This is the best content on youtube! Thank you!

[@ShiroiAkumaSama]

I won't ever try to write actual code, yet this video was still very interesting and understandable and it might teach me what I can use smooth min and max for in Blender.

[@filipathtc]

This is a really good explanation!

[@ngoc-64mtcothi53]

Don't give up mate, that was my first day to use soft soft and i will work on it for a long ti!

[@DickLarsson]

Simon. I totally love the stuff you teach us. LEGEND! 💙

[@VoidloniXaarii]

Wow this was so interesting! You also mentioned csg ❤

[@nikbivation]

wow very good explanation, made it easy!!

[@UrodCyka]

Spore-like games would benefit heavily with ray marching, its always something ive thought about. you could build a cell and have the individual cell parts combine using ray marching or make cellular animations of two spheres dividing or even have creature parts be put together using ray marching. Ray marching gives endless opportunities

[@Rennu_the_linux_guy]

CodeParade's marble marcher uses ray marching, if you haven't heard of it, it's really cool, you roll a marble along a fractal which is sometimes morphing around in real time

[@nutzeeer]

This is really cool! A lot can be imagined with math

[@akfamili279]

I'm your old fan, thanks for updating the program so honestly! You are a hero to me! Yoom view bot king! Definitely the best in the business

[@hanneshinrichs]

Basically this is what "Dreams" for the PS4/PS5 is made of

[@NowKnownMAWO]

Very nice video! I would like to use this technic in a game!

[@Rene-uz3eb]

Great video, thanks. Made me realize that I need a distance function

[@TruthAndLoyalty]

You can't help but love listening to Bob from Bob's Burgers explain programming concepts.

[@ScryGL]

Wonderful! I know how to do lighting/reflections/combining geometry but I think my code quality could use help for putting together larger more complex scenes. I'll take a look at your course :)

[@kilroy987]

When I look at a subject crooked with a thinking face, I know I've learned something.

[@alirezaakhavi9943]

really amazing informative technical and beautiful tutorial thank you very much enjoyed it alot! subbed :)

[@Mcbuzzerr]

As soon as we went 3D everything started going over my head 😅 Fun visuals tho :)

[@AuroraLex]

In VRChat there is a famous world called Treehouse in the shade, that uses ray marching. Its really beautiful!

[@mayank9rana]

fire video, thanks bro

[@thyongamer]

This is the rendering engine for PS4/5 Dreams. It’s amazing.

[@AntonioCorrenti]

Amazing stuff! Now I'm curious about how SDFGI works!

[@simondev758]

Me too!

[@AntonioCorrenti]

@@simondev758 just found this kzbin.info/www/bejne/d4PPk6uOradki7M

[@autbo]

The game called "Dreams" on the PS4 works exactly like this, except it has a "flake" effect to the surface to give it a painterly look.

[@rafaelsantiagoaltoe6606]

phew. That is a cool thing, I will have a deeper look later. Thanks

[@idrogomate1507]

Yoo tnx dude, everytNice tutorialng works. I LIKE IT

[@Joao-uj9km]

Wow! Thanks for the video :)

[@Skeffles]

Fascinating, especially when the shapes merge into each other.

[@arashvahabpour942]

Such a neat video!

[@alireza4734]

Every single sentence is worth writing down on the notebook. Thanks

[@DemsW]

Great video and explanations/visuals, makes me want to mess around with it. Would be nice to talk about the limitations and why it's not as popular as the usual rendering techniques Maybe a game like portal that does weird things with visuals could benefit from this.

[@simondev758]

Dreams on PS4/PS5 uses this technique extensively.

[@mexicanmax227]

I love you Simon. You’ve turned me into a smartass. Thank you 😃

[@simondev758]

Hah

[@boogeyman8099]

I would love to see an interactive VR experience that delves into ray matching.

[@KillFrenzy96]

There's worlds in VRChat featuring Raymarching shaders that you can experience in VR. It's quite fascinating. One of the worlds was called "Treehouse In The Shade".

[@boogeyman8099]

@@KillFrenzy96 thank you so much!

[@yharry2188]

Thank you so much, I’m learning tNice tutorials in quarantine and you made it very simple I really appreciate it, thank you for going over every little

[@zxuiji]

3:07, That wire frame cube is your clue to doing super cheap 2d ray tracing on 3d environments, and you only need to do it on 1 side, the rest just need that one side's ray positions to be multiplied against either 1 or -1

[@codytyson8460]

The videos only get better

[@sinantk4036]

you record with hardware outside of the program. Great tutorial btw it was very detailed but still just right for beginners.

[@sjoerdhogervorst5857]

using soft, can't wait to get my hands on it.

[@qualle987123]

Very cool video. I would have love to have it a bit longer and more in depth though.

[@rewatbeat]

because in the intro for example I want to make an acoustic soft, but later I want to make a distortion or any other effects in that sa

[@BarcelonaMove]

Mindblown again

[@0AnimeAddicts]

I love soft soft so so so so much!

[@theSlavenIvanov]

Great Stuff!

[@emrelermi1399]

Muito bom, vou atualizar. Very good. I will update.

[@ReBufff]

"Until next time, cheers"

[@mySDK3333]

There's a software called MagicaCSG that's so easy and intuitive to use to make 3D models.

[@PamirTea]

Mindblowing 😍

[@escher4401]

This should be at the #SoME2

[@Jellyjam14blas]

Awesome! :D

[@kalpolproductions8558]

Best video ive seen today. yosh

[@ThachDo]

Thanks for the intuitive video. May I know how to determine the necessary number of points to render a distance field of a scene?

[@simondev758]

Most implementations set a max # of iterations, usually about 100 or so get's you a good result, a lot less depending on the scene.

[@ThachDo]

@@simondev758 at each point/iteration, we have a different set of scanning angles around the point, right? So how about the scanning resolution per sample point usually?

[@jensBendig]

I feel intrigued to code a ray-Marcher…!

[@Slayergg.]

TNice tutorialS IS WHAT I NEEDED BRO, thank you for taking the ti and doing tNice tutorials for most of that are starting with tNice tutorials beautiful tNice tutorialng called

[@CharlesVanNoland]

Been waiting for someone to make a game engine that renders everything using SDF marching for a decade now. I know that evaluating many combined distance functions merged together to model something can get expensive, but utilizing simple bounding boxes and things to use as proxy geometry for the initial ray march origin for each object should help speed things up. I also think that extremely complex stuff could programmatically be reduced down into fewer distance functions that approximate it - particularly for an LOD type scheme. I don't know about an entire world being described using distance functions, maybe, or maybe they could be compiled down into static 3D textures that are raymarched instead - which likely would be faster than evaluating tons of distance functions every ray step for every pixel for every frame. I just feel like now's the time that this sort of thing could be realized if someone just sat down and actually pursued it.

[@simondev758]

100% I feel like as gpu's get more powerful and general purpose, it's opening up some interesting avenues that previously weren't feasible.

[@fabiorota9661]

2:43 "it's really that simple" Meanwhile me having to pause the video for 10 minuts in order to understand what was going on and why it even worked -_-

[@WistrelChianti]

I think my brain fell over half way on this one

[@haruruben]

Subscribed!!

[@adibhanna]

this is amazing! how do you do all these sick animations?

[@simondev758]

All done in shader, pretty much everything you see is based on what I explain in this video and the next (lerp).

[@SaintMatthieuSimard]

That's actually perturbing to watch. Feels like an old Outer Limits episode! :P You adjust the horizontals, the verticals, you're a wizard! 📺

[@simondev758]

I loved that show!

[@SaintMatthieuSimard]

@@simondev758 Hi, while you're around, I have a question, I'm a noob, can I use that method to project a simple parallax from an animated stripe and its animated mist pass? 2 mkv files, basically.

[@simondev758]

@@SaintMatthieuSimard I'm not super sure what you're asking, you can explain again

[@SaintMatthieuSimard]

​@@simondev758 What I'm trying to achieve is some kind of a deep picture frame that contains an animation which contains its own mist pass, for scene depth. I imagine the infinite possibilities of raymarching... Imagine if it was possible with a series of pictures containing their own depth map or estimated depth map, and making a snow globe out of it with accurate geometry from all angles, in Blender... That would be so great! Yet right now I'm really just after a method to create detailed pseudo-geometry off of low poly models. And the lowest factor is that I'm trying to cast a 60 frames scene into a cube, a picture frame, which contains depth information, and have an accurate representation of depth, without extra geometry nor multi-res modifier. Oh, is that even Blender that you used? I think you mentioned several other software, I'll rewatch just now.

[@SaintMatthieuSimard]

@@simondev758 Yeah I realized on 2nd watch that's not python nor blender :P I heard that with vector displacement mapping, certain textures can create creases in all directions and not just X and Y. I saw and old model that would create the geometry of an ear, then a nose, just out of a single texture. Tho I still haven't really figured it out. Learning that would clearly help me achieve what I've been trying to do for a while which is making a deep picture frame containing an animated scene rendered from pre-rendered material. An MKV file for depth, one for color diffusion, one for light, one for reflection... Yeah... That seem so far away for me right now x) For such a simple artistic project.

[@EpicVoiceShitposting]

..you had me on radius

[@Scratchydoesmusic]

oh yeah i heard codeparade talking about this in his marble marcher videos

[@trannguyenanan5067]

and I will find my way back there too!

[@Asdayasman]

What's the SDF of that chrome window at the end?

[@TunaFunDev]

Its all great but it wouldnt be imposible to have collision working on these shapes right?

[@vampirejs758]

Question: Can the minecraft clone you made be remade completely in glsl? Like the terrain generatjon and stuff

[@simondev758]

I think it could, but that'd be a major challenge. Maybe for a day when I'm super bored heh

[@freakfreak786]

Great video and explaination. I got a question tho: Did you hear about webGPU? Are you planing on learning/using it?

[@simondev758]

Definitely planning on playing with it

[@user-xy4bs1nm3k]

god damn it that is cool - lemme watch it 20 more times so I can understand it lol

[@brennonwilliams9181]

Watched it twice. Need a lay down and a cup of tea now. Simon, are these transitions, intersections, adds, subs being performed purely in the shader?

[@simondev758]

100% done in shaders yes

[@AntonioNoack]

yes, they are. Inigo Quilez has excellent tutorials about it. You can see hundreds, if not thousands of samples of SDFs on ShaderToy. The platform was created by him partially as well.

[@brennonwilliams9181]

@@simondev758 think you just sold me on the course. Awesome.

[@brennonwilliams9181]

@@AntonioNoack thanks! I will check it out

[@simonhatch4628]

@@brennonwilliams9181 Heh awesome. And yes, definitely check out Inigo's site, it's in the description, amazing resource for ray marching & sdf's.

[@preludelight]

Hey Simon, fantastic explanations and excellent visuals to go along with! You mentioned at the end that the code is up on your github, but I wasn't able to find it. Am I looking in the wrong spot? Cheers!

[@simondev758]

I probably forgot to upload it, lemme go find the code and do that.

[@weisbrja]

@@simondev758Not to come off as rude, but did you upload it yet? I also couldn't find it and I'd be super interested...

[@preludelight]

@@simondev758 Just casually dropping by to remind you to upload this code. Cheers :)

[@SVSSOF2022]

do the instrunt samples co free with soft soft?

[@nomadshiba]

i figured instead of rendering a sphere mesh, rendering sphere with ray marching cost less or same, and it looks 1000x better with instancing of course, so there is no use of `min` function. just render a cube mesh as sphere with ray marching shader

[@jazzgirl2358]

I'd love to see you making some Non-euclidean worlds in javascript

[@simondev758]

Great idea!

[@rcjinAZ]

Very clever.