This Is Ray Tracing Supercharged!

❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com/papers
The free light transport course is available here - let me know if you enjoyed it!
users.cg.tuwien.ac.at/zsolnai...
ReSTIR improvements: research.nvidia.com/labs/rtr/... + Code: github.com/NVLabs/conditional...
Displacement mapping improvements: research.adobe.com/publicatio...
Real-time: arxiv.org/abs/2301.11376 + Shadertoy implementation: www.shadertoy.com/view/dsGBzW
Cloth rendering low memory paper: github.com/shinjiogaki/shinji...
Help developing LuxCoreRender! - forums.luxcorerender.org/view...
More on displacement mapping (not the first work on it, but a great report): d1wqtxts1xzle7.cloudfront.net...
📝 My latest paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Brizzee, Gaston Ingaramo, Gordon Child, Jace O'Brien, Jie Yu, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Putra Iskandar, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.at/~zsolnai/
Twitter: / twominutepapers
#nvidia

[@infowarriorone]

I remember ray-tracing my first cube, decades ago. I think I was using a 386 PC at the time, and it took hours. But it was amazing. Even more amazing is how far the technology has come.

[@gavinderulo12]

Now we have full on path tracing in fully fledged modern open world games like cyberpunk.

[@MrMBSonic]

386 🙈 with AGP Graphics Card and 3dfx accelerator Card 🙈. Absolutely crazy what has happened since then.😁👍

[@geort45]

pov-ray?

[@infowarriorone]

Sounds familiar, I believe that was it. It was a command line input. @@geort45

[@infowarriorone]

I had a Trident Mach-32 graphics card, if I remember right. 256 colours! @@MrMBSonic

[@ewerybody]

4:33 in game dev lod0 is usually the highest resolution model! ☝🤓

[@einhalbesbrot]

so "Level of Detail" "(LoD) = 0" is higher than a LoD = 4 ?

[@mho...]

whoah 2 minutes ago, i need to hold my papers tighter, so they arent blown away by the speed!

[@mrrfyW_]

Heheh

[@annekedebruyn7797]

These raytracing techniques and fluid simulation papers are probably my favorites of this channel.

[@marshalleq]

Yeah, I remember raytracing my first image with pov ray back in about 1990. My friend had a 386 with a maths co-processor and my Dad got us a 486DX - one of those rare times you were lucky as a child. With my friends maths co processor, one frame would be made in approximately 6 hours I think, the same frame on the DX was about 1 hour if I recall correctly. Now we're talking about ray tracing in real time at more FPS than we need. Sometimes I just marvel at how far we've come, though looking at it retrospectively that is actually 34 years which almost feels slow! Love your work!

[@phaeb]

I keep wondering: What is life like if you are so criminally intelligent to come up with all this in the first place. There is such broad and deep knowledge needed to be able to make something like this from scratch. It's amazing what the human mind is capable of.

[@avalokiteshvara113]

You assume it's just ONE person and discount the collective and collaborative intelligence of humanity

[@phaeb]

@@avalokiteshvara113 no I don't and no I don't. What even makes you think you need to come herr and tell me what I assume and what not? Have we met? It undoubtedly takes a lot of intelligence from everyone involved and I made a very general statement.

[@waterfish3886]

@@avalokiteshvara113

[@avalokiteshvara113]

@@phaeb you used "you", so I thought that's what you meant. English is not my first language. Maybe I misunderstood?

[@phaeb]

@@avalokiteshvara113 in this context, it means the same as "if one is so intelligent". Your first comment wouldn't have me assume English isn't your native language.

[@redthunder6183]

Wow, thank you so much for your ray tracing course free! I’ve been working on a game engine (to make a game with) recently, and I just hit the point where I need to work on lighting, this is perfect!

[@m0nologger]

looks like this was inspired by Unreal 5 with the LOD stuff. also super cool that the displacement works so well with caustics!

[@briansavery]

Displacement and "micromeshes" (aka micropolygons) has been around way longer than UE5. For example Pixar's REYES algorithm.

[@DeletedDevilDeletedAngel]

@@briansavery he means the current displacement micromeshes not previous

[@tirushone6446]

if your thinking of nanite that's totally different. nothing to do with displacement

[@user-tu5un8jc9v]

That's not how 3d modelization works

[@FG-418]

5:09 I was not expecting to see UdeS and Cégep de Matane in these videos!

[@ogsamoe3353]

I remember back in the mid 90's using Strata Studio Pro which had raytrace rendering and another rendering algorithm called raydiosity built in. I was running a Power Mac 8100/80 AV and it would take a few hours to render one scene let alone an entire animation. I wouldn't have imagined being able to render raytracing in real time!

[@waffle911]

I remember watching an interview with a former Pixar animator saying the original Toy Story could be rendered in real-time with today's hardware. That was a few years ago. I wonder if that was real-time on a render farm, or a single workstation?

[@Glowbox3D]

You are awesome, love your vids. Keep 'em coming! Happy New Year!

[@volvo245]

CONGRATULATIONS SHINJI CONGRATULATIONS CONGRATULATIONS CONGRATULATIONS

[@Aim54Delta]

I am really interested in using a parametric modeling system as a real time physics and rendering baseline. As most architectures are memory bandwidth constrained, I suspect that parametric modeling would ultimately be faster and scale into hardware better than most triangle based modeling systems. I am basically wanting to fly before learning to crawl, though, so there is that.

[@WDShorty]

That fractional LOD is basically what they use in Zelda botw and totk for the terrain, I think what we need is the ability to fake it for smaller objects for real time and only visually displaces the information in 3D on the edges of the objects with another fake 2D displacement for the rest of the object that blends together seamlessly

[@Madlion]

This is amazing. And i almost forgot about your course which I planned to take at some point when I have free time. My question is does your course touch in the topics of realtime applications for GI too or is it more for high fidelity offline rendering for cinematics etc?

[@ianohara6493]

THE DISPLACEMENT PAPER IS NUTS I ALMOST DROP MY PAPERS!

[@JackabeeT]

Cool, so when can I use this in Blender? We've seen like 10 papers this past year on improving ray tracing and basically none of these crazy advancements have made it to any useful software for the end user. I WANNA RENDER VOLUMES AND CAUSTICS FASTER, DAMMIT!

[@jdfhgkvdjb]

Nvidia's financial and marketing departments will decide on that in the near future, don't worry, you're in good hands ^^

[@_burd]

They have to figure a way to lock the new processes to the next gen of cards to generate demand for buying the next gen, even though it's a software improvement ;) "In good hands", sure, if you're a shareholder :P

[@kylebowles9820]

It'll take longer than that though, the person who put that implementation on Shadertoy did it unreasonably fast and well, and Shadertoy is built for research, Blender is incomparably more complex and will take a lot longer. The Blender devs aren't the researchers writing these papers. That's why Shadertoy is great, it takes the complexities away and let's you focus on the algorithms

[@JackabeeT]

@kylebowles9820 I'm aware the software devs aren't the scientists. I just hate that we're seeing more advancements in Ray tracing than ever before, and there's no sign of any of it being useful for the end user anytime soon. It's hard to be excited about something that may or may not even happen.

[@dimka_dev]

The example with the woven basket was mind blowing!

[@Julzaa]

Didn't know about Shadertoy, cool stuff!

[@Mranshumansinghr]

I have been doing Rendering for over 20 years. This is the time to be Alive!!

[@SomethingImpromptu]

I’m antsy to see us to start modeling the full complexity of quantum processes/mechanics already, building worlds from fundamental constants & physical laws upwards so that the higher levels of complexity are emergent from the underlying ones, the way our own world works, but all this simulation of light is an exciting step in that direction. It’s certainly far beyond what passed for light simulation on the N64, or the PS2, during my childhood.

[@ITS_MEEE333M]

i mean those walking light look neat :D

[@lemurpotatoes7988]

One of the links in the description is broken. Which paper was recommended for more information on displacement mapping? Tyvm!

[@acommon]

I wonder if this can work on high ISO photography as well. Similar principle

[@Frank-do1bg]

what a time to be alive!

[@yaazarai]

You should cover the new GI technique Radiance Cascades.

[@mnomadvfx]

Oh wow hadn't heard of that technique before, thanks! Alexander Sannikov sounds like a new John Carmack on the scene. Nice to see game devs creating and applying new techniques directly in the industry - if anyone knows the constraints and how best to work around them it would be game devs.

[@TreesPlease42]

Is that similar to NERFs with radiance fields?

[@nexusphreez]

Lol, I remember Ray tracing my first scene of a plane, a reflective sphere The wood texture sphere when I was probably 15. I guess it would be 30 years ago, on my Atari 1040 ST. 1 MB RAM, Motorola 68000 processor, using GFA ray tracer. It was all done by text no interface. Yes we have come a long way since then.

[@chrissears9912]

Same! And building the world via text script using basic shapes and geometry

[@foxthroat3410]

NVIDIA just keeps on popping off

[@samson7294]

People complain about Nvidia being expensive. I complain as well but there is no denying that they push the industry forward.

[@mnomadvfx]

The thing is though, they have no choice but to do so. When your big, brand new touted GPU feature and excuse for pumping up the GPU prices turns out to be utterly dependent on texture detail stealing denoising just to get an acceptable image quality and frame rate then you have a serious problem convincing game devs and gamers to take it seriously. nVidia launched RT literally 7-10 years too early before the software was even close to where it needed to be, let alone the hardware. Look no further than nVidia stooping to using Quake 2 and its mediocre 90s low poly geometry as a demonstration for real time path tracing for the bare truth of that statement. They might as well have strapped a sign on it that said "oops we screwed up BIG TIME". You can bet your ass that if nVidia were already this far in 2018 that they wouldn't have been so furiously at work trying to improve real time RT/PT software techniques that only end up benefitting both of their competitors in the market - it's doubtless already been a huge boon to XSX and PS5 devs.

[@predaalex3210]

@@mnomadvfx What timeframe do you have in mind for path tracing becoming universally implemented in games and able to run noise free ?

[@samson7294]

@@predaalex3210 I'm not really sure but I think the pace it has gone so far has been pretty impressive. The good thing about Ray/Path tracing is that it's optional. If people can't afford the hardware, they can still play the games without it. Console gaming is where you see universal implementation of features. Right now the consoles do a weaker amount of Ray Tracing so I suspect that we will see them have way better quality ray tracing in the next generation of consoles. So maybe 2027 give or take. If someone had told me in 2020 that Cyberpunk would have path tracing in 2 years, I would have laughed them out the room. Does Cyberpunk have some ghosting and noise issues? of course but compare Ray Tracing from 2018 to now. Its night and day!

[@Mrzeseb13]

@@samson7294they push them 'forward' not the industry AT ALL

[@btarg1]

I want to see these techniques in game engines immediately, this is HUGE for games!

[@mnomadvfx]

It looks better but if you actually read the paper itself it is too slow at the moment for use in games. For lower accuracy high speed PT rendering it's fine though.

[@B.D.F.]

Considering the PS4 game The Tomorrow Children was able to provide real-time ray tracing without any dedicated hardware I think techniques like this would improve performance.

[@MadsterV]

@@B.D.F. it was very sparse though. Kind of a special case.

[@S41L0R]

ohhhh these were different papers!

[@timetobargeinandgivemyopin7260]

Absolutely incredible

[@ranks6670]

BRAVO Shinji

[@itskixbro_]

I'm not old enough for this university level stuff but I still find it interesting 😭🙏

[@AirmanCS]

Just get to it, age means nothing

[@Embassy_of_Jupiter]

You always dig up the best noop apers

[@matorhdd]

oh, this is new format of beginning the videos ? I miss the "This is Faro Jona I Fahir" introduction ! :D

[@WeeGeeX]

no link to the tutorial series in the description?

[@nanow1990]

The acceleration has only just begun. What a time to be alive!

[@NoSoyMako]

1:20 "nor perfect" WHAAAAAT? FOR ME THAT'S BEYOND PERFECT, I CAN'T SEE ANY NOISE

[@dixoncider9billion]

lambdalabs on-demand pricing increased. H100s are no longer $1.99/hr

[@TwoMinutePapers]

Good catch, thank you! Will update it soon.

[@siquod]

So this channel is now "three two minute papers".

[@ferenccseh4037]

I'm happy to see ray tracing get some care too! Hopefully it will soon become a viable alternative to triangles!

[@pawetylman7678]

Great video! Thanks :) Is this the new technique used in Cyberpunk 2077 Ray Tracing Denoising?

[@nabushdinosaure]

No, cyberpunk uses ReStir Gi with the latest 2.1 update

[@anterprites]

Showing noise in youtube video :D Ambitious :D

[@lucian6172]

At 4:35 the level of detail is incorrect. Level 0 has the maximum detail and the mesh gets decimated more and more as the distance from the camera is increasing or the LOD is increasing. That's how it works in Unreal Engine. It works in reverse, more like LOD/distance.

[@cocobos]

Makes me wanna continue playing Jusant 😅

[@tmvkrpxl0135]

When he uses "we" instead of "they" we know this one's gonna be extra cool

[@rb8049]

I live in a simulated world. I’m a Teslabot being trained to serve. Once I die, my training will be complete.

[@loneIyboy15]

Watching this at 144p doesn't have the same kick.

[@kylebowles9820]

Shadertoy

[@SomeGuyOnHisDreamBike]

I want your new tech in Cyberpunk 🤩

[@mnomadvfx]

I wouldn't hold my breath on CP 2077 seeing that much future investment in better RT/PT. The whole reason they are switching from in house developed game engines to UE5 is they want to stop wasting time on this stuff and concentrate on the gaming experience/assets again. It will doubtless make it into UE5's path tracer though, and RTX Remix based projects should all benefit in the future once the technique is better optimised.

[@Gringottone]

Does LuxCore work with 3ds Max?

[@marinomusico5768]

We got to PERFECTION. Just add small imperfections and it will become indistinguishable from reality ❤

[@Tr33fiddy]

Can it really calculate caustics with just a displacement map?!

[@kylebowles9820]

It used photon mapping on the displacement mapped geometry

[@Tommo_]

Actually thought this was a digital foundry video before I clicked. Someone send this to Alex

[@CoyotePark]

People, also use more UNIGINE Engine. It deserves development and usage

[@mikelwrnc]

The noisy output looks like the output you’d see partway through a diffusion-based image generation network; wonder if folks have tried generating scenes with an increasing number of rays, for use as input/output to a diffusion image generator that, once trained, generates images approximating the high-count images from the low-count.

[@Nobody-sp7ug]

That sounds like a description for ray reconstruction.

[@descai10]

that's how AI denoisers work, diffusion models were originally born from work on denoising

[@mikelwrnc]

@@descai10​​⁠ah, cool. Seemed sufficiently obvious that I figured it was probably a well-worn path.

[@sanketsbrush8790]

where can I play with this renderer ai ?

[@MaxIzrin]

I thought this was going to be about cone marching. Still good though.

[@lllllMlllll]

I AM MIND BLOWN 🤯

[@beanieteamie7435]

Eight Minute Papers

[@altkev]

Is there an AI that can remove overly dramatisized pauses in speech?

[@squirrelwood8008]

Is there an AI that can add them?

[@CreamusG]

There's an extension called Jump Cutter.

[@altkev]

@@CreamusG I appreciate the serious answer. This was mostly a joking jab at how the video creator chooses to talk with unnatural pauses imterlaced in their speech.

[@exosproudmamabear558]

@@altkev It adds flavor I think.I like how unnatural and enthusiastic it sounds

[@jwhi419]

@@exosproudmamabear558 i dont. But thats what the 2x speed is for

[@Merrily-in1mq]

Two minute papers, what do you think about the word 'cheeky'

[@MrTomyCJ]

Reusing footage from older videos that say "New technique" in them is kinda confusing

[@theftking]

This video could've been like 4 minutes.

[@petterlarsson7257]

what are walking light blobs

[@UpstreamNL]

Me: oh neat KZbin compression: AAAAAAHHHHH

[@_cul8r_]

You sound just like Keralis, lol!

[@punk3900]

My intuition is that all those intelligent algorithms that simulate the real world (or rather trick the viewers into believing that this is genuine 3D, real light, etc.) will be at some point replaced by multimodal AI that will simply "imagine" each frame based on what it saw in the past. Remind me of the post in 10 years. PS. Thanks for this exciting video.

[@wilburdemitel8468]

I'd say 5 years.

[@adonisengineering5508]

🎯 Key Takeaways for quick navigation: 00:00 🤖 *Ray Tracing Algorithms Enable Photorealistic Image Synthesis But Have Limitations* - Ray tracing can simulate light transport and create photorealistic images - However, ray tracing algorithms have a huge problem with noise 01:18 🛡️ *New Techniques Reduce Noise in Ray Traced Images* - New algorithms like ReSTIR produce less noise than previous methods - But some still have artifacts like incorrect specular highlights - A new technique reduces noise and matches reference simulations 03:37 💼 *Displacement Mapping Makes Simple Geometry Appear More Detailed* - Displacement mapping adds detail to simple 3D geometry - A new technique does this up to 11x faster with 3x less memory - It supports continuous level of detail transitions 05:47 📚 *LuxCoreRender Enables Photorealistic Rendering and Needs Contributors* - LuxCoreRender is a free, open source photorealistic renderer - It implements advanced light transport algorithms - More contributor help is needed for this community project Made with HARPA AI

[@philtrem]

This is incredible!

[@djayjp]

Missing performance figures.

[@mho...]

i mainly want to know if these are software "fixes" or if we need to wait for the next 1k$ gpu's to use these new techniques?!

[@Danuxsy]

NVIDIA will probably show us really cool things when they announce their new GPUs later this year

[@mnomadvfx]

No these are software technique improvements.

[@hulejul9748]

the noisy imagine kinda looks like what everything looks like to me irl. guess my denoise filter is broken 😂

[@exosproudmamabear558]

Time to put chips in our brain to get 8k in real life

[@krele14]

This is at least the 4th time you claimed raytracing has become supercharged. I've been a fan of this channel for the last 5 years, but considering how much you blatantly spam videos just to satisfy your sponsor I think I'll unsubscribe until someone takes your place. Good run.

[@fritzschnitzmueller3768]

Yeah I agree. This video is kinda a mess and he just uses a lot of enthusiasm and fancy images to overshadow it

[@krele14]

@@fritzschnitzmueller3768Used to be such a cool channel, diamond in the roughest of roughs. What a fall from grace.

[@cheezitupp]

Light transport, computer graphics, virtual worlds? Oh my! What a time to be alive!

[@NTKM-om9vn]

You need Nvidia 4090 Super to pull this off

[@ChrisGuerra31]

It's even harder to contain my excitement now that I have an rtx

[@HansMilling]

Imagine all the old VHS tapes that you could denoise and upscale with this technique.

[@ThisIsNotWhatItLooksLik]

If games can be fully raytraced, does that mean they can be rendered with multiple GPUs resulting in linear fps increase per gpu?

[@incription]

yes, but that is completely unpractical... go tell gamers they need 2 gpus minimum spec lol

[@FinaISpartan]

No, the frame synchronization creates alot of overhead. Also, post processing effects can really only be ran on one of the GPUs, so yes, rendering of the meshes will scale linearly, but FPS will not scale linearly.

[@gustavodutra3633]

Too much latency, so, no.

[@ThisIsNotWhatItLooksLik]

What post processing effects can only be ran on on GPU?@@FinaISpartan

[@ThisIsNotWhatItLooksLik]

Why would there be more latency with 2 GPUs than with 1?@@gustavodutra3633

[@michelprins]

great video thx

[@ZackThoreson]

Finally, non ai bs

[@user-me5eb8pk5v]

Gourad is fine, you just need multi core modeller. I remember I made jet pack jack back in 2010, he only had 15,000 polygons, but due to the storage capabilities, he moved at 1 fps, like the grand canyon, 500,000 polygon dimensia. I gave jet pack jack literally nothing, a belt with cone bombs and grenades, I mean holy shitty programmers batman. I mean, I don't want unreal engine, tell me everything before you even write the code Unreal Engine. How would you move mars and saturn around in a simulator? Want to use the 10D pivot radix data.

[@alphamegaman8847]

At 2:45 Tesla Bot 3.0! 😁 2028?🤔👍 Mike in San Diego. 🌞🎸🚀🖖

[@phonix6494]

This is a bit misleading at 1:00 the ReSTIR PT method is used to demonstrate that denosing can not get rid of walking ligth bloks. Shortly after, 1:11, MMIS is used as the previous technique wich does not seam to have thiese walking ligth blocks then MMIS is replaced with the new one and then denoised. Why is the new one then not compared to a denoised MMIS image but a wores denoised ReSTIR PT image?

[@descai10]

it's pretty clear that the denoised new method will be better than denoised MMIS, since it has significantly less noise

[@phonix6494]

@@descai10 yes my point is taht denoised mmis was neaver eaven shown

[@stuffy.design]

Is the voiceover in this video made with AI? :v

[@qrcus]

Okay but who cares about it if it's not implemented in games ? Once it will start getting actually implemented, then i would actually be interested.

[@el_teodoro]

Kids like you shouldn't even watch these video. Let the adults work on it so that you can enjoy playing your silly games :)

[@MrCrackbear]

no one cares that human innovation isn't exciting to you and that you just want to spend all your time wonking your willy and playing video games. the world doesn't revolve around you.

[@t1460bb]

Fantastic!

[@ewerybody]

Quite a mixed bag of stuff actually 🤔 I'd rather see shorter videos dedicated to one (or two closely connected) paper(s)

[@jopo8281]

I swear you didn't say what a time to be alive. What sorta joke video is this?

[@TalynStormcrow]

Is this splatting?

[@heckensteiner4713]

It's a LOT faster if you ray trace a single photon. I don't know why these people are using millions of them... 😏

[@Alex-wg1mb]

Slap this technique into blender cyclesX add Turbo Tools v4 Addon And Redshift bites the dust

[@mnomadvfx]

Why would Redshift bite the dust? This is not viable for production rendering where accuracy is paramount. It's great for path traced preview rendering though, no doubts there. There has been a lot of movement in the non real time RT/PT arena of research though in the last couple of years, so no need to hang all your hopes on nVidia there as plenty are working on everything else.

[@descai10]

@@mnomadvfx it looks a lot more accurate than previous techniques

[@oBCHANo]

You need to pause less between words.

[@DdZ-lv3jq]

Watching this video on 360p does not really help. Just a small tip for today.

[@TupperWallace]

It’s interesting to see how many people can’t cope with a non-native accent and phrasing patterns. Hopefully all their future instructors will have midwestern US broadcasting school pronunciation and delivery!

[@jadeviper1290]

This isn't just "non-native accent and phrasing patterns". Nobody talk like this in real life, it's robotic. Whoever is making these videos does not understand the English language in any capacity other than pronouncing words one or two at a time and then stringing them together while cutting out the time in between where they used google translate to get the pronunciation.

[@ChillyFries444]

​@@jadeviper1290 so? What the fuck are they supposed to do?

[@jadeviper1290]

@@ChillyFries444 maybe learn the fucking language they're attempting to communicate in?? If you can't speak a language without pausing every 2 words, maybe don't make a clickbait KZbin video in that language, or just use a text-to-speech bot like the rest of these copy-paste "content" channels.

[@talananiyiyaya8912]

"NVIDIA" okay 2 minute shill

[@tomtomkowski7653]

1. Remove "nvidia" from the title 2. say about a drawback which is a 10x slower process

[@asta3457]

i love your videos

[@natmarelnam4871]

Cool... I look forward to Zero games implementing it.... The fatal flaw of Ray Tracing that makes it useless. It'll be cool in the future but if you're spending money on it now, you got played.