UE5 and Nanite: Is It Finally Ready Yet?

In this video, we take a quick look at Nanite, and whether it works with VR. For me, it always been the most exciting feature of Unreal Engine 5. However, when the engine released, it wasn't ready to work in VR. Since then, that has changed, and we're seeing a lot of development in VR games as a result. Lets take some time to explore some of the strengths and weaknesses of Nanite and VR.
Some of my UE4 Products:
www.unrealengine.com/marketpl...
Please consider subscribing for more VR, Design and UE5 content!

[@iainstevens-guille5561]

Welcome back! People speaking to the technical grit of VR production with grace and elegance are rare. Please continue.

[@AlistairHume]

Thank you very much for the kind comment :) Definitely more to explore in VR develoment, both on the cutting-edge and in general. So more to come :)

[@violentpixelation5486]

💯🔥

[@tellhimimeating9141]

If you could do more examples of how to truly optimize this engine for extremely high performance, I'd love to learn more in that direction. Scalability at the very least, VR at the most, they're fantastic goals for any project.

[@mantas3d]

I Would be very interested as well :)

[@7_nels]

yes please, making an environment in VR is a nightmare, between draw call issues, texture resolution, framerate and compatibility issues between nanite, lumen...

[@ChrixB]

Very encouraging to see, I try nanite on VR with 5.1 and that was crashing all the time, we are getting there. The Lumen sunset at the end was really nice, can't wait to be able to do it with good performance. Thanks for sharing!

[@AlistairHume]

Thanks very much :) I also tried the sunset with no GI, and it ran smoothly (but disappointing visuals). I still do get crashes in 5.1 when developing for VR. Strangely, more for the Index than for Oculus. I think that's got something to do with the way that rendering resolutions are interpreted by the VR runtimes. I'd say that for every 10 'Play in VR' tests I do, I get one crash. A year ago, there was no chance of anything working at all. So I'm optimistic that Epic is now working on VR for UE5 as one of its development threads, even if it takes a second seat to things like Procedural content placement. Apparently 5.3 has more performance stabilizations, so I'll be looking forward to continuing on with the tests :) Also as a fun aside for Quest/Mobile VR, Unreal 5.2 is 5% more efficient to render, based on RHI optimization alone. So VR is coming along nicely :)

[@Glowbox3D]

Thanks for the video. We are developing a world in UE5 that will be for both flat and VR--it's been a real challenge. Love seeing videos like this.

[@theneologic474]

Man, this is insane stuff! Keep up with amazing videos like that! I'd love to see how it evolves and how far we can go down the rabbit hole!

[@henryfrankfox]

I was just thinking there's an Alistair Hume shaped gap in my KZbin notifications lately. Welcome back

[@AlistairHume]

Thanks so much :) Its great to be back! Hopefully some more to share soon. Nanite is getting really good for VR, and its not far off being able to ship a game, in my opinion. As long as the creators know how to generally optimize games, Nanite is a huge uplift

[@nathansharp3224]

My main issue with these high-performance engines is that they seem to be continuously refining and enhancing versions of Myst. While the graphics are undeniably impressive, I find myself yearning for genuine interaction with the environment and physics. For instance, when I encounter a bed of flowers, I want the ability to pick one and examine it. Similarly, if I come across a heap of stones, I'd like to have the option to pick one, break it apart for a closer look, or even dig beneath the surface. Despite their stunning visuals, these games often feel static. Granted, numerous games offer a degree of interactivity, but these instances often feel isolated or highly selective. I yearn for a game where this level of interaction is woven into the very fabric of the gameplay.

[@joelface]

I think for that kind of interaction we're going to have to move beyond basic polygons and start building with voxels. Voxels are like 3D pixels... so they're like little cubes that make up all of the graphics of the game. And the magic of voxels is that they can each be imbued with properties that make it similar to different materials (rubber, wood, metal, cement, etc). It's not even a VR game, but I think the indie game Teardown got me excited for the "interactive" future we want. It's sort of blocky, like minecraft, but as technology improves, those voxels will likely shrink smaller and smaller until we don't really notice them so much. I guess, actually, Minecraft is also a voxel game, but luckily the blocks in Teardown are much smaller. If you've heard of it, or end up checking it out, let me know what you think. I really think voxels are the future of VR.

[@MilieuChris]

@@joelface There was work on point clouds a number of years back too that might have more promise for real-time now. Note however, the groups i followed at the time ended up with their own engines. I do not know Unreal well enough to comment, but the global knowledge in the community may be sufficient if it is not a rational thing to try with Unreal itself.

[@zebius4157]

Did you know you can use nanite with chaos physics? It's all there

[@Owlboi]

thats not the fault of the engine. thats the fault of the game design.

[@zebius4157]

@@joelface here's a video explainer on nanite that goes over why voxels weren't used for it from 1:39 to 4:14 kzbin.info/www/bejne/m6fMhKyhpriIi9k

[@TimofeySop]

Welcome back! I am sure that with such quality as yours, you will gain back the views very quickly :)

[@AlistairHume]

Thanks very much! Its good to be back! Hopefully some more to come soon :)

[@gandepuun]

Welcome back. It's been really long since you dropped a video. Hopefully in the future, Lumen and Nanite will fully be supported in VR.

[@AlistairHume]

I really hope so too! The one that I really want to try soon is RTXDI. It looks absolutely incredible for large-scale scenes, but absolutely is not ready for this generation of consoles. Still though, could be a lot of fun for a far-future video

[@gandepuun]

@@AlistairHume Yeah, RTXDI. It's less expensive than Lumen right?

[@AlistairHume]

​@@gandepuun I hope so. My understanding is that its wildly fast, as long as you have great baseline raytracing features on your card. Whereas Lumen can run without specific hardware features, as long as you have a system that supports DX12. I personally feel that Lumen will be old before it really gets going, thanks to RTXDI and RTXDI (if they can work with Nanite, I guess)

[@ChrixB]

@@AlistairHume that's interesting, why do you think Lumen will not take off compared to RTXDI ?

[@AlistairHume]

@@ChrixB I think it will definitely take off - there are some amazing games that are being made for PS5 and Xbox that use Lumen very well. But I also think that by the time that Lumen hits its peak (+- 3 years of development cycle from now), there's a chance it'll already be outdated compared to some of the crazy methods that are being seen in development right now. Lumen was once described to me as a crazy combination of a few different techniques to achieve realtime GI. Meanwhile, RTXDI and RTXGI (and other methods from other companies like AMD) are just purely optimizing the path tracing method. I think there's a good chance that Lumen is a stepping stone to techniques like that, which will truly take us to the next gen :)

[@mattchamberlain5273]

Nice video, would love to see a series where you are optimizing a scene and see how much performance you can squeeze out of it

[@IRONFRIDGE]

Nicely explained, would love to see more experiments like this...

[@GregkotGaming]

I didnt expect this to be as fascinating as it was. Earnt a new sub.

[@joos3D]

Great video as always. I think I'm going to have to do some experimenting. It's a shame to hear that having many unique meshes could be an issue, but I guess it makes sense. Maybe streaming assets could help? Wouldn't really alleviate the problem if you struggle to fill a room with everything you need, but maybe with larger areas

[@FlyingWithFeathers]

this video got reccomended to me at the right time didn't even know it was vr but. I am starting my VR UR5 journey today and this answered a lot of my questions thank you

[@thechilledpotato6253]

Good to see you back!

[@musashidanmcgrath]

Set my first VR scene up to test Lumen/Nanite in 5.1. The scene was quite heavy, all built from decimated Substance Modeler and Zbrush meshes. I had to do a LOT of tweaking to get it do run OK on a 3090ti, but I had to drop settings to medium and shadows and GI to high.

[@AlistairHume]

Yeah that's often the case for anything that runs Lumen. But with a bit of tweaking, baked scenes can look great and run pretty fast :) The real key is to disable basically all screenspace effects, even things like AO and Bloom, and have only one stationary sunlight. Then maybe you can afford an extra light or two for accenting, like a swinging bulb, or a flashlight. Finally, Virtual Shadows are amazing but slower. Oldschool shadows, at low(ish) settings for dynamic shadows will improve perf. Also, as a final step, I'd keep the Nanite pool to meshes of like 200k max, rather than a few million

[@musashidanmcgrath]

@@AlistairHume Thanks. I was surprised to see you using light mass here. I left the games industry a few years ago and I swore that never again shall I even utter the words light map UVs... 😂 I'm mostly using UE5 for personal work these days so I'd rather avoid baking lighting. Great tips though.

[@MrLozzinator]

Super interesting, great video!!!

[@VRDivision]

It's so good to see you uploading again chief!

[@AlistairHume]

Wonderful to be back :)

[@GamerReality]

You brought up all good points as to why I'll stick with Unity for multiplayer VR gaming on Quest. I love the detail and narration in your videos as you explore a lot and specifically for the VR niche.

[@teawa_]

You should put VR in the title. Great video!

[@bassemb]

Happy to see you uploading yet. I'm still holding out for further improvements. And wishfully expeting an in-VR editing mode for UE5, better than the one in 4...

[@AlistairHume]

We can live in hope, haha! I think that there's plans for VR editor tools centred around virtual production. That could be a road in - I vaguely recall Epic talking about next-gen VR editor tools once, as something that they're thinking about. If not, perhaps someone on the marketplace wants to take up the challenge :)

[@MooseheadStudios]

Good video. Nanite lumen and VR are the tools iv been wating 20 years for

[@breakmaker7372]

Thanks for the great video! One thing you did not mention in your tests is whether your scenes were set up with forward or deferred shading? I assume deferred because I believe Nanite does not work with forward shading? Could you confirm that. I am in a situation where I am trying to optimize a scene like the factory one to work in VR. It does work on a high end machine with about 20ms latency but I would like to go down to 14 ms... I am thinking too many texture and draw calls like your suspicion... I tried Lumen too and as long as you use something like Nvidia DLSS on a highend machine it works OKish but crashes quite easily... I too think we are far away from dynamic lighting in VR.

[@MultiverseMoo]

Does make me wonder if some of the challenges are why Firewall Ultra have delayed their big gameplay trailer (which was supposedly "done" a while back but missed the PlayStation Showcase). They are using UE5 but not sure if nanite is part of the equation. They are using eye-tracking in the VR2 for different things, so I'm assuming that includes eye-tracked foveated rendering. Wonder if that could somehow work alongside nanite (if its fast enough).

[@importon]

Nice to see you again. Can we see your results of using mostly baked lighting but a dynamic sun light?

[@TheBaldrickk]

Do you have anywhere that you publish runnable versions of your experiments? I'd love to be able to experience what it is you're able to make with this tech, even if it's unfinished or the examples are unpolished.

[@tome4161]

+1 on that, share the demo .exe want to be immersed also

[@thejesterserrand]

Came to learn about Nanite and VR, stayed for the sunset.

[@2dozen22s]

Would be interesting to see an auto culling probe, so any triangles that won't be seen by the player doesn't need to be stored in the final pool. You'd probably need to hand build meshes to act as volumes where the probes will spawn and traverse, and any registered triangle would register anything within a certain radius to fix small occlusions. But it could be interesting for really large scenes/scenes with many overlapping meshes?

[@patbocchicchio]

Fantastic video

[@mantas3d]

Very nice sunset. So for that scene u had lumen on 2080 and fps was low. But if its just nanite and no lumen? Does it run 90 fps? Have u tried? And sorry if its stupid question, iam a newby in ue and havnt even touched 5 th version.

[@TheUriahFoundation]

Just wondering if the eye tracking used for foveated rendering in the PS5 could be used for discreet nanite rendering.

[@FarhaanLevy]

Here before you reach 100k subs

[@eldridgefelder3427]

I didn't notice, but are you still using Lumen when you baked your lighting?

[@erictrinque6513]

Great video, I have a small constructive observation. Think your bass balance on your master is a touch to high, you have a bass heavy voice and on stereo and headphones it gets a bit muddy in the vocals. Just my experience. Keep it up bud

[@a.aspden]

great video but music is too loud

[@robertadams3925]

Do you still have to use the forward renderer or are they working to optimize deferred rendering?

[@brettcameratraveler]

Been waiting for this exact video! So, when it comes to Nanite/VR, if one wanted to increase draw call speed and have even more variety of models in the scene, would you recommend building a system with a GPU with more vram? (Roughly how much would probably be enough for almost all scenes?) Would faster or more DDR5 ram be helpful over DDR4? (How much?) Same system spec questions but now throw in Lumen as well.

[@halycon404]

Ram isn't the bottleneck for games engines really. DDR4 or DDR5 will be fine. If I were building a system right now I'd worry less about what type of ram, and more what type of ram the board uses. I'd end up going DDR5 so I'd have a future proof board that could handle upgraded components in a few years, not because I want DDR5. 16gb is enough for most use cases, but now days I'd run 32 just in case. Though I do have a machine with 64gb that's more work focused than gaming focused. The big thing is VRAM. We're seeing a generational change right now because of the new consoles. Anything built to run on the PSVR2 that cross platforms into PC you're going to need a card with 16gb of vram. Just assume that's the minimum for all games going forward, people will say 12 minimum but it's going to end up being 16. Here's where it gets problematic. There are very few cards on the market with 16gb or more vram. Nvidia makes a few, AMD makes a few. And they are all the high end. Currently it's about a 800-1200 US for a card with that much vram depending on the deal you can find. Which is just not worth it. Nvidia will have the 4060ti in a couple months, the 16gb version will drop to.... 600 bucks. But that's in a couple months and there are no actual reviews to know if it'll be good or not. People are saying performance will be on par with a 3070, just with more vram. Who knows. The take away is that this particular generation of cards is a wash. Unless you wanna buy a 4900, 4080, 7900xtx, 7900xt, or a 7800xt... don't buy one. Nothing in Nvidia or AMDs pipelines for this generation currently is going to be enough at anything approaching a reasonable price. They both got caught flat footed by the explosion of VRAM needs. The 4060ti could be, maybe. But I would not be surprised if the next generation cards have 20gb as the bare minimum across the entire spectrum of both manufacturers. We're in a weird limbo spot. Upside, this has happened before. Last time the consoles got a major update this also happened. One generation of crap cards anyone with a brain would actually buy that sucked, followed by a huge vram jump across the entire lineup. Things got better. So. Shrug. Most people are not buying GPUs this generation. Nvidia and AMD are seeing a big drop in sales for the consumer graphics market since everyone knows this generation is not worth it. Noone without huge disposable income is buying. Everyone just decided it's better to wait 2 years.

[@brettcameratraveler]

@halycon404 Excellent buying advice for those building a system for next gen gaming. In my case, I'm not a game dev trying to ship a game that will work with a majority of systems I'm the coming years. I'm doing local, one off VR experiences and virtual production. With that said, I can max out a high end systems specs. With that said, are you saying the speed of a draw call (and thus the amount of unique nanite meshes) is sticky bottleneck by how much vram is available - and that system ram (or SSD speeds) are not the limiting factor?

[@halycon404]

@@brettcameratraveler draw calls are interface lag. An NVMe drive will help if something needs to be loaded from the drive, but there's built in lag across the PCIe bus that cannot be gotten around. Well, it can.. but the technology is new and no games exist to support it fully. 5.0 spec for NVMe is out, don't buy it. Outside of professional use cases nothing exists to take full advantage of NVMe 5. Games probably won't for a few years, and by then the price will have dropped to something sane. Stay with 4.0. which is pretty much the standard anyway if you go buy a drive. VRAM is because textures and effects are up across the board. All that stuff eats graphics card memory. With new games you're going to get textures that don't fully load, which is seen in this demo in places unless you have loads of VRAM on your graphics card. The cards just can't handle the new texture sizes. We're in a wonky place right now, everything is kinda in transition.

[@brettcameratraveler]

@halycon404 Really appreciate it. I'm working with a 4080 with 16g vram. Will just have to do a lot of "real" world environments and find where that 72fps edge is in all directions.

[@vvrr6dif]

Keep me updated on VR stuff please🎉🎉

[@FlockersDesign]

Man i made a video on this working a month ago already with mega scans and man it ran amazing

[@lolpl0000]

this was, admittedly, pretty awesome. :)

[@adr2t]

Yea I feel like nanite has a lot more headroom we just havent tap into yet. DFR + Nanite I feel has a ton of performance goals that just hasnt taken place. For example, LOD effects could be set also taken into effect where the user isnt looking either vs always rendering full detail because you couldnt always know where the user was looking along with distance LOD. This is to say, this would add or work on top of FR as they can still be render even lower than that. Same with lighting as you don't have to send as many rays outside of the retina area with the "lower detail" areas getting half or even a 3rd of the rays. A lot of this can be tested with FFR so when DFR comes into play it can be there. Does the nanite mesh scalling work with DFR as well? IF not, again, more performance could be gain from that with only the retna area getting the best detail with everything else getting a lower mesh count. IF you did all this I wouldnt be surprise to see a 100% performance uplift or even double the FPS we get now. Less rays, less mesh points, more LOD, less objects, lower texture qulity/memory foot print, etc etc all while adding all that back into the center for the highest detail its all there if we add DFR.

[@danielkissgremsperger3242]

Lets hope we get hdr for vr to really make a visual diffetence

[@MrLarsalexander]

The one cliff on that scenne you made with the city in background looks to need the two sided material turned on for the shadows to work proper... Not very found of that still beeing a issue on megascan or photoscanned objects of that quality.

[@NicoleSummer8]

Awesome 👍

[@darmelli954]

Wow, where did you get that arid mountain range where you where changing the sunlight on at 8:37 That’s beautiful

[@AlistairHume]

Its a Megascans asset, which means you should be able to get it quite easily :)

[@lipo9499]

i think eye tracked foviated nanite might be the ticket

[@jollygoodshow]

Make some VR demos! I miss the DK2 days with all the fun demos to try

[@thedudely1]

Very interesting

[@hellerart]

Can Nanite handel some basic animation? Could it for examply calculate slowly floating rocks like in Avatar? I mean static geometry, not deforming, but rotating, moving along axis? Would this be possible? Or can the nanite engine only deal with not moving objects? Of course I understand it does most probably NOT work with character meshes, that deform.

[@isaacsnediker-morscheck3382]

Nanite supported non-deforming dynamic objects since its first release, and now has experimental support for world position offset vertex animations. Nanite itself doesn't really care whether an object moves or not as long as it doesn't deform.

[@hellerart]

@@isaacsnediker-morscheck3382 Okay this is wild...

[@Gear_Snack]

I take it for characters with skeletal deformation using nanite, you'll still need to create your LODs manually and with care to preserve the look of how the mesh deforms.

[@Secret_Squirrel_Scottishgamer]

im waiting for the vr accessory where they release a hearing aid style usb port that you plug into your ear...

[@triggerfishgames]

Nice video! We have converted our last project over to UE5.1.1 and right off the bat with all the new settings for nanite, but without lumen, the gpu and cpu framerate is about 2x less than what it was in the previous version! (UE 4.26) This is fairly disappointing to say the least, and we are really hoping that the situation soon improves....

[@hyl]

Isn’t frame rate doubling a good thing? Or are you talking about frame times? Or were you expecting a higher performance increase?

[@triggerfishgames]

@@hyl Hi there. Sorry, yes you are right. 2x less is what we meant to have said (now edited). Thank you!

[@ThaWoundedFox]

What gpu does this creator use. Curious about limitations.

[@SinanAkkoyun]

That deserves a sub

[@SinanAkkoyun]

Would've loved FPS measurements though

[@MetalGearMk3]

What about the nanite fallback mesh artifact?

[@SimplyStream3D]

Great video. Thoughts on WebXR?

[@AlistairHume]

Good question! In general, it really excites me. I've tried a few pixel-streaming demos, where a powerful server renders PCVR content to my Oculus Quest. It worked beautifully, and would solve the standalone power issue. However, that comes with drawbacks too, like no longer owning one's own hardware. And in the long-run, most subscriptions end up more expensive than buying things outright. So I'd only go that route if there wasn't an alternative. As for WebXR natively, its fantastic! The challenge is that it isn't really supported, and it isn't a very efficient way to deliver large experiences. I've seen some really cool stuff with it, but if we used that as the main delivey pipeline for VR, it would need a huge amount of work

[@BHBalast]

Thanks

[@G0ldmoon]

you know, I feel like that SUN set could make a Nice VR benchmark?

[@expooll]

Great video! What is the song in the first minutes of it?

[@AlistairHume]

Abstract world by AlexiAction :)

[@BeardGuardian]

Have you looked at RTXGI as a possible lumen replacement? I figured it should be more VR friendly since it should be baking the results to probes rather than fully compute them in real time

[@AlistairHume]

Sort of, though I haven't tested it yet. For now, I'm not sure that it works with Nanite, so I'd love to see that get hooked up one day. But as a future pipeline, I think something like RTXDI (for shadows), RTXGI (for bounce) and Nanite, would be so powerful. I'd guess that by the time we reach that, everything will have new names - but that's what really excites me about next-gen games :)

[@ibrews]

@@AlistairHume RTXDI does work with nantite and lumen. In fact, NVIDIA discontinued development of RTXGI bc they decided Lumen basically does that just as well if not better at this point.

[@AlistairHume]

@@ibrews Oh wow! That's very interesting! How would that piece together? RTXDI instead of virtual shadows, Lumen for GI and Nanite geometry? Very cool if so! Also, how have you found TSR for VR? I had to run it at full resolution, and didn't see any clear path to getting upscaling benefits. I also found it didn't work at the same time as Foveated rendering. If you've found the golden combination that I've missed so far, would love to hear :) Big fan of the rendering experiments you've been doing over the last few months!

[@IRONFRIDGE]

Hi Alistair, what you think about 5.3 is vr getting Nanite now for real?

[@AlistairHume]

I'm testing it at the moment at bit - its starting to get really good! I'll have to do some detailed stress tests a bit later to find its limitations and performance guidelines, but its already unlocking possibilities that weren't there a couple of years ago. We're probably getting close for teams to make a call that VR games can be made with nanite, with a view to ship on 5.4 or 5.5, etc, as it updates. That's the view that we're taking internally :)

[@tatianaes3354]

Can you try this in Unity? They have released some major updates in the last couple of years. Also, they collaborate with Apple, so Unity projects will go easily to its headset.

[@saxelectro]

Very good video! What's your VR set recommendations?

[@AlistairHume]

Hard to say, as we're sort of going through a generational transition at the moment. Quest 3 is on the horizon, Apple is about to do something crazy, maybe a sequel to the Valve Deckard, etc etc. But as a couple of recommendations: Best all rounder: Quest 2 (or Pico as an alt) Best PCVR headset of the last 4 years: Valve Index (but its now getting a bit old) Best new VR Headset: PSVR2 probably, but harder to develop on unless you're a bit more set up Best wildcard next-gen headset: Bigscreen Beyond Upcoming headsets that keep me up at night: Valve Deckard Quest 3 Apple XR Anything by Varjio Somnium XR (though its still very unclear if it'll be good) Pimax 12k (probably) If I could only have one headset, it'd be a Quest 2, Quest Pro or an Index. I still develop on the Index as my go-to, but I'm looking forward to what's next :)

[@saxelectro]

@@AlistairHume Thanks a lot for the info!

[@RM-xr8lq]

this is how the illuminatee designed the moon landing

[@Imhotep397]

When you say “optimize” are you meaning LODs? Also, can you combine LOD with Nannite even though I know nannite was designed to eliminate the need for an LOD pipeline? It kind feels like that’s where the rubber would meet the road for high speed photorealistic VR.

[@AlistairHume]

Optimize is quite a broad term, in general. As a very basic overview, I would optimize shaders (texture size, shader complexity, number of shaders per mesh), post processing (reflections, AO, bloom, etc etc.), shadowcasting, particles, etc etc. I'd also make sure that despite nanite, I only have a certain number of blueprint actors in my scene that contain code. If I need loads, I'll only load them when I'm near. There's also dozens of CVars that you can use to tweak performance and final frame settings for specific use cases. There's also VR specific optimizations, like Stereo rendering or foveated rendering, and forward shading as a separate rendering pipeline. In some use-cases, forward shading is very efficient, but I typically prefer deferred rendering for PCVR (some find that controversial). In past engine versions, LODs were one of the main ways to optimize the number of triangles being drawn and rendered per frame. However, with Nanite, the LOD system for about 90% of your assets will be replaced with on-the-fly LOD generation from a shared pool. Every nanite mesh also has a fallback, and that can have LODs in case you are shipping for a platform that doesn't support nanite. Also, if you're using baked lighting with nanite meshes, you need to make sure that the fallback is close enough to the nanite mesh (you can tweak the detail in the fallback generation), so that the bake lines up with the nanite mesh properly. If you have skeletal meshes, that won't be rendered through Nanite, so you'll still need to think about LODs for those. For very massive scenes, even with Nanite, you may want to think about HLODs, to bake large proxies for far-away meshes. So in general, I find that optimization is a crazy dance that combines art, technical art, a bit of science and a lot of trial and error. As a final note, now that we're getting more into photorealistic VR, we need to think about different techniques to the ones that got us through 2016-2021. A great example of this is MSAA, which no longer holds up for super-complex environments. So we're in a strange spot, where AA techniques need to be rethought. TSR would be ideal, but its a little laggy. I find that TAA, with a lot of CVar tweaking, is the sweet spot for now.

[@TiagoTiagoT]

If performance in hurt by number of individual meshes, would it be possible to merge the whole map into a single mesh to bypass that?

[@AlistairHume]

Sort of. That's actually what Nanite does, in a way. But it then only draws and renders parts of the combined mesh-pool as needed. There's more to it that that, but it basically combines mesh draw calls into a single pool. But if there's a lot of data in that pool, you'll still be bottlenecked by trying to draw from the pool, as seen in demos like the Matrix Demo. Also, other things contribute to the frame time. Dynamic shadows, shaders, etc. Usually merging the meshes the old fashioned way will make performance worse. But so will splitting up the meshes to finely. There's a balancing act, that combines instancing, object size, draw calls and culling volumes, to get a smooth running game. Then you scale the detail, rendering features, textures, etc depending on platform target. All of that still applies, except that now Nanite handles the meshes for you. You have a lot more budget (like a looot), but its still not infinite :)

[@TiagoTiagoT]

@@AlistairHume Hm... How much would need to be change in the way things work so that you can have a predictable consistent maximum performance impact based on the smallest triangle, measured in pixels, you let the engine use, regardless of the complexity of the source 3d models, number of objects in the map etc?

[@odiseezall]

2024 will be the year Cyberpunk will run well at High/Ultra settings in VR on a ~800$ video card. Great time to be alive!

[@dreamphoenix]

Thank you.

[@MusicByNumbersUK]

Would love to see what you could do with a 4090! :)

[@borisdubois]

It would be nice to have a new test with the same scenes with UE 5.3 🙏

[@AlistairHume]

We're working with 5.3 at the moment, and it seems very promising! Will post more details on it soon :)

[@EnzoBergstrom]

I know its not most peoples favorite thing. But what about using syncronosonic space warp? Oculus pc has a very good version of this, aswell as virtual desktop and steamvr. Then it only requires 45fps to get an 90fps output.

[@jaxxtrend]

SSW has limitations and strange artifacts when these limitations are not respected

[@saimon1987]

Using stationary skylights in baked scenes makes no sense, generally, and delivers lower quality when compared with a static skylight. In fact, we, Archvis artists ALWAYS set the skylight to static when baked. And thats for a reason.

[@TheFunDimension]

I wonder what would be the case is three years. Would it be finally ready to be used on standalone headsets? Otherwise, it would be such a shame.

[@AlistairHume]

Hard to say. Alledgedly, third party devs have gotten Nanite to run on the Quest. But realistically, the Quest 2 doesn't have the power needed to account for Nanite's base cost. Furthermore, some platform architectures don't support Nanite, like Apple's M chips. Which means the new ultra-powerful Apple headset (allegedly) won't benefit from Nanite either. Maybe after a few years of optimization and chip changes, Nanite will run everywhere, but we'll need to see a power increase from Standalone for that to be the case, IMO :)

[@SkeleTonHammer]

Hm. Temporal effects are a big no-no in VR in my experience (I work in Unity though, not Unreal). For me the AA answer is MSAA or none at all. TAA is very poor, worse for VR. Just too sludgy. I'm currently creating a PCVR game that is quite ambitious (hopefully I won't have to tell people it exists for long, word of mouth will hopefully be quite explosive for popularizing it soon) but I'm finding that there has been no "magic" method for getting everything to look great. In a way, I have to go back to basics; I use VERY few full-screen effects. Far fewer than you would in any modern flat game anyways; I do some exposure correction and a mild color grade, a very, very subtle bloom effect that only affects very blown out light sources, and a high-performance HBAO solution (with no temporal AO accumulation). I do this so that the resolution can be as high as possible while the game still runs well, and then I put a mild 2-4x MSAA on top. Of course, we have to save VRAM elsewhere too. From there, it's a lot of very strict, disciplined, and... well, 15-years-of-game-dev-experience'd approach to my assets. I crunch my UVs together as tight as mathematically possible to squeeze out as much clarity from as little VRAM as possible and so forth (we run a heuristic on every UV unwrap in UV Packmaster to maximize texel density, any UV that has space will bunk up with a smaller object that will map onto it and share its material, etc.), I emphasize texture layers that have the largest effect on the level of surface detail (so we deprioritize loading color mips, and emphasize mip loading of things like normal maps and roughness maps, because we find being able to see surface imperfections/smudges is so much more valuable). Then we do some other things we normally wouldn't prioritize in flat games, for instance in our project the rule is there are NO SHARP EDGES ALLOWED. ...Unless it's supposed to be sharp, like a knife or something. We bevel all edges, or if an object is too small and intricate to be beveled, we bake tiny bevels into the normal maps by baking a normal map in Blender Cycles using the bevel node. Perfectly sharp edges just look so much more obvious and worse in VR, like you're seeing a world constructed out of folded paper. For this reason, we don't use any "fake" depth for any kinds of severe depth, just normal maps to show fine texture and surface imperfection. If we have a raised area because there's a screw head or something popped up, we need to model it, because its lack of true depth is emphasized in VR. Half-Life Alyx remains the best-looking VR game, and it came out in 2019 and runs great. So, that gave me a lot of hope that great-looking VR might just be a matter of restraint and intelligent decision making. So that's what I've been doing. We have a small-ish playable cyberpunk-styled city (about 4x4 city blocks) with fully playable interiors and so forth all running on a 3070 Ti at an easy 90-120 FPS. We utilize some other little tricks too, like the game exclusively takes place at night (every day the player heads out at like 10 PM to do their business, we've woven the reason for being out at night into the plot just to get this performance boost), so we don't care much about day/night cycles and needing dynamic environment light or anything like that. And we're definitely not even making like, an uninteractive walking simulator. We're making a very, VERY interactive immersive sim. We're just picking our "battles" so to speak to get great visuals on VR.

[@Mac_Daffy]

OctaneRender has released their first implementation of LightMass for their UE plugin. Baking textures reliably in Octane quality and with multiple GPUs should be a things in a few more releases. Going to sub your channel to see if you want to give it a try.

[@grahamulax]

ah man, wish we had lumen working! I do have a 4090 (for ai training. NOT gaming lollllll) and I didnt even think to test it out. Your methodology is great though. Always be efficient even if you do have a beefy machine is my rule. Theres lots of settings with nanite stuff too! Even fall back models that you could theorhetically create a low poly mesh and bake the textures onto, but then again that would be another material and object slot..... AH! What a puzzle, which is fun :3

[@Borszczuk]

What music is @4:00 please?

[@Zoddom]

whats that track id from 09:00-12:00?

[@gamingdayshuriken4192]

thx ;)

[@victortortelli9992]

the only way VR will be mainstream is when they upgrade mainstream games to VR without many work, that's the most important thing to it!

[@brett20000000009]

image quality is taking a huge hit for all this stuff

[@Kavukamari]

my question about nanite is, if we're using megascans for our assets, doesn't that mean that we have to raise the filesize of the game as well? if there's no LOD pipeline, we're just serving huge assets...

[@JorgetePanete]

You can choose the precision of each model

[@e-frame5344]

Are you going to do an update for this in UE 5.3.1? :3

[@AlistairHume]

Just posted one :) VR in 5.3 is really good!

[@e-frame5344]

Guess what I was just watching xD @@AlistairHume

[@BusinessWolf1]

One thing I don't like is that no one cares about making tools for making games that (actual) low end systems would be able to play. Even with the power increases in hardware, mid level components aren't really getting cheaper. We need to start consolidating and optimizing, making performant tools from the ground up.

[@quackcharge]

can't wait to have 50bil polys and real time pathtracing in 2 years :)

[@Hotte1967]

The Marketplace things allways have a licence for use in Unreal Engine Only. Is it ok to make Videos with this?

[@AlistairHume]

To my understanding, as long as the videos are made using Unreal Engine, it meets the criteria of the license. Its more about using it in other software to render

[@Hotte1967]

@@AlistairHume OK thank you for your answer.

[@tomtomkowski7653]

All we need is a 6090 (maybe 7090) manufactured with 1nm non-silicon-based technology TSMC and MIT are working on right now :) Imagine... 5x more transistors within the same area so 3x-4x faster GPU than today's 4090 in a couple of years from now.

[@xdmcnugget5411]

I know we say it looks almost real every time something cool happens since 2000, but are we not almost at the point of realism now. but really this time?

[@TheTrumanZoo]

360p?

[@AlistairHume]

4K on the way :)

[@TheTrumanZoo]

@@AlistairHume nice :D cool video in any case, nice.

[@avinashlobo]

Consider not speeding up the parts where you're navigating 3D space. Watching it on a big screen is a recipe for a headache. For someone concerned with a good VR experience, this should probably slot into the same ballpark.

[@nikko3d]

Did you run this on an rtx 2080?

[@AlistairHume]

Yes indeed :) RRX 2080, which has approximately the same performance as a PS5 or Xbox. So a good development machine if you want to target different modern platforms

[@MrDj200]

"90 FPS minimum" > me chilling in vrchat with ~20 fps take some notes vrchat devs. smh

[@Mnerd7368]

Nanite and Lumen will not happen for several years in AR/VR/MR 😂. Maybe 2030?

[@Neosin1]

Wait isn't lumen working in vr?

[@AlistairHume]

It does work - its just very expensive to run, so you'd need a very powerful GPU to run it properly. Most videos that have VR Lumen demos on KZbin use RTX 4090s. It means it should work in the future, as consoles & PCs get more powerful, but not for this generation

[@walker1583]

What alway's bothers me when seeing VR clips is why doesn't the character have arms/legs, but only seeing hands or weapons floating infront? Besides that awesome video

[@noergelstein]

Because they aren’t tracked. The only thing you reliably know is the head and hand positions. You can try to estimate the arms, but without tracking the elbows, they aren’t accurate either. For legs you have basically nothing to rely on, its pure guesswork from the height of the head.

[@walker1583]

@@noergelstein It is actually possible though it's seems more that the cost is way to high for the consumer. This video does explain it abit. kzbin.info/www/bejne/qWXRhKeAopqHj68

[@cr0uchingtiger]

My biggest disappointment in the VR world is VR Chat. Might be a little off-topic, and it uses Unity, but the level of immersion and the social aspect (I met my fiancé there and we're getting married in 2 months) has so so much potential. But when you say you have to aim for 90 fps, man, we would be getting about 35 max. I used to dream that it would get better over time and we'd have a Ready Player One type virtual future, but it's just not happening.

[@CasualG-mer]

UE5 for VR is pretty exciting, graphical wise but Source 2 is the better engine for VR. Games like Half-life:Alyx, looks good enough visually but the amount of physics and interactivity makes it so immersive.

[@zebius4157]

Did you know use can use nanite with chaos physics? It's all there, someone just needs to advantage of it

[@tatianaes3354]

Also, do not be shy, speak a bit louder. You can do it! 😊

[@user-gu9hy9iy5u]

круто

[@Torrente70]

with this engine you could bring google maps into vr

[@bradleymcwilliams3139]

You can already do that now with the latest Cesium update :) they added Google Maps API

[@tickledonions9483]

The resolution of everything up close is already terrible in Google Earth so...

[@Torrente70]

@@tickledonions9483 that would be the job of the KI

[@QuentinBrutus]

rtx 7090 might be able to run this shit in vr if we are lucky. And its only because a 50% increase in Calc performance means 20fps more which is ridiculous.