Unreal Engine 5.4 - Style Transfer / Neural Rendering Tutorial

[@erban.3d]

Incredible stuff. Thanks a lot for sharing.

[@behrampatel4872]

WOW ENCORE ! Will you be doing any more on NNE in the future ? For e.g use NNE for taking a webcam input and using Midas depth for making a UE object ( shader or particle system) react based on the distance to cam. Cheers, b

[@openroomxyz]

That's nice but is it possibile to get temporial stability and like 1080p or 4k output ? Btw nice video thanks for creating

[@nawhz6292]

it looks amazing but why is iot so blurry?

[@SaschaRobitzki]

Because the model's output is only 224x224. You can see that in the neural profile. You can add a super-resolution model to the postprocess to get a higher resolution output from the same style transfer model. There's a super-resolution ONNX model in the GitHub repository mentioned in the video. Unfortunately, that model is only a single channel model, so you have to run it once for all three colors in three passes. I just tried it in UE 5.5.0, and it works. Alternatively, you can use a larger model with three color channels from other sources (i.e., Open Model DB). I tried one as well. The quality is higher, but it consumes more resources.

[@nawhz6292]

@@SaschaRobitzki dangit bro. I appreciate your time and research. thank you so much.

[@hugomontanes880]

@@SaschaRobitzki Hi! How can you implement the super resolution model to the postprocess? I'm trying to do it but it doesn't seem to be working I'm using UE5.4

[@erban.3d]

Yet i have a question. I cannot render this even if its perfectly normal in veiwport? it crashes. any solutions?

[@SaschaRobitzki]

Did you try a different NNE runtime? What GPU do you have?