How to Make 360 VR Environments for Quest with AI - Stable Diffusion and Blender Tutorial 2023

[@albertbozesan]

To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/AlbertBozesan/ . The first 200 of you will get 20% off Brilliant’s annual premium subscription.

[@trinsic6652]

I could see this working really well with environments and scenes created based on a novel, with the audiobook accompanying the user. Basically listening to audiobook, while in VR with a visual environment based on the novel.

[@albertbozesan]

Totally!

[@alecubudulecu]

Awesome vid. Can’t wait to try it. Quick note. Suggest doing inpaint before upscaling. Works much faster

[@albertbozesan]

I suppose, but then you’ll lose resolution in the details and have to paint those back in anyway. Would you say the benefit outweighs that?

[@sarpsomer]

Clean tutorial without cutting or skipping the important parts. Thanks.

[@albertbozesan]

Thanks! I’m glad I managed to keep that balance.

[@ArtfulRascal8]

Mind absolutely blown. Back in august of last year, i had no idea this would be coming true so fast. Thanks for the tutorial!

[@taserface8147]

Just a suggestion, to make things better, skip the depth map creation initially. Upscale the image and make the final tweaks, then create a smaller size version of it for depth map. this way you can have more room for tweaking.

[@Iskarioto]

I started to learn Blender from 2 weeks ago just for can use your Environmake addon, Nice work!

[@albertbozesan]

Wow! Hope you enjoy it 😄

[@Iskarioto]

@Albert Bozesan Much appreciated 😅

[@taavetmalkov3295]

hello. Good tutorial! i was wondering if i can take the generated space image and somehow base the next image on the previous one so that the details remain the same,,, for example if i want the same room but a view at the side of the couch...

[@albertbozesan]

Not precisely, but you can check out my latest tutorial to get more control in rooms. You can also research “IPadapter” to get similar images.

[@Raulikien]

2023 is wild, so much new amazing tech

[@wedgeewoo]

this is a great workflow

[@albertbozesan]

Thanks!

[@unheilbargut]

A very cool method again. Thank you Albert, this again is a perfect, cool, unique tutorial! You rock! ✊

[@albertbozesan]

Thank you!! I try to keep my ideas non-obvious :)

[@TrentSterling]

Wow; I've been generating skyboxes for a little VR indie game using blockade-labs and I had no idea there was a 360 Lora available for SD! This will make fine-tuning so much easier. 😃

[@Nanotopia]

Great tutorial! Thank you for sharing 🥳 I wonder if you might know of a plug-in for VIVE/Index rather than Quest? Meanwhile, I am Googling that now 😁

[@albertbozesan]

Thank you! Not that I know of, I only have the Quest 2. But I'm sure there must be an option out there :)

[@fifasglitchesita8614]

thank you, really, to share this. you gained a new subscriber.

[@albertbozesan]

Thank you! I'm glad it was helpful.

[@erank3]

Great tutorial, you have great design skills 🙏

[@albertbozesan]

Thank you!!

[@yorkewu9946]

Awesome video, can I achive the same using unreal engine? any diea thanks a lot

[@albertbozesan]

I’m sure that’s easily done, but I’m no expert in that engine.

[@yorkewu9946]

@@albertbozesan Thank you~😇

[@elisabethfischer873]

Hey, thanks again for the great tutorial! How would i send this file to someone so they can try it out on their Quest without having to build the environment themselves? Is that possible?

[@MrMahoark]

LoRa means additional parameters that come from a finetuning on top of a model, so they cannot be used with any model, right ? just the one on top of which it was finetuned

[@albertbozesan]

LoRas can be used with any checkpoint as long as the base model is the same (SD 1.5 vs SDXL for example). But it’s correct that the checkpoint a LoRa was trained on is the one it works best with. This is why older LoRas in general work worse and worse on newer checkpoints.

[@ReinBijlsma]

Excellent video! For some reason my panorama viewer doesn't allow me to rotate te screen. It just shows the generated photo, but I'm not able to pan the screen. Any clue what i did wrong? Everything else works fine.

[@jasonjmp]

Thanks for this so cool! Any chance we can see examples anywhere that can be reached from the Oculus browser? That's probably another video. :)

[@albertbozesan]

Thanks! What exactly do you mean? What do you want to see in the browser? 😄

[@kiksu1]

Very cool! (I see what you did there with the word FUNCTIONAL 😂) I don't use autoupdate, thank god.

[@alexxa345]

Great video! Maybe you should try the skybox labs as a 360 generation method. I feel as it may simplify the process even further.

[@albertbozesan]

BlockadeLabs ist excellent, I just feel like figuring out and teaching people how to do it independently on their own computer is more valuable in the long run 😄

[@alexi7941]

Great video! Qq, is this seamless in headset with the end result?

[@albertbozesan]

Horizontally, yes. The top and bottom aren’t very pretty.

[@craceyu]

Great tutorial and so helpful! Would you like to publish your home to MoonVRHome where people can install this custom home in VR?

[@smartin-g2u]

Great video! Can you recommend the steps it might take to convert this environment into one you can use in the Unreal engine so that a player could navigate within it?

[@albertbozesan]

The .gltf I export towards the end should import perfectly in any game engine :)

[@alyx918]

when trying to generate on the enviromake tab i get error bpy_prop_collection[key]: key"specular" no found. Any known fixes?

[@LacyPerryArt]

I am getting the same error

[@MarcoSilvaJesus]

Thank you very much for this class. I have a question, what kind of minimum equipment is needed to run this application locally?

[@albertbozesan]

For okay performance you will need a Windows PC with at least 8GB RAM, an NVIDIA GPU with at least 8 GB VRAM, and as much hard drive space as you can (models are quite large).

[@나나-l9l7p]

Hello! I really like your video and tried to follow through the process and I have one question. When creating a skybox image as a scenery, the tree is too large and appears to shrink in the upper part of the image. Is there any way to fix this problem? I used the Asymmetric tiling + 360 Lora. Thank you :)

[@albertbozesan]

Thanks, glad you enjoyed the video! The way to fix that at the moment would be manual editing in photoshop, using generative fill and just classic image editing techniques. There’s no shortcut quite yet - the top and bottom of the 360 image is just bad for now.

[@나나-l9l7p]

@@albertbozesan I see! Thanks:)

[@cosmicbrewery]

Is there a way to add an environment sound?

[@albertbozesan]

You’ll have to ask the people who made the converter in their discord.

[@cosmicbrewery]

I need help! I think I must have a setting wrong. All my images are looking like a bad oil painting. I have the lora and the model you suggested and the sampling method DPR++2M Karras. what am I doing wrong. I want ultra realistic looking interiors.

[@albertbozesan]

Sounds like a more general Stable Diffusion problem unrelated to this specific workflow. Maybe something with a VAE, or using an SDXL model with an SD 1.5 resolution. You could post a screenshot on Reddit? It’s easier to help there.

[@visualdrip.official]

the blender startup file is no longer available :(

[@albertbozesan]

Ugh, that’s a bummer. I have an updated version of this vid on my list but it could be a while. Working on a longer course right now.

[@cr4723]

I have installed the depthmap-script and restart the UI, but i have no depth tab. What did i wrong? Any solution?

[@albertbozesan]

Did you make sure it’s active in the extensions > installed tab?

[@cr4723]

@@albertbozesan Yes, I think it is a bug. Sometimes it is there sometimes not. Thanks for the video!

[@thisguyoverhere8595]

I have a brilliant idea. How about creating an ai program where you type in a paragraph to describe the environment that you want and the ai creates a vr environment just like how the ai art generator works in the Wombo dream app.

[@albertbozesan]

I mean, that’s kind of the ideal scenario of this. With an idea like that, it’s all about the execution and being early 😄

[@sisyvusvr]

Moon VR Home on Quest Applab does that afaik.

[@Hambxne]

Welcome to the singularity y'all, is it everything you hoped it would be?

[@alecubudulecu]

And more!

[@elisabethfischer873]

Thanks for this tutorial! I have one problem, when i click Build and install environment, it says that the APK build was finished, but it doesn't automatically launch on my Quest. I can find the app in the Unknown sources, but nothing happens when i click on it either. Any idea how to proceed?

[@albertbozesan]

It’s not an app - it can be found where your other home environments are. Settings > Personalization and then it’s probably at the very bottom :)

[@elisabethfischer873]

@@albertbozesan Perfect! It works now :D Also, i tried to use SD upscaler in img2img because it usually does a much better job than the ones in the extras tab, but it created a really weird image with a bunch of connected tiles. I guess it's not meant to be used for equirectangular images. ESRGAN didn't really seem to fix the resolution much. Any idea about alternative upscaling methods?

[@60tiantian]

Is it can't using on and gpa ?

[@albertbozesan]

Stable Diffusion only runs well on NVIDIA GPUs at this time.

[@60tiantian]

@@albertbozesan thank you for reply! your tutorials help me a lot, great work!

[@giraffeguitars6021]

So it won’t run properly on any Mac?

[@albertbozesan]

@@giraffeguitars6021 No, they don't have the required GPU. It will run very slowly in apps like Diffusion Bee.

[@FieldMajor76]

Any tips on how to create a video like this kzbin.info/www/bejne/mpPTaaeajM91eJo so we can experience multiple AI created rooms using youtube VR in Meta? Thank you for posting these informative videos!

[@albertbozesan]

Easier than the process I show here, actually. You don’t even need all the depth steps. Edit your 360/180 images one after another in any video editing program, and make sure you set the KZbin settings to 360. Any 360 video tut should help.

[@TransitionedToAShark]

Only cringe kids use VR. Ever seen the matrix? It starts here kids

[@albertbozesan]

I never thought I’d read “cringe” and “the matrix” in one breath.

[@TransitionedToAShark]

@@albertbozesan matrix is cringe

[@regal360com]

Seriously @@albertbozesan, it's hard to believe someone can be so out of touch that they are hating on VR and the Matrix while on a VR tutorial. Who are these lost souls with nothing better to do but spread negativity?