Oculus Quest - Hand tracking on flat surfaces

Transform any surface into a giant touch screen. While many of us spend almost all day in front of a table, there are no apps/games taking advantage of this providential armrest. @unity @Oculus #OculusQuest #MadeWithUnity #XR #SpatialComputing For more information: / 1218220146675732480

[@meowmaro]

Now THIS is "roomscale" VR

[@Kat21]

Your literal room in VR. amazing

[@oogityboogity6644]

I love this but I have animals and that’s scary

[@aseembaranwal]

@@oogityboogity6644 Actually that's AR

[@coolguy9036]

@i hate anime imagine calling someone a name to roast them but not even being able to spell degenerate so just shut up go cry to mommy that someone called you out for trying to shit talk someone who puts what they want on their account it's life sometimes it can be unfair and not everyone has the same mindset as you if you don't like something then who cares unless it's an actual debate so are you going to tell me that my pfp is stupid because by the looks of it you don't have one so go to mommy and cry you degenerate

[@trentoncahillvlogs938]

Actually house scale

[@Shinkaze33]

Oculus Quest VR is going to be what AR has always promised. Greg, your demo changes everything, thank you.

[@GlitchedPenguin]

Coming from the industry I think that's unfair. Over the course of the last 7 years we have seen large improvements in VR tech. However, nothing really for AR. The only use AR got was mobile game. Most VR systems use cameras, what would be scarier playing a horror game? Or walking around your own place with creatures imported into the world. AR has massive potential but isn't being developed because the attention is on VR.

[@robvh2]

@@GlitchedPenguin AR is being worked on every bit as heavily as VR. It presents much larger technical challenges though because you're trying to integrate two worlds in real-time while also being light enough to wear for prolonged periods!

[@axserwz5022]

@@GlitchedPenguin look at the new google Ar headset, trust me it's being worked on

[@purvdragon-sensei]

Jehudi Phillips >google LMAO

[@Maelifia]

Jehudi Phillips I wonder if it’ll be as bad as stadia.

[@wille3416]

less than 5 years until stuff like this is commercial, looks cool

[@jorisbonson386]

It's already commercial... where have you been?

[@anarfox]

@@jorisbonson386 The headset is. But very few consumers will take the time to model their own apartment. The concept in the video won't be a commercial product until the headset can map your living space by itself.

[@RandomPlayer717]

YOU COULD FINALLY HAVE A VR GIRLFRIEND !

[@happily7514]

Ah nice another way for facebook to make money off of my data

[@norkshit]

Kevin Pham if a service is free, then you are the product.

[@iamLucid]

What the actual f...

[@tak4shifwd927]

found ya

[@misgoatfr]

found you

[@aiden.m4a]

iS lUcId CrYiNg??

[@YourAverageSpelunker]

What's up checkmark

[@alexgreiser]

imagine putting something in the way irl and forgetting in vr that it's there, then tripping over it and landing face front on your quest... ouch

[@RandomPlayer717]

THE PADDING ON THE INSIDE SHOULD ABSORB ANY MAJOR DAMAGE TO THE QUEST 🥳

[@mrticklenuts2124]

@RANDOMPLAYER 717 I don’t think you know how physics works

[@RandomPlayer717]

@mr tickle nuts I don’t think you know how your mom works 🥳

[@xdC33]

@@RandomPlayer717 OH OH OMG OOOOO ROASTED OOOO

[@RandomPlayer717]

xd C33 SOMEONE CALL THE WAMBULANCE CUZZ HE GOT BURNED! 🥳

[@RicardoMenezes]

Hey Greg, this is a game changer! 2 questions everyone wants to know: 1. Did you tweak the real room to make it easier to model in VR? and 2. How long did it take to model the whole thing into VR? Impressive stuff!!!

[@SinanAkkoyun]

Of course he modelled everything by himself and then matched it with the real world, which is amazing

[@cardibeatzz7653]

excellent questions ..and your reply is....

[@loganadams711]

I need a tutorial

[@draketurcotte4760]

Sinan Akkoyun oh, I thought it was a program that just happened to match up perfectly with his room...

[@mataskart9894]

@@draketurcotte4760 oh don't worry m8, it probably took him weeks to map out if he was doing it in his spare time

[@RudySkaVR]

I would love to know more about how this was done. The Mapping process, and programs being used. This is truly fantastic.

[@sroguthrie4438]

Rudy Ska VR its fake dingus

[@bilianab.7791]

I'm interested as well!

[@RudySkaVR]

@@bilianab.7791 From what I have gathered he used unity and blender. However this is just speculation mostly

[@hideofreakingkojima5457]

@@RudySkaVR Someone get Corridor Digital on this one

[@SkoolConnor]

@@sroguthrie4438 it's not fake dingus

[@DavidAddis]

Amazing! The dream would be that the Quest could auto-generate your room layout just by you walking around in it.

[@doughy_]

oculus is working on it

[@rajveerjolly7933]

@@notrutger leaks ? Tell me more please

[@brite7038]

rutger roughly when will the new quest be released?

[@notrutger]

B Rite nobody knows, definitely 2021 or 2022

[@Fwedada]

Maybe the next gen quest will be powerful enough to actually use custom avatars on vrchat... lol

[@keirandcarlshow]

This changes the game. This is the most applicable use I see VR and AR having in the future. I feel like this is going to become the normal way of experience. Possibly integrated with the rest of the world as a whole.

[@andrewochoa5100]

Look up neuralink!

[@mcatu3]

You know I always have thought that if virtual reality evolves to a point where it can nearly perfectly mimic reality, it basically would have created an alternate dimension

[@frostyjeff]

Ready player one ? Lol

[@Grandmaster-Kush]

DMT Beings: *Allow us to introduce ourselves*

[@mind1763]

Yeah imagine living a second life in the vr world.

[@lemon64k49]

ready player one had a brief version of what yo ujust said,at one point they were tricking the villain into thinking he was in real life,where the main characters where threatening him with guns,until he realized he still had the headset on

[@mcrmffaffriedyak]

Follow the white rabbit

[@dante7430]

This is exactly how I've imagined the future of VR will be like. Didn't think someone would do it so soon though this is awesome !!

[@KAIDAX]

I am more impressed that you created your room exactly in VR. That is actually really cool!

[@Haza3137]

Ohhh imgaine a vr app that can allow us to draw your room boindries then allow u to manually add in 3d objects,lights from an asset store. That would be sick

[@entropy7999]

YEEEEEE

[@Upmacher]

Hazem A google magic plan App!

[@BREIKO_1]

Oculus home? It exists, its very epic

[@franklydoodle350]

It does exist, although it's only for SteamVR. It's called SteamVR Home. Unfortunately it does not work on the quest unless you use oculus link or something like that.

[@Some1uNo]

can we get a tutorial ?

[@sblevy0229]

I could totally see this (probably painstaking) process of remodeling the entire apartment virtually being a 5 minute automatic setup process in the next few years. Could be really really big.

[@fyro9346]

your entire room is in a frickin sd card

[@jgperes]

Everybody gangsta until someone pirates your house

[@plastic6568]

JG Peres lmao

[@carriguerrero34]

JG Peres I’m dying

[@greengamerguy623]

@@jgperes lol

[@landonsims9153]

So you recreated the room in VR? Now that’s cool by itself! The hand tracking is a bonus!

[@alelexex]

Crazy If everything was scaled perfectly from fridge to stove you would be able to live a life in VR

[@adamfarmer7665]

imagine a world where everyone is wearing vr-ar headsets.

[@Art-re4ex]

This is honestly some game changing stuff for VR, would love to see a follow up video about the whole process.

[@thebestthereisthreads6849]

I’m more impressed that he made his virtual home look like his real home.

[@Gurpefy]

Oh my god. I would love to make my whole room in VR, then use hand tracking,. I wish i knew how.

[@dennisrichards2540]

By the looks of it you have made an accurate 3D model of your appartment to overlay your real one (great work) just think, someone is going to come along soon and write a program that uses the headsets hardware to automatically build a model of the enviroment your in (ie home, friends house, hotel room etc) and everyone could have access to this incredible feature. what a time to be alive.

[@OneGoogol]

You know what would be sick? A vr set that before you launch a app/game scans your entire room and puts it into the home screen

[@fredericfaddoul]

I had this idea when I bought my Quest a few weeks back, and seeing this video wants to me do it even more!

[@angrygroceries1]

Imagine a vr game that teaches you piano by lighting up the keys you should press to play a song.

[@sevn8757]

Or you could just get an electric keyboard with keys that light up... Yeah they already have them.

[@shadowmistress999]

I was expecting he plays the piano with the sheet in vr😂

[@brotherscro]

In the future VR with Lidar technology that would be great This is amazing :D

[@joemulkerins5250]

I don't like apple but the iphone 12 has lidar and this can be done with that and the oculus quest. I now want the iphone 12 😄

[@Xport3]

I always remember movies showing us how they pull up a screen and swipe it like robots and stuff and i can’t believe we can actually do that now

In an interesting note this could be done autonomously by photogrammetry by taking a lot of fairly simple photos of the environment. Oculus CTO Michael Abrash gave a hint this is being worked on, probably on nextgen hardware. (Quest has B/W cameras only) Future seems cool, since you could "invite your remote friends" to your room.

[@bo_yt]

Where did he give that hint?

[@osterulio7933]

waaaaaaw dude how did u do that ?😲😲😲😲 is it a reconstruction?

[@GregMadison]

Yes , it is a reconstruction by hand.

[@mighty_vex1014]

Damn they need a game that maps out your room

[@Thegooob95]

Mighty_Vex straight up

[@paulcousins6535]

@@GregMadison What software was used to create the reconstrucion? Amazing job by the way, you've sold me with my quests capabilities by doing what youv'e done!

[@KimberleyB]

Quest 2 has a way for you to map out a desk surface and a couch in your guardian space, but this is over-the-top awesome.

[@MrLivingreality2012]

Amazingly Nice

[@chewyankles5783]

This is what the Oculus Quest deserves. Thank you.

[@swamppifi6186]

imaging having a plain room, with just the basic plain table, chair, and lounge, and then model it in VR, you could create all sorts of cool furniture and stuff in the VR version, you could have you house the way you want, without spending a cent, in VR, just come home, slip the head set on , and live in VR

[@Andy464020]

Amazing! this would be good solution for "boring" home office rooms. you just need a room (maybe in the cellar, or container in the garden) only hardware: desk, chair, a keyboard an a mouse (for the real office feeling) maybe a simple couch. and then you overlay all with this technology, and you can choose any view/look you want. Maybe you should think about a "all-in-one-VR-homeoffice-box" for the garden. 5x5 or so. it can also be used as a Quest gaming room. i would buy it instantly :-)

[@KevinFi747]

Are you using some external camera to capture the room or did you model everything yourself? I want to get into AR/VR development and have been wondering if there's a way to put the VR on and add things to the real world like you would with AR, but see them through the goggles as opposed to your phone.

[@shiBuyaking109]

Wow as long as they keep developing this! VR is going to have a bright bright future

[@amirlanbaltabayev8177]

Wow, realism is impressive! This CG technology seems very curious.

[@nisseproductions7052]

My question is just.... how did you manage to get the tracking on the oculus quest to be so accurate??

[@mataskart9894]

well it actually works acceptably in PERFECT lighting conditions, that being said his hands disappeared constantly because the quest lost them

[@MadeInJack]

@@mataskart9894 it lost its quest

[@Fab--]

The inside out tracking on the quest and rift s is actually really close to the quality of lighthouse tracking now, It's come a long way since the early days of the wmr headsets, tracking only falls out on controllers when they're behind your back and away from sensors obviously but other than that it's seamless.

[@Shiro_47]

Sooner or later, this is gonna go viral like everybody’s recommendation or Daily Dose.

[@beesong2729]

absolutely blown away by this honestly

[@FlorianEagox]

I followed this to a tee, and then I ruined my trackers when I washed my virtual hands and forgot that I was actually washing my real hands.

[@saltofearth4902]

This is seriously amazing! Are you available for a podcast interview?

[@freeminded8509]

It would have to recognize people ' objects and animals to be safe. That is the future for sure.

[@diamondxdragons]

I have a Rift S already but I definitely want a Quest still

[@Robman92]

That’s amazing! How was the room-recreation made?

[@BREIKO_1]

Using blender or some other app, then render it and export it to unity, ue4 or other, script and add little things for the hud, table map etc, render that and play it, i think its like that but not sure Edit: also scale it once you are on the engine, thats the difficult part plus making sure you did all furniture the right size Edit 2: also puting the furniture as the exact place as irl Edit 3: wtf man this is so impossible this guy is a genious

[@jonahlee5943]

В Я Ξ ノ К Ø ツ but how would you run it on the quest

[@jooaquin]

Such a great demo! Love your exact modeling on your house! keep it coming! I wanna see how it progress

[@snakeplissken3613]

I have ALWAYS had this idea!!! This is the BEST representation of it!!?? How long have you been a coder? Did you just make this with a dev kit and Oculus Quest!?? I think you have something big.

[@MyIronman8]

You give the whole new meaning of room scale

[@melvinluner1454]

Wow how I wish the next VR would have a lidar room scanner to map out ur real environment to ur VR environment

[@datboismurfy_4970]

The apartment is really cool, I like the windows especially

[@donediddlydoo6949]

Technology never ceases to amaze me..

[@unknownpaws]

Best idea I have seen in my life

[@Jona69]

Would have liked to see you try to play the piano with the slight handtracking delay.

[@MH-sf5ml]

Holy shit, can you imagine when these things are the size of glasses and all you need is a fake keyboard to unroll so your fingers can feel it! Amazing glimpse into the future.

[@ge2719]

This is what everyones home space should be.

[@fyregus]

Imagine people just sitting in their house watching tv in virtual reality instead of staring at their real one lol.

[@rolannebouchard356]

I heard somewhere that a Tony Stark like VR was going to come soon. I'm not sure if it's something to do with an Avenger game but it would be cool to have a Jarvis like experience.

[@vanda477]

It would be soo cool, if there was an app for the Quest, allowing you to walk around your room in pass-through, while creating basic geometry and placing it where real objects stand, just for reference. Then replace this basic geometry with more detailed virtual objects, as you do in PC's Oculus Home in order to make your own personal home, corresponding to your real surroundings.

[@angelduenas746]

All that's left is to remove the bulky headwear and replace it with a ear to eye head piece with special lens (SAO)

[@Ninja123546]

I can see this becoming a business in the future where you hire somebody to vr-proof a room in your house, or just your entire house

[@Nate4Life01]

very interesting and intriguing idea... but I'm not sure if I should hope it comes to fruition.

[@dm3402]

If you seein this.. This is the shit that we aim for, this is the future we were promised and that we will bring. May be just me, but this inspiration of a high degree. I want to know more, I want to help, I want to go in.

[@samanthacalhoun6820]

great concept for a home design that would replace oculus home and make your room your oculus home room

[@imamwahyudi4259]

This is Awesome.. If this virtual and reality link together, It can be the next level of Augmented Reality. Like more accurate for AR things in pokemon go. Imagine seeing a very well hologram, with a correct xyz axis and still aware of surrounding when you use your device or your vr glasses

[@inceptional]

See, VR headsets can do AR too with some cameras attached. I will always choose VR over AR for my fun, but I think this is a glimpse of the future where modern VR headsets also do AR perfectly well too, so you'll only need to buy one headset going forward to cover both experiences.

[@AkurA87]

Wow thats so cool. I try to do the same thing and i already have my home as a 3d model but i have some trouble to spawn at the exact location where i stand in reallife. So im always some cm off of my real location. How did you achieve that?

[@AkurA87]

@Ryan Beck At the moment i solve this the following way: I walk the carracter via the joystick to te exact position where im sitting in reallife. best way to do it is in a corner. this way you can touch the wall with your oculus controller and place the caracter for X axis and then do the same thing with the other wall to place it for the Z axis. Its not very effective since you have to do it every time i start the application, but it does the job for now.

[@JOELwindows7]

This is your daily dose of Recommendation Table VR with helping table Also Replacement VR. Not to be confused with Mixed Reality

[@theomen13]

you can attach vive trackers to chairs and thereby objects in BP will always stand so where they stand in real life

[@dandavidson4717]

This with Tabletop Simulator would be amazing.

[@no-replies]

Now this dude knows how to use VR.

[@namonamo1554]

First, you need to make sure there are no LEGO blocks on the floor.

[@jaytwentytenone2068]

or glue them to the floor and carefully model them in the quest

[@MrGooshy]

Imagine walking into someone while you're exploring you virtual house xD

[@ASMRMoto]

Jesus fucking christ ... bro this is next level Im not even sure you know what you just did . We can scale this by implementing it for each user independently - every single apartment or house or mansion can have their space rendered in VR - then we can even render well known parks so people can play there and soon enough the whole fucking world will be mapped out and at that point we will have created the most interesting company and a real simulation will exist accessible to the whole world. wow.

[@cityheron7106]

Did you model your room in vr? Thats really cool

[@spaceghostbloq2497]

No harm intended but why are people so impressed by this? You have to reconstruct the entire room in 3D plus set up and place all of the surface functions manually aswell. The only thing you get out of this is having real life boundaries that fit to the VR Room you're seeing. I can't see something like this being as adaptive as AR which is actually working with your real life environment. As soon as one Object is moved a little bit further away from it's original place the illusion is falling apart. The Handtracking is only "impressive" as long as the furniture is in it's place so your hand won't clip through the object. If this would be connected to a real-time room scale it would be a whole different story. This isn't meant to talk down upon your work and the time you put into this. It's still a very interessting concept to actually visualize - I just want to put a different view on it compared to most people in the comment section. Keep up the great work!

[@davidm6848]

This is awesome! What would you say I should I start learning to create spatial computing experiences like this one? I already have a bit of experience in 3D basics. My next step on the list is Unity, but what focus would you recommend for a learning path? Thanks for posting this, really inspiring for the average human.

[@CoBB1eUA]

Nice synchronization!

[@Hoyyes]

Intéressant. Tu as un véritable projet entre tes mains l’ami. Tu peu vraiment en faire quelque chose de grand. Bonne chance

[@thompsonevergreen8006]

Quest vr: now you too can live in a worse place than your actual apartment

[@MrPawelz88]

Something beautiful. What program did you use?

[@trg1408]

What would be nice is if you could sync up what is in the VR world, with what's in the real world and say if you turn on the TV in VR, you're TV turns on. Say specifically to show guests what you're seeing in VR or also make a way to make it interactive for guests in the real world to decide what you see in VR.

[@abe6495]

Cool, but VR can only be worn for a set amount of time before your eyes and head start hurting. Something needs to be done to perfect this technology.

[@WhoZ_Error]

I not going to lie this is dope

[@antonsinistaj4216]

That is amazing! Great work, now please make mine do this.

[@Tr1xf0rk1dz]

That's not how it works, you do it this is not as easy at it looks

[@stoneapollyon8313]

This is what commercials be like and then you get the product and it’s nothing like this at all

[@michelemarchesini8743]

how do you make interactions like that? it would CHANGE LIVES! and those words aren’t just thrown out!

[@GregMadison]

I used Unity Software to develop this demo: store.unity.com/#plans-individual

[@vladdrach9626]

He can just turn off the light. Then he'll just pay for the electricity that VR headset used

[@radicalxg8282]

For those who still doubt that the future wont be a cyberpunk / ready player one dystopia

[@vrictor]

It would be cool that the Oculus mde this automatically like a set up that uses room scanning and then invite your friends to your real house

[@dirkdex]

Very cool! How long did it take to do this? I’ve read the hand-tracking is not very great, but this video is awesome.

[@asylumrain]

I wouldn’t mind living life like that

[@nocturnoculto6898]

just wow guys i don t have any words this is really awesome

[@relic4948]

Imagine bumping into something headfirst with your *$399* headset

[@jakcarn4184]

I hope to play games with hand tracking soon

[@EdoVro]

HOW DID YOU DO THIS AND HOW CAN I DO IT TOO

[@divyanshsh]

Unity 3d

[@jakep2394]

@@divyanshsh Who tf would model in unity?

[@divyanshsh]

@@jakep2394 ez with probuilder

[@jakep2394]

@@divyanshsh ProBuilder is really bad...

[@fuzzeden6602]

create a near perfect model of the room in blender or the modeling tool of your liking then match up the measurements correctly to match up with the real room, do a bunch of software stuff idk, disable guardian walls/warnings, add a bunch of virtual buttons that activate/control certain things. I know this is really really oversimplified, I don't actually know how all of this works please don't take this as actual advice on how to do something like this h

[@razpor513]

The Oculus quest should have gloves with different colored dots to help the censor know the position and pose of your hand

[@johnnyfrankenstein0123]

now we just need to track every object in vr which will definitely come in the future

[@notme5309]

Its weird how excited i am to have a blank slate apartment with one of theseto decorate

[@maskyy5316]

Might as well do Augmented Reality instead if things were to be misplaced, which is very frequent.