Quest 3 is going to go ALL OUT (full body tracking)

[@EricForPresident]

With the magic of Hand Tracking 2.0, I don't think this concept is far from the truth. Thoughts? Follow me on twitter for VR news and chitposts twitter.com/EricPresidentVR

[@XRWorkout]

We'll have built SlimeVR support directly into our Quest app and will be going with that for now, but what I see coming in the midterm is kind of the small brother of FBT with hand tracking for the hands and simultaneous controller tracking for the feet.

[@brettcameratraveler]

Love these white paper videos. More please. We need a channel like "Two Minute Papers" but focused on XR R&D news.

[@boudenasouhile]

What a time to be alive

[@Ila_Aras]

squeeze your papers

[@cedricknapp2889]

Hold onto your papers fellow XR scholars! Haha jokes aside, if you transitioned down this path for your channel you would have a very loyal subscriber here!

[@spiritaam6689]

Yesss! Would definitely love this

[@Daniel-xu8rq]

I would be extremely impressed, when they actually realize full body tracking with the controllers meaning that I don't think so in any way. Even in the video for the paper, you can clearly see that almost all of the time the women has her hand wide in front of her so that the cameras have a clear view. However realistically, one often has the controllers in a position where they are close to the body such that the whole body is simply not visible from the view of the controllers. For full body tracking you need a really reliable system such that it doesn't destroy the immersion and you glitch around. That being said, I also hope for that to come but I don't think so in any way with this system.

[@BoltRM]

Maybe when the controls can do body tracking for Riverdancing? 😉👍

[@deadpianist7494]

instead of using camera how about heat tracking of our body?

[@tano99999]

@@deadpianist7494 same problem as before. You still need a way to “see” that heat no matter where the users puts their hands

[@Nilloc777]

yeah this is only really a viable option when there is ideal conditions as shown in the video and even then fast movements (possibly even as trivial as walking) probably create tracking artifact as in the video there was only very slow and controlled momvent.

[@BoltRM]

Using a couple of Wyse cameras would be nice!

[@SjaakSchulteis]

I think that expecially with fitness games where one still can do a lot of things wrong, with full body tracking implemented in the game, it might help you to correct your position.

[@warbearin]

Full body should be a standard in VR. Imagine playing pavlov with full body, being able to propperly utilize cover/leaning etc.

[@stormvr1070]

Yes! I can't tell you how badly I wanted to kick head crabs in Half Life. Or just kick the door open.

[@SirPonnd]

Controllers that stare at you with cameras, and an incentive to stand in front of a mirror while using it? It's cool tech, but REAL sketchy considering the fact that this is Facebook.

[@SteveKnows]

this is such a take bro! YES a thousand times!

[@IsItLunchtimeYet-1]

I've been trying to tell everyone that Meta is going to use the cameras on the controllers as full body trackers. You will set them on either side of you. Hand tracking has become even better.

[@Shiffo]

Yup they are going all in on Computer Vision

[@penguinhunter]

I assume the amount of lag you’d get with a camera on the controller would make it borderline unplayable.

[@imacmill]

Did I miss the part where they say what happens when you put the controllers down by your side?

[@theoriginalrandomman]

I agree that keeping friction to a minimum is vital, and using cameras on the controllers to get more info is a neat trick, but even given the optimistic conditions shown in the video, the results aren't great. I can maybe see this method being used to back up other methods, like the mirror thing, but on it's own I don't think it's going to get there.

[@mica4977]

I agree but thankfully the most important thing is for it to exist. Once it's out there being sold there will be more demand for it to be improved & perfected whether through the same or redesign of the concept. Semi similar to how inside-out tracking has become a very viable option.

[@Shiffo]

You could say the same thing about hand tracking and look at the current state. It's a matter of refining neural networks to remove the jitter and training the networks.

[@SkeleTonHammer]

Agreed. Nothing beats perfect tracking from an external lighthouse and a tracker. Furthermore, you're going to have to wear the right clothes or else the tracking likely won't work (especially if you're wearing something loose fitting). Occlusion issues. So on and so forth. Camera-based computer vision is horrendously inefficient and for poor results. These things are fine for really, really casual VR users probably. Like, people who are so casual they don't care or notice when tracking is way off.

[@davidholt8937]

I wonder if they plan for the controllers to full body track while in the docking station, so you can use the hand tracking too. That would be great.

[@thenatetful]

I’m a little hesitant with the full eye tracking in the future, I don’t want anyone keeping track of how often I look at other peoples certain body parts

[@deadpianist7494]

ur not the only one :)

[@tugsuumovies1999]

lmao idk man, the fov is not that big so I feel like it's obvious when someone stares in certain places even now

[@booshting3520]

Its when they start tracking your boners you'll need to worry

[@iGege83]

Instead of on the controllers, I think that additional wirelessly-connected cameras could be just laid down on the floor. It should do the trick allowing full immersion without controllers

[@BoltRM]

Little wireless cube cams ftw! 💪 From Wyse?

[@14rs]

it’s not that easy

[@PopLolli1]

Thst would defeat the purpose of wireless

[@z771p]

This is just way cool. I love that people are going and demonstrating what's possible. FBT isn't really that valuable to me right now, but chicken-and-egg. If they can show that FBT is doable at less end-user effort (heck, even my haptic vest isn't getting much love and it's relatively easy to put on), maybe it'll give more developers the confidence to go build something awesome that'll give more people a reason to want FBT.

[@DanielCCaz]

When did Boz say Quest 3 in 2023? I’ve only seen rumors and speculation.

[@johannes5076]

There is a really noticable lag to the presentation. Hope theyc an optimize further

[@glenrisk5234]

The positioning of the camera's on the controllers that have been circulating wouldn't work for full body tracking. But the video you show of people working on it is definitely interesting. I'm not into VR chat but the tech would increase immersion in VR for games so I am definitely for it.

[@VisibleMRJ]

Just give us another 2 controllers and we will stick that to our legs for full body

[@manp1039]

I literally stuck the existing quest 2 hand controllers to the top of my feet while playing recroom as an experiment and used them to kick the basketball in the RecCenter. Had i a seceond set of hand controllers i could still use the joysticks on them for moving around and for manipulating a dashboard and other functions i need with my hands.

[@Shiffo]

Fully agree that these papers are interesting as hell. Meta is working on a lot of exciting stuff, codec avatars, neural rendering, computer vision, DNN chips on the hardware. This video and those papers show me that there is no way anyone is catching up.

[@JiiruKoga]

I feel like the only way the full body tracking is working through the controllers is when she's holding them far out, big amount of time in VR I'm just standing with my arms hanging by my hips/thigh. Isn't it more likely that they'll activate full body tracking by standing at a mirror like in the demo? One more thing, SupernaturalVR owned by Meta, showed off leg tracking from the headset for its game recently, so they are pushing for it.

[@legendp2011]

I still think 2x wireless external connect style cameras should be the easiest method. all the other methods are going to have temporary visibility issues based on where the hands and head are looking......in this method the controllers are held pretty far away from the body. external wireless battery trackers would be very easy to place and move from room to room

[@runvnc208]

They should really just make something like tiny base-station cameras that don't need to be mounted particularly carefully or high. Will be able to see a lot better. They should carry a charge for a long time. Also make it easy to charge them and to position them.

[@legendp2011]

yeah. 2x wireless battery powered base sstations (with little tripod bases) would be a much easier and more reliable method. right now they are trying to run before they can walk

[@pineapple-rr5hl]

i I had a theory that if they where going to have full body tracking it would be with the controllers but how would it track if you have your controllers to your side if the quest came with base stasions that had a camera and with the same function then that would be cool like a xbox kinect

[@TropicalCyc]

It wont be that good with an external camera, nono vr already did it, and its badly tracked

[@pineapple-rr5hl]

@@TropicalCyc the kinect works pretty well on pcvr and with the software meta has I wouldn’t doubt that they could make it work pretty well. The same was true with inside out tracking but after a bunch of software updates its pretty good

[@onerimeuse]

My thoughts exactly. Two standalone kinect sensors front and back and you're good to go.

[@sportsbanter3885]

Shooters in VR are my favorite part, wonder how that will be incorporated with holding weapons, especially without controllers, maybe a gun controller/prop with a working trigger....

[@Coconuts738]

I think if the removed the ring on top the meta controller and put it on the sides of the hands like the valve index controllers shouldn't the camera on the top part of the controller be able to track your individual figures as well right after?

[@NahFam13]

I both understand and find this counter productive, on the one hand they show off "hands free tech" and then propose body tracking functionality by using the very same controllers. (not the same but you know what I mean) I am excited for the future nonetheless. Thank you for making this video!

[@mica4977]

Another comment mentions the possibility of resting the controllers near by with view of you to utilize both hand and full body tracking. So perhaps they can be combined neatly.

[@attq3980]

it would be better to add base stations and make a proper full body tracking using them

[@johny79action]

I know VR tracking doesn't like shiny/reflective surfaces but I wonder if a large mirror would make FBT more possible with inside out tracked HMD's!.

[@maxjames00077]

I can imagine doing workouts with full body tracking and the device telling us if posture is good or bad

[@onerimeuse]

If they don't go the below mentioned standalone kinect censor, they should at least add two more cameras below the oculus for arm and torso kinematics. Otherwise, the controller idea is alright I guess. But like mentioned below, I really think two standalone kinect style sensors front and back would work well. There might be some latency issues, like there is with the air link, but I think since the tracking data is so much smaller than, like, streaming a game, just having them be Bluetooth would rock.

[@DarkGaming-2734]

Keep up the great work!👍

[@adamb89]

If they get rid of that stupid Facebook requirement I might get on board with them. Until then, once my Rift S dies my plan is to go with the Vive Pro 2.

[@illDiology]

That "feat saber" demo looks like it was created by someone that has never played Beat Saber before. Good luck extracting any useful data from those controller cameras when your user is flailing like a lunatic! To answer the question posed at the end of the video: Whoever creates a slightly better Beat Saber that uses your arms AND legs is going to make a metric shit-ton of money.

[@Tahllia]

Love the idea for cameras, but I do wish that they would maintain some sort of ring around the hands. Weird I know,

[@onikaze8445]

research papers are great and can show what may come in the future. Going from published to an actual product can take a while. For example, the face haptics you showed off a video or two ago. I saw and tried those sonic arrays way back at siggraph '08 or '09 and just now there seems to be more movement with it, albeit once again in experimental form. While I think it would be a nice added integration I'm not confident in seeing it any time soon(soon being within 2 yrs) This looks promising but I wonder how feasible it is in the short term( next 2 years). With people already throwing concerns on the battery life impact of the Cambria controllers just doing on-board slam positioning to now add these extra computations along with the assumption that people are T-rexes and they stand in a resting pose with the arms held out in front of them. cool that the future looks bright but it may be premature to don the shades right away.

[@Refreshment01]

Interesting. I thouvht vr controllers were handling calculation natively, thats how the wii remote was worked back in 2006. It basically the same concept actually when one considers now they will be doing inside outside tracking with an optical sensor. Of course way more advanced now.

[@onikaze8445]

@@Refreshment01 right. I would think that everything is on-board the controllers and they just send the minimal vector data across bluetooth, but if they are now expected to do this image composite and skeletal tracking from images I would think it would need more compute and then be a larger drain on the batteries. the Wii motes were just sending accelerometer tracking along with the low-res infrared that was just tracking 2 blobs in 2d. Not sure if that was on-board or not. of course I'm going from memory so I could be incorrect here.

[@Refreshment01]

@@onikaze8445 Yep, a big deal was made out of the triangulation calculations being done nativly in the WiiMote. By no means an expert here, but i also remember that the Remote could track 2 extra IR points (4 total), so many enthusiast were wondering why Nintendo didn't eventually map the TV screen 1:1 or made an IR array that worked better for absolute tracking. There's even videos of using 2 wiiremotes to 3D track goggles. About the skeletal tracking, at this stage that basic skeleton seen in the video could easily be done on board. i mean is not more complex than what the kinect was doing in 2010/

[@thev5546]

Haptic feedback touch. That what I'm waiting for.

[@Darnelj1]

Even at 700$. Decent full body and all the extra tech upgrades Out of Box would be a literal steal. Nothing could touch it. Not even great full body with extra accessories for 1800.

[@stormvr1070]

1800? geesh. I spent $300 on mine.

[@tubeyou3303]

the only issue i see is who plays a game with their hand in front of em 24/7. If my hands aren't in front then they r at my side or in my lap. i think the tracking will fail just like on the headsets now that have an ok time tracking your sides

[@Gr3mlinfloof]

Wrist tracking, arm tracking. tracking leg.tracking hand tracking, face tracking, waist tracking a device you can run on that makes you move in vr ?! Now possibly a headset that already has built in trackin* lol

[@whatsup7253]

How bout if they make the visuals more realistic.

[@HalkerVeil]

Lower bodies were done right a long time ago. They should look at Second Life. Where we made whatever we wanted. Including legs if we so desired. All 3 of them.

[@BootzBTrippin1]

Wouldn't Gyros, and accelerometers be more accurate? I mean I optical is good when the cam is pointed right at you, but when it cant see you drifts are gonna happen. I really think Meta should look into what SlimeVR is doing

[@BarbokVA]

Yes always yes

[@mrspicolli]

kind of fun timing that fix iz rite at the end of its q2’s throne days, not like quest2 dies that day but certainly the quest3 will have bttr hrdware for tracking than the 2 w/ software. And will have games q2 wont I would prioritize these: 1-do not tie oculus store purchases to one’s Facebook account, keep them separate enough to when a Facebook‘s PC police ban you, they don’t get to keep your money and deny you your games. 2- controller tracking, esp. in bright light 3- smart lenses(can tell the best settings for your immersion in VR) 4- casting/recording. No more out of syncNess, easy, quality right in the settings, and casting should be able to work off a phone‘s cellular Internet not absolutely require Wi-Fi even on Wi-Fi quest2’s is slow or choppy, needs to look awesome, and display that way when transferred. Because that’s the whole point of it being able to do that in the first place ✌🏼🥳

[@vincentpaniccia109]

I was thinking would it be possible if they just made their own tracker using the same technology inside their new controllers.

[@TheDaydreammaster]

LOL! w0t!!! Ok. Eric, you just officially made me excited for MQ3....

[@Thundermuff1n]

NEXT YEAR. Bro I got my quest 2 in December and I haven't even played beat saber. but I have played one pop, superhot, medal of honor, rec room, gorilla tag, alto space, etc.

[@ShiroNekoDen]

The fact you need to use controllers for it is a deal breaker, also it just looks unnatural the pose you're stuck in.

[@apngdog3493]

With the white paper thing, are those cameras available as an accessory for quest 2? Because I believe the woman in the video is usng a Wuest 2.

[@HalkerVeil]

Mirrors? I thought those were the enemy for tracking since the camera see's a copy of the controllers in reflection.

[@Synthwavu]

looks meh, might still have the same output as the xbox kinect but slightly better?

[@BCBtheBeastlyBeast]

I'm still convinced that the mirror is virtual. It shows a person trying on clothes--of course you'd want a mirror in VR for that.

[@jacobc.2899]

definetly buying a quest 3 when it comes out. so awesome!

[@kamillatocha]

5:00 yah sorry i do not think holding your hands straight forward at all times is very enjoyable ..

[@PJxpanterx]

His eyes are so close together

[@SirCharcoal]

This seems like it would only work if your hands were not close to your body

[@YukiigoVROOM21]

The thing is that a lot of people will feel that it's unfair As in They spent thousands or hundreds of dollars to get the tracking and pc required But then there's people who just bought a possible 500-900$ headset that has fbt already in it

[@sb2jtechnologies734]

Meta Quest 3 or PlayStation VR 2 in 2023? If consumers still aren't able to buy a PS5 in 2023 because of shortages, PlayStation VR 2 is gonna have a rough start.

[@Blobby192]

if there is no improvements on FOV then i will stick with my rift s

[@alwaysyouramanda]

Who else wants FBT over face and eye tracking-!?

[@CC-rg7hp]

I'm considering purchasing Vive FBT for about $450 to connect with my Quest 2 and PC. Would it be wise to wait until the Quest 3 (or Apple VR) instead of Vive FBT?

[@stormvr1070]

My opinion is just get the Vive trackers if you can afford it. It's simply better and nothing will be it's performance. Keep in mind, Meta showed off fbt 3 years ago, so don't expect anything any time soon from them.

[@jasonhunter6125]

I foresee more of a ring on each finger for AR

[@n0xxm3rcyxx]

So no index 2...yet...?😓

[@frenchguyst-croissant3432]

is that why they remove 90 hz native on my quest 2 ..so the next one could be an upgrade XD

[@froggyblocks]

Should I wait for the quest 3 or get the quest 2

[@richardnickolich8846]

so has anyone ever thought about using a non newtonian liquid or something similar in a rubber ball for a vr treadmill

[@pdbsstudios7137]

why don't they just make tracking cameras that you can place around you in standalone mode?

[@MichaelBTryn]

This is my favorite kind of tech!!!!!!!!!!!!!!!!!!!!!!!!!

[@ScorpioFortuneTarot]

Do we know how much this one is gonna cost?

[@jamesmuhuhu7987]

"So the quest 2 is coming in 2023" Me after buying the quest 2 three weeks ago: 👁👄👁

[@Synthwavu]

same here 👁👄👁

[@drxplxrd]

soccer in vr

[@Jona69]

The name feet saber made me laugh.

[@tapeokagaming1656]

Maybe they can make the controllers compatible for the metaquest 2

[@kusumotirtoanggoro_2451]

when is quest 3 coming out? because i just bought quest 2 😂

[@zantrex4]

Wish they invested the RnD for "full body tracking" on BCI research or actually good apps. Couldn't care less about full body tracking on my end 🤷‍♂️

[@justreallybored6014]

And yet we still cant get most of the basic streaming services to be added. I literally use mine to watch media and would love to have all of the streaming services make apps for the Quest. I really dont understand why this isnt a bigger issue. KZbin works amazing, Netflix and Amazon Prime work but kinda suck in what they can do compared to KZbin. Yet, all of these apps are over 3 years old now and nobody else has entered the market. Also, Netflix and Amazon really need a refresh.

[@manp1039]

i am not clear what you are talking about. Are you refering to streaming services being better intigrated into VR? or?

[@at8833]

I didn't feel youtubes superiority here. Most streams are completely pixelated (for vr) so I did not feel it makes much sense rather than a gimmick. Out of the box Netflix worked best for me. Prime is often glitching or just simply not being responsive. Would be nice for both to get updated environments especially when you move the screen (in Netflix only void theater).

[@justreallybored6014]

@@at8833 I have never had an issue with pixel/resolution issues. Its KZbins ability to change the size of the screen and move it around that I find odd that the others dont have. Also, Amazon load times take forever.

[@at8833]

@@justreallybored6014 Both of them ha e this function. For netflix you have to choose the void theater at the top to do. It's a bit clunky in design.

[@justreallybored6014]

@@at8833 Im aware, but no, they do not fully have this function. KZbin can fill my entire viewing screen, netflix and amazon only do about half of that.

[@c6jones720]

Its all cool stuff, but if you put the cameras in the hand controllers to track your feet and body, dosent that then mean you then lose hand tracking ability, because you have to use the controllers?

[@manp1039]

if the hand contollers are put down in front of you.. wouldnt the cameras on the hand controllers still work? what i am concerned with is gw much power the cameras will require. a single AA rechargeable battery will probably get drained in a couple hours? currently i can go about 2 weeks or more of several hours per day use before needing to recharge the batteries. I use nmh AA batteries. and i just swap out the drained set for some freshly charged batteries.

[@krishnaaskr]

when quest 3 will release?

[@jessiestarr4600]

The pimax 12k has super peaked my intrest

[@mr.wigglemunch3856]

The one thing they have to change is the field of view. Oculus, the tunnel vision is so passe. human like field of view please!

[@azurestrike6551]

The graphics remind me of ps2 or GameCube days

[@lastlvl]

Eh as much as this sounds cool, I think they should focus on great software as well. As much as I love the quest 2 and vr. It can get really boring sometimes.

[@Shiffo]

Content will come with the growing user base of VR and growing market size

[@Staticgames1209]

I want them to add steam vr without the need of a computer

[@LaurenMiddleton28]

4K OLED with 120HZ Lenses should be Amazing. Supposedly the Apple VR lenses are laughably terrible. 2 Grand for something that doesn't even beat the Quest 2. Zuckerberg has been spending a ton on innovation and visual engineering.

[@JustSomePerson998]

yes i do want a full body tracking with all that f-king adding item's it seem unnecessary to have

[@VENOMxVR]

Pulling out that Quest 3 clickbait eh?

[@Snaphoo]

dude, its not a "quest 3"

[@gymkhanadog]

Wow this looks janky. Like old TrackIR tracking before we had good VR and had to cobble a bunch of tech together to get any sort of 3D tracking. I really doubt this type of camera setup is going to be viable given that nobody plays with their hands held out at full extension at chest level all the time. And in most interactions you'd need body tracking this tracking position won't work. Pay attention the next time you're playing and see how often your hands are down at belly/hip level or being held horizontally.

[@SkeleTonHammer]

Yep. Camera based full body tracking technology is terrible. Even a Kinect - with depth data on top of image data - can't resolve the proper position of a human skeleton without a lot of jank. Then, add on top of that - what if I'm wearing slightly baggy clothing? What about occlusion? And how poor is the frame rate that it's able to track? Because with "real" tracking using a lighthouse, it can track at hundreds of samples per second. Guaranteed to not be so with an algorithm that needs to build motion from images.

[@texmutantvr1464]

wow; are you okay bro. go outside

[@TheLoungeVR]

but instead of using cameras why not make a full body suit with nano haptics...

[@cesarsmith5534]

Meta don't have lower body yet. 😐

[@dathyr1]

Looks like the Quest 3 full body tracking is in its infancy stages and needs to be more refined to make it look more real. Right now looking at those videos, it looks too jerky and not in sync with the real person. We shall see what it looks like in 2023 with more reviews. I dont know why I would even need this with my SIM racing games since we are just sitting in a car and all you maybe see is hands on the steering wheel. Anyway, thanks for the video.

[@muhammedyousufrajput1736]

Love this all but I’m kinda sad since I have the quest 2 and I’ll have to spend more money lmaoooo

[@Emma-kz3zr]

Absolutely, the first out of the gate with full wireless and body tracking is going to rule the future market share.

[@ten10_ten10]

Personally, I don't think that camera tracking will ever be the way to go, at least not on the controllers. Quality aside, the consistency of camera-based tracking will just never be at an acceptable level if the cameras are constantly moving and rotating. I mean it'd basically instantly sight of your legs if you put the controllers close to your body or above your head. If Meta is really dead-set about not making any tracking pucks, then they should take a page out of Microsoft and Valve's book and make a camera basestation that'll stay in a fixed position. No additional weight or things to put on your body, still extremely easy to use if they make software do all the setup and hard work. Of course that's easier said than done, but to me that sounds 10x easier than the controller camera idea and about 100x better.

[@deadpianist7494]

i agree.

[@afishperson]

Cambria sucks, it is for work and not much else, also it is over 800$

[@KungFuChess]

I'd rather see better FOV then body tracking

[@noelle1314]

Wider IPD or hard pass!

[@Sora-o8l]

it's a university trying to make quest 2 full body not oculus themselves doing it, don't spread false information.