VR Full Body Tracking Using Quest Pro Controllers!

[@TheMysticle]

Hey guys, just in case it wasn't immediately obvious, this video isn't here to show that you should go out and buy Quest Pro Controllers for this. Absolutely not, it's just not practical. While its pretty accurate and actually works kind of well, it's more expensive and less practical than other solutions out there. This video was created along the lines of "just because we can" 🤣 Anyway, have a fantastic day!

[@Bildnow.]

pov you dont even have enough money to get a head strap let alone quest pro controllers

[@emeraldjakegaming9561]

Why you tell us 2 hours after posting now im £1,400 short😢

[@TJXD]

It's all in the processing chip those snap dragons are why That worked so well also the cameras 😅 lolol I wondered if you could do this and you made a video about it that's awesome 👍

[@numeral382]

you should talk about oculus touch link. OTL is a steam vr driver that let's you use cv1 controllers as trackers and it's just as good if not the same quality as the vive trackers. The only thing I needed to buy was a third touch controller for a hip tracker for 17 dabloons. (edit: This method is great for cv1's bundles with broken displays cause you will be using virtual desktop) the only problem is if you were to cover this method I feel like a lot of people would start buying old rifts due to paying 130-200 max for everything compared to 300 for the trackers alone. This might lead to either a price increase for the rift and it's parts or a short supply of rifts on the market because this is the cheapest way to get actually good fbt.

[@numeral382]

also what 3d printer do you have?

[@vicktorfilmbusker5492]

What I really think meta should do, is to allow those controllers to connect to you pc wothout a quest, and use them as steam controllers. That would be game changer! For those who wants to buy a bigscreen beyond or a pimax crystal etc, and don't own or wants to buy base stations, they would easily just get the quest pro controllers. Being able to use those for any pcvr headset would be amazing.

[@virtuallyaverage9357]

The base stations track the headset though. You need the base stations for the headset in the first place

[@shiftednrifted]

​​@@virtuallyaverage9357 for beyond, yea. Pimax crystal uses camera tracking by default though

[@adr2t]

3rd party was dicuss before - but Meta wont do that that - they're too much "Apple" like to want to do that even though they could've taken the world with their tracking method a while back.

[@KiegKillsReality]

Dude the Big Screen is basestation tracked

[@vicktorfilmbusker5492]

@@virtuallyaverage9357 ah yeah, didn't think about that 😅. I was mainly thinking about the crystal, which is inside out tracking. But nonetheless, I'm sure there are still people owning other pcvr hmds who'd love the quest pro controllers. The index knuckles have a tendency to break a lot and while comfy, they're far from ideal for all kinds of games. Pro controllers are very small, feels just solid and are never clumsy. I'll miss them when I get my Crystal.

[@critikalmax341]

Great video as usual, keep up the good work 👍

[@axeljefyua9387]

Love your videos! thanks for all the news

[@Ed_memer]

Number 1 vr KZbinr right here💯💯💯

[@SSCminion]

So I'm guessing this wouldn't work if I were to use both quest 2's standard controllers as well as quest pro controllers? I see that in openvr calibrator, the pro controllers are recognized as the quest 2's controllers, and I know from experience that you can't have two sets of controllers paired to the same quest 2 device. Would manually pairing the pro controllers to the pc via bluetooth work?

[@miki4321]

Hello to whoever looks at comments, have a good day

[@lefrayy]

Aw thank you

[@gyhgamer0]

you too

Thanks man. You too...

[@axeljefyua9387]

Thank you, you aswell

[@Ausaradio]

Thank you, you aswell

[@flashyygt]

YES I WAS WAITING FOR THIS

[@BackHellsa]

These work well with the waist because the waist is the one place that causes a lot of problems with base stations

[@eliezerortizlevante1122]

im having one strapped on chest and waist. So they wont become useless controllers

[@edge3d]

Is there any plan plans of using the cameras of the controllers to read the position of the parts of the body? It seems to me a better way of having full body tracking because it has positional awarness and from those fiferent points of view it almost always see the legs and the rest of the body. Starting from the point that there are camera solutions to full body tracking it seems natural to me that meta might develop this at some point. What do you think?

[@quadstrike]

short version, it's not that simple and would be a nightmare with lacking results

[@mrsweetkandy7673]

It is easier and more practical to have a device able to track its position rather than guessing your body parts from cameras. Plus, the way you move controllers might easily make them lose track of your body; that's why you normally strap trackers on you, or have cameras tracking you from a stable position

[@adr2t]

Bad idea really. You would be better off with a stand alone camera external to the body if you want something like that. Like the old CV1 base cameras. You would be losing tracking of your body and its last position every time you move the controllers. It would be mess really.

[@virtuallyaverage9357]

I wonder if you could pair It with slime hip tracking 🤔

[@TKcopYT]

I already have quest pro controllers, I will haft to try this out for the hell of it.

[@loverrlee]

You can use 1-2” elastic to attach them to your feet :)

[@MunchPremium]

I figured that it would work i got the right files for it but you can also just use driver for vr

[@KaraokeCody]

what about using quest 3 and using the quest pro controllers as the trackers, is this possible?

[@muridsilat]

Check out Alexander Winkler's video showing AI-generated lower-body movements using only data from a Quest 2 headset and hand controllers. The video didn't demonstrate jumping, kicking, or handstands. Walking, jogging, turning and simple standing poses, however, were very close to the real-life movement despite being completely extrapolated. I can only imagine how much more impressive this would be with tracking data from Quest Pro controllers. Because of Meta's infamous statements about legs in VR, a lot of people don't expect to see a full-body solution from them. I suspect they're just waiting until they can introduce a complete paradigm shift. I wouldn't be surprised to see FBT added as an experimental feature of the Quest Pro with no additional hardware.

[@wonderowl530]

Tru, prob the reason why the quest 3 camera placement is changed

[@TJXD]

Can you use quest 2 controllers instead of index

[@CentopyIsHim]

i have question that nobody has gotten back with me yet on and its about my pcvr not working. I was able to play pcvr, steamvr, etc. but now, i never restarting my computer between this happening, it says my graphics card is not capable which I understand but its confusing because i was able to run it just fine for several months with no issues.

[@Yipper64]

im currently just marking into a budget someday for some kind of VR upgrade in the next year or so, probably the quest 3, but depending on how much I like full body with slime, it might be the vive trackers.

[@Tropical-]

Love the content keep it up

[@deliriousvrc5252]

You can’t just rename the extra controllers in steamVR so steam knows not to use them as hand controllers like we do for Vive trackers?

[@jayes5066]

where'd u get that danny devito wax statue

[@eliezerortizlevante1122]

Quest Pro issue has the issue with shocks. If it has shocks it trips the magnet and disconnects them

[@tandyVR]

Using alvr you could also do that

[@emmanuellaude9932]

Thank you for this great video. But… I hope you’ll understand, I find it so unfortunate to be able to finely capture the movements of the hands and the body but not the face when you use the only headset in the world able to capture the movements of all the face. You could do it through AirLink but you could not take advantage of the important stage tracking function of VD. It seams that ALVR can do it but with a bad Iatency or with bugs. It would be amazed if you could show us how to work around the problem.

[@jmhbjre2186]

you can use owo track for hip tracking

[@emmanuelgoldstein3682]

Someone's been sipping something rather orangey

[@Shadow_banned_by_YouTube]

You have a wrong definition of perfection 😂 6:10

[@TheMysticle]

Well, I did say they lost tracking a few times xD

[@Shadow_banned_by_YouTube]

@@TheMysticle it wasn’t great all along either

[@TheMysticle]

I'm going to have to disagree. When they didn't loose tracking they were bang on. The only issues they had were with the hips. Since there was no hip tracker. The jittering comes from them being attached with tape and wobbling. The tracking itself was really good.

[@ifrite3]

​@@TheMysticle right..

[@Shadow_banned_by_YouTube]

@@TheMysticle ok sound reasonable. Waiting for HTC ones to shine

[@nickdarcy7453]

Hello everyone, I have a question about using the quest pro controllers with fbt (2 vive 1.0 basestations and 3 vive 3.0 trackers). Does it help to improve on the occlusion issue when quest 2 controllers get close to the trackers?

[@Jkauppa]

hey instead of complex light houses, use the beacons as starts, and any imu controller has light distance (and angle) photodiode detectors, yes aling your location based on star beacons (tm)(r)

[@Jkauppa]

more quest pro controller track pods

[@Jkauppa]

hey why does a mouse controller need to run an android, wth, bluetooth it as a device to a hub, or whatever, ok standalone but why, just one main device hub or pc, over bluetooth/wifi/whatnot

[@WreckedElk]

Is it possible to do the same, but with Quest 2 controllers instead of Index ones? I already have Quest Pro controllers for my Quest 2, so it would be pretty cool if I could do FBT like this even if it's kinda scuffed 😅

[@ImmenseFox]

He said this at 0:36 that you can but its not able to see the quest 2 controllers properly at all times so messed up.

[@WreckedElk]

@@ImmenseFox He was talking about using the Quest 2 controllers for the full body tracking, rather than actual controllers.

[@zlatansavic9629]

it works well im using my cv1(oculus rift controllers ) for full body tracking

[@r_olawg]

Hey @The Mysticle , I’m in the market for a good PC for VR, and am wondering what your PC specs are? If anyone has any recommendations please do tell me. Other than that, thanks!

[@ImagineBeingBored1]

I heard that you could use Nintendo Switch joy-cons as body trackers

[@Therepairman67]

These controllers are great when they work I play almost every single day on PVP But out of nowhere, the track and gets lost all the time and drift off and the controller starts overheating every 30 minutes and I’m not the only one with this problem

[@MooshPaw]

Ok so if the vive tracker is as good as the quest pro controller tracking, FBT seems surprisingly good and comfortable? Without the need for base stations (that personally look bad) and only needing 2 trackers and a phone for hips, i can see Full body be very appealing, the trackers just need to be decently cheap, doubt the $100 mark will happen although that would be an insane deal, 200 seems like the most expensive amount

[@unknownfrost1218]

Would it work with quest 3 controllers? Please reply so I know not to buy actual trackers or pro controllers 😂

[@Caliell]

only with VD

[@reenz1156]

Could u please make a video with Nintendo Switch Joycon Controllers as trackers, please 🙏. There is one video out there, but I am interested on your perspective 😊

[@LittleRat6969]

you should do this with the owo hip tracking and this would be a god set up lol

[@DevAnimate]

What if you used a mirror 🤔

[@adr2t]

As for tracking the body, it would be really neat if someone could just come out with a vision base tracking instead. For example, two external cameras that uses AI to module the tracking / body and feeds that information back to either the game or headset. It would be cheaper than pucks and a bit more useable instead of putting them on and off every time along with charging them.

[@lavarsch]

Something like this already exists, but the problem is that is very heavy on calculations.

[@adr2t]

@@lavarsch Its not that bad really. A cell phone can almost do it already. Just need an SoC design around the idea now to really push it. Over all system would still be in the 100$ range sure, but over all cost would be still lower than having to buy soo many pucks in the first place.

[@kaot1k]

The controller knows where it is at all times because it knows where it isn't, by subtracting where it is, from where it isn't, or where it isn't, from where it is, whichever is greater, it obtains a difference, or deviation.

[@perfox7395]

don't forget about the controller guidance sub-system

[@perfox7395]

@@rangerkayla8824 why'd you comment that in this comment's replies? lol

[@perfox7395]

@@rangerkayla8824 the original comment you replied to was just a joke reference to something unrelated to controllers and battery life. this is the video kzbin.info/www/bejne/mIvIZn1uiLt2j7M

[@andredoxer5646]

Thank you 🎉

[@Gc9User]

just get a vive ultimate tracker for the hip tracker and you're done i guess

[@andrewthomson5266]

I thought someone would do this, full body tracking on the quest pro controllers. Though I was hoping it could all be done with the quest pro controllers only, maybe one day someone will hack them to use the output from the cameras.

[@deliriousvrc5252]

I figured this would happen soon.😅

[@kimmy9046]

combine this with owo track for the hip tracker and you got yourself the most frankenstein fbt solution there is.

[@quadstrike]

this is mostly in proof of concept, i don't think mysticle thinks this is actually practical. comments need to chill 😂

[@TheMysticle]

Yeah no, this was along the lines of, just because we can 🤣

[@dmagic1one]

They are like $300 or so for a set right? Just get another set For quest owners it's cheaper than base stations and two vive trackers.

[@ricardo762]

Here we go again.

[@callmidell]

so basically you have two of the new unreleased vive trackers

[@cli]

try owo for hip

[@ObviouslyPoptart]

Ok

[@nazaxprime]

Considering what I paid for them, I'd expect them to also make sandwiches. 😂

[@ToshiZ64]

For $150 per tracker, this is one of the most expensive solutions yet. This is more expensive than just using Vive Trackers if you have even a single base station. The only viable use is for people with non-base station hip tracking (like owotrack or SlimeVR for example), and even then, people are better off waiting for SONY'S Mocopi trackers or the wicked new inside-out/camera-based Vive trackers. See Thrillseeker for a product demonstration/trial. They're literally Vive trackers with camera tracking added. The best of both tracking systems in one product, and they're possibly MORE accurate than the existing Vive trackers thanks to the possibility of drift-negation! Ideally, the camera data will sync with tracking processing and completely eliminate Vive tracker drift once and for all! Those don't require base stations either (though it's Vive, they're gonna be 100% compatible).

[@TheMysticle]

It's expensive and not practicle. However, it's possible, so of course I was gonna try it 😂

[@ToshiZ64]

@@TheMysticle Ayyy thanks for responding Mysticle! You're a real bro/homie.

[@adr2t]

Yea price wasnt the goal - its more of it was possible. That opens the door for more options later down the road.

[@Mr.Anders0n_]

A great solution... for the few hundreds worldwide who have both valve and quest pro headsets and are interested in fbt 🙃

[@acakeshapedlikeatrainonatable]

if they are rich enough for that then why would they do a makeshift solution like this 💀

[@Mr.Anders0n_]

@@acakeshapedlikeatrainonatable some people are geeks and are into DIY/hacking stuff for the fun of it. So this situation is for a niche market within a niche market within even a nicher market 🙂

[@acakeshapedlikeatrainonatable]

@@Mr.Anders0n_ this will help dozens of people worldwide

[@claudioalejandro9731]

HOLYSHIT KOKICHI OMA

[@Riletin]

Use a phone as a hip tracker and then by that you have full body tracking

[@TJXD]

I don't think meta is doing this because the fact that they would need to put a snap dragon in every tracker

[@emiryusuf9765]

Mark... İ Just Want This Fucking Standalone Tracking. EVERYONE İS POOR!

[@KWez256]

Okay 🙌🎉

[@Cart1416]

Why

[@acakeshapedlikeatrainonatable]

lol he said, its fbt for free if you have the index and the quest pro... but not many people have that

[@moonut9643]

Tracking sure "works well" , yep nothing wrong with it !!!

[@TheGamingExperiment007]

how you need 4 controlers thats crazy

[@rekusloth3979]

Long new vive trackers

[@LyeryCh.]

for anyone wanting to do this but with the quest 2 controllers: please don't this works "nice" (actually not) because the quest pro controllers can do 360 tracking without needing the headset to see them quest 2 controllers can only do 180 degrees of tracking and requires you to either put the headset in your hip or place them close to the controllers also like others pointed out, this is more expensive than just buying vive trackers

[@gar_ee8884]

Pro tip, you can leave the controllers strapped to your feet when you start. You can run the calibration software with the HMD on your head, just press the index controller onto the top of it, grab on and move your head around. On the Quest 2, it results in much better calibration quality as well since the headset tracks way better than the controllers do anyways. It also solves the problem of having to start and stop virtual desktop to get the quest 2 controllers to f*ck off and stop getting pulled in as primary controllers, as you just don't need to use them at all since you can start steam vr just by turning the index controllers on

[@JohnssWorldClub]

I hope no one walks up on me in VR like this

[@Lunarizau]

W

[@gyhgamer0]

hmmm

[@SargonDragon]

Why isn't Meta selling this technology as FBT trackers when they obviously have the ability to do so? Consider the ramifications of it. Horizon Worlds still doesn't even have virtual legs on their avatars. If Meta were to release such a product, it wouldn't work with their own software! Instead it would just drive more users to VRChat.

[@DeadlyBman]

Run doom on it

[@adr2t]

Meta simply wont do it though. They're whole thing is "Apple" and they wouldnt want to share their eco system with someone else. Even though, it would bring them pretty much the whole market.

[@boxboy7999]

Yo 2 minutes ago?

[@arnoldbliss6238]

THANKS! [:]~))

[@beanbaground1]

just buy real trackers at that point

[@Nicalay2]

Yes, it works but... A Quest Pro controller is $150 (assuming you buy the $300 pack), which is more expensive than a Vive Trackers 3.0. You also need 2 others controllers that need to be base stations tracked controllers (and watchman dongles), because you can't use Quest 2 controllers and Quest Pro controllers at the same time, so you need base stations for them. Lastly, you need a Quest 2 or Quest Pro headset (most people have and/or use one of those headsets, but not all people do). Overall, it's more expensive than Vive Trackers + base stations and the tracking is just worse : nothing can be as precise and reliable as base stations tracking. Instead, you can buy a OG HTC Vive Full kit for $200-250, which come with 2 1.0 base stations, 2 controllers (that can be used for feet tracking) and the headset (which would be used as dongles). And for much cheaper, you can get a more precise tracking (because base stations tracking), and the 1.0 base stations will also act as IR light for the Quest (and so let you play in the dark). Oh and you get a backup headset if your Quest need to be repaired/RMA for some reasons.

[@shiftednrifted]

Camera tracking is just as good as lighthouse tracking. See thrillseeker's video on the camera based vive tracker. Its the fact that the cameras were being used to track external objects that gave people the impression is not as good. But the headset itself tracks just as well as lighthouse and by extension so do the controllers with cameras. The idea is not to buy controllers for this but to use the controllers you already have. Say you have a quest pro or a quest two for standalone and you bought pro controllers for a better away from home vr experience but when you're home you just use your index controllers. Now you have two more trackers free

[@Nicalay2]

@@shiftednrifted About camera tracking, yes accuracy wise it's as good as lighthouse tracking, but the reliability isn't. Oh and turn off your lights and say me how the tracking is good.

[@arsteel2388]

So you're saying I can basically buy a set of Quest Pro controllers to use for full body, at the same price it would take to get Lighthouse and Vive trackers? Noice.

[@TheMysticle]

I never said it would be practical 🤣 Sometimes we do things just because we can 🙂

[@monashaVR]

it is fun experiment xd

[@kellanaldous7092]

Just in case you can afford 350 bucks for the quest pro controllers, but can't afford to buy body trackers.

[@KittyKite]

Furry