How to set up iPhone face tracking for VSeeFace using VTube Studio

A quick tutorial on how to set up iPhone tracking using VTube Studio and VSeeFace. Free and good quality tracking.
By using iPhone tracking you also reduce the load on your CPU. OpenSeeFace tracking can take up a lot of resouces so if your computer is not that powerful this can be a way to offload your computer of the face tracking.
Like my content? Support here:
www.buymeacoff...
liberapay.com/...
XMR: 82sLiWqHfpG17HhwpRyE2kZMRNJgxBxHK9moyYjv4vK8AQUAardGCbESrCd56gkjCC1LMMrgWeLDQ5aKK5y7rEmtVnQfM9b
Links:
nuemedia.se
stream.nuemedi...
sns.nuemedia.s...
/ nuekaze
joinpeertube.o...

[@cappared]

Thank you this was incredibly helpful. Quick and to the point. Got it working immediately.

[@cyan_2169]

Awesome, worked for me! Just need to find a good phone holder now. Thank you!

[@jesmcdevlin3116]

Thank you so much for this straight forward tutorial! Real life saver!

[@94752yesterday]

this was a complete life saver thank you so much, just subscribed !

[@kinsiamv]

thank you for the amazing tut, it really worked. Keep up the great work!

[@MrCaffeine-g4i]

Tack för Guiden! Tyckte jag kände igen dialekten xD

[@spacey5122]

thank you! got this working perfectly with this tutorial

[@shinnii3073]

thank you for the video :) Very short and helpful

[@yoonah9869]

Thank you So much!! I didnt saw the 3rd Party Thing on the bottom.. Thanks!!

[@Jordi_214]

I have a question. Can you use this method to have VtubeStudio handle the expression capturing as well? Everytime I activate expressions it uses my PC's webcam instead

[@nuekaze_media]

I think that expressions are only for use with OpenSeeFace/built-in tracking. You can not use it if you use VTube Studio. I have not got it to work at least.

[@russianotter]

I'm having the strangest of issues. Vseeface is connecting to vtuber studio on my iphone, I can see the connection incoming and it dims the screen and says streamer mode enabled. But absolutely nothing happens in vseeface, no movement, no blinking.Wondering if you have any suggestions.

[@aquafox_vt]

Same

[@Ailyn_101]

Thank you! You got a new sub today! 😊

[@PavisGladius]

tysm this was very helpful

[@planchettetheghost2299]

Thank you so much! I tried using Waidayo but it randomly stopped working and I tried everything to get it back! This saved me!

[@Catmyau3]

Hii! How to do vtube studio to vtube studio

[@minuettu]

this worked! tysm lovely

[@ssniper_cat]

Vet inte om jag har råd att köpa iPhone har redan en Android men funkar webcam bra eller ska jag satsa på en iPhone? Sparar också just nu för att ha råd med en PC

[@nuekaze_media]

Just denna guiden är för att använda iPhone men man kan köra VSeeFace med webcam också. Till Andoid finns det en app som heter MeowFace som kan vara värd att testa. Den ska fungera ganska lika tror jag. Annars fungerar webcam också bra så länge man har bra ljus i rummet så den hittar ens ansikte ordentligt.

[@vamyuna]

Heya! C: I need help! Which Version of VSeeFace are you using? Even though my Iphone says its connected, it doesnt work and I dont know why :C My 3D Artists also gave me the right ArKit file so it should work

[@nuekaze_media]

Hi, Do you know if the VRM file contains the ARKit blendshapes? I think you can import the VRM into Unity to check. If everything is set up correctly with the VRM file it should work. There are some free ARKit avatars you can try just to make sure the iPhone connection works. github.com/hinzka/52blendshapes-for-VRoid-face I don’t remember which VSeeFace version is used in the video but you can always try the latest from the website.

[@Patchywork]

Followed all the steps but it wont work. I have a model that works with my webcam so i know it should work but I cant get the two apps the connect. Yes theyre on the same wifi. Yes I checked the IP address for my phone. Idk whats wrong at this point

[@nuekaze_media]

In the VTube Studio app where you activate 3rd party clients it should display if any device is connected. If the status icon is yellow then no device is connected. Make sure this connection works first.

[@PIZZAPOWERS]

great video thank you! everything worked BUT.. my eye tracking is just not working and I have no clue why :(

[@nuekaze_media]

Make sure all the bones and blendshapes are set up correctly. Does the eyes work with webcam tracking? Does you model have ARKit blendshapes?

[@PIZZAPOWERS]

@@nuekaze_media it does not work with webcam tracking, it’s just straight up made & taken from VRStudio (PC), didn’t know I had to do something else as well

[@ricewzrd_alt]

Works for me

[@Anxiety_Gh0st]

Ça marche sur Android ?

[@jampyplays9729]

what was the second app you use to activate the traking?

[@nuekaze_media]

The only app running on the iPhone is VTube Studio. The tracking starts whenever the application is running in the foreground. It will not run if you close it or try to run it in the background. Just make sure to activate the 3rd party clients setting.

[@AlweizSummer]

Does it only work with iphone? Can it work with a Samsung s24

[@nuekaze_media]

This is for iPhone specifically. For Android there is a tool called MeowFace that you could try.

[@AlweizSummer]

@@nuekaze_media thank you!

[@kaleidoscope3433]

how do i change settings it doesnt have that option for me in vseeface

[@nuekaze_media]

Maybe your version of VSeeFace is to old. Try downloading a newer version.

[@flatearthrec9484]

its not working at all my VRM is frozen what am i supposed to do

[@nuekaze_media]

Check if the VTube Studio app has any connected clients. It is visible right below the 3rd party apps setting that you enable. Make sure that your phone is connected to the same wifi as your computer. If it is connected you can try using another VRM model that has known working ARKit blendshapes. github.com/hinzka/52blendshapes-for-VRoid-face This model is known to work. If this model work but not your model, then the blendshapes are probably not set up correctly.

[@YOKERMEISTER]

Nice tutorial thanks! Is there any way to send that to VSeeface using Vtube Studio for PC?

[@nuekaze_media]

I have not used VTube Studio on PC so I don't know it the option exist. If the "3rd party PC clients" option is there in settings I assume it could work but I do not know.

[@rade_rizu]

thanks

[@OK-Virtual-Show-ENVtuber]

Thank you for the help, but am wondering is it harmful for the iPhone battery?

[@nuekaze_media]

Any facetracker app will use quite a lot of battery. I would not use it without a charger plugged in. I don't know how the iPhone handles power management but as long as the battery is not drained it should be fine.

[@OK-Virtual-Show-ENVtuber]

@@nuekaze_media thx

[@OK-Virtual-Show-ENVtuber]

@@nuekaze_media hello bro am having a new problem now i bought a new router to upgrade my internet speed now vseeface not tracking my iphone from vtube studio can you help me on this one please!!!

[@nuekaze_media]

@@OK-Virtual-Show-ENVtuber If you change router, all your devices have probably gotten new IP addresses. Make sure computer and phone is connected to same network and check iPhone IP again.

[@OK-Virtual-Show-ENVtuber]

@@nuekaze_media I did that first thing, nothing happened

[@javigunda5399]

i simply dont have the option to choose "iphone/arkit tracking reciever"

[@nuekaze_media]

Your version of VSeeFace may be too old. Try to get the newest version. I think it was added in like 1.13.36n.

[@javigunda5399]

@@nuekaze_media thank you so much! i got it to work now

[@kaotickitten]

I dont even have an option for third party clients to connect.

[@marik_hadare]

It is same as using IFacialMocap?

[@nuekaze_media]

I have not used iFacialMocap but I guess you configure it the same way. Enable some setting in iFacialMocap maybe and enter the IP in VSeeFace as I show in the video. But select iFacialMocap instead of VTube Studio in the drop down list.

[@sadieparker4715]

vtube studios says its working but its not tracing my face

[@nuekaze_media]

Does the character in the VTube Studio app move?

[@BabuPookz]

its not working

[@thekingofcookies4877]

done everything step by step exactly the same, doesn't work and yes I am using the model it should work for.

[@nuekaze_media]

Make sure that the iPhone is on the same network as the computer. If both computer and phone is on same wi-fi is should be okay, but of computer is connected by cable it may be on different. Network connection is the only think I can think of if all else is good.

[@blindguitarfox]

@@nuekaze_media your iphones wifi has to be switched off of private when you are connecting to the wifi

[@Daniel-ht3bq]

[@edgyboi5816]

Broh se autodoxeo

[@AsashinDaka]

doesnt work

[@nuekaze_media]

Could you elaborate?

[@AsashinDaka]

@@nuekaze_media i follow the exact steps and nothing happens. the avatar doesnt move on the phone or vseeface

[@a.j.wildes4467]

Same for me, the mouth tracking seems to work on a massive lag if it works at all. :( No head movement at all.

[@SpaceSentinel]

@@AsashinDaka Also have this issue. EDIT: To clarify the VTube studio app tracks fine, but on VSeeFace the model doesn't move at all despite VTube studio showing that it's connected. EDIT 2: So I fixed this. Turns out, even after creating a rule in the firewall when you launch VSeeFace windows automatically makes more rules that block it. You have to go back and change the rules to allow and then it'll work.

[@MisBlackMutilation]

OMG THANK YOU! Apparently just like @@SpaceSentinel said just go back into advance --> firewall --> inbound rules --> scroll down the list to all the VSeeFace rules you will see some blocked --> right click on them --> properties --> Allow connection

[@Kiraluna03]

Android?

[@nuekaze_media]

Look for an app called MeowFace. It should be able to produce similar tracking.

[@LuvTonique]

"For free! :D First you need an iPhone" That is not free, that is $500 minimum away from being free

[@nuekaze_media]

It says iPhone in the title so I kind of assume that you already have an iPhone if you watch the video. And the app is free compared to other iPhone tracking apps.

[@LuvTonique]

@@nuekaze_media I didn't expect such a quick response or even a response at all and now I feel bad because re-reading my joke it looks like I'm actually serious and upset and another idiot on the internet who grasps at straws to find something to be angry about.

[@ViolettMouse]

Geht nicht

[@notyouraverageaustrian]

average finnish person sorry swedish doesnt make a difference tho

[@SilverRead]

정말 유익했습니다.