Ultimate Vtuber Application! FREE! Hand Tracking, Face Tracking, Body Tracking - VSeeFace

This video is demonstrating and providing a quick how to on how to use this free real time face tracking application with finger tracking.
I also stream on Twitch: / fofamit
This video is brought to you by Private Internet Access: www.privateinternetaccess.com...
- - - - Software you're going to Need - - - -
Waidayo for iPhone:
apps.apple.com/us/app/waidayo...
VSeeFace:
www.vseeface.icu/
Apple ARKit Ready Made Models:
hub.vroid.com/en/characters/7...
Where to buy or get other free avatars:
booth.pm/en
- - - - Hardware You're going to Need - - -
iPhone's with FaceID:
iPhone X: amzn.to/2EyOfyQ
iPhone XR: amzn.to/367d1kW
iPhone XS: amzn.to/30bbWF8
iPhone 11: amzn.to/2S2niXE
iPhone 11 Pro: amzn.to/2GbgBQl
iPhone 12: amzn.to/33643lJ
iPad Pros: amzn.to/2TeI6f5
Leap Motion Controller for Hand Tracking:
amzn.to/3fxgw5z
Webcam (used for body tracking):
amzn.to/3i9cKRz
-------- Other things supported -------------
To use these you will need to use an application which sends VMC protocol data like Virtual Motion Capture (VMC): sh-akira.booth.pm/items/999760
Vive Tracker: amzn.to/3pRdi3c
Tobii Eye Tracker: amzn.to/3pP4Elp
------ Related Videos -------------
Full Body Motion Capture Video:
• Real Time Face & Body ...
How to convert avatars to VRM:
• Converting Avatars to ...
Waidayo for iPhone:
• Waidayo: Free iPhone, ...
Buying your first avatar from Booth:
• VRChat Avatar 3.0 / SD...
VSeeFace Previous Video:
• Best FREE Vtuber App -...
-Fofamit's Recommended VR Computer for Streaming-
A computer parts list recommendation for something that will be quiet and cool, but also be able to run stream and vr at the same time. I picked parts based on what I currently use or would want to use in the future... also picked parts that would look nice together.~
Motherboard - Gigabyte B550:
amzn.to/3afGnxO
CPU - AMD Ryzen 5600X:
amzn.to/38s9duY
GPU - RTX 2070 Super:
amzn.to/33BWiVT
RAM - 16-32GB DDR4 3200:
amzn.to/31sWELI
Power Supply - Corsair 850W 80+ Gold:
amzn.to/31s69uE
SSD - Crucial M500 1TB:
amzn.to/3aaHkXS
Hard Drive - Seagate 4TB:
amzn.to/3gKSkOk
CPU Cooler - Noctua D15:
amzn.to/3fFV0eP
Computer Case - Fractal Define R7:
amzn.to/3kszJbV
#VSeeFace #Vtuber #free
---Content of video --
0:00 - Intro
0:48 - Filmed in VSeeFaced
1:47 - What you will need
2:26 - Where to get avatars
2:42 - Free Face Tracking Avatar
3:09 - Waidayo
3:29 - VSeeFace
3:39 - Activating Tracking
4:11 - What's wrong with your FFAAAACCCEEE?!
4:25 - Leap Motion
4:35 - ??????

[@Fofamit]

You can still use regular webcam tracking for basic face tracking! But the new update allows for full facial tracking and this video shows you how to do it~ You can also capture full body motion capture through a similar method as well :)

[@PhantomDragonexe]

so we don't have to purchase an iPhone to track and can use just a webcam if we can't really shell out the money for the leap motion just yet? Do you know if you can upload 2D rigs to this as well? Sorry that's a lotta questions in one go. Thank you by the way for this video, very informative and to the point.

[@itsmemaldy7071]

And for android ?

[@judaslink3588]

MystykM Art as of right now I don’t think any 2d rigs like live2d can use hand tracking or 3D tracking. I think they have to be made with vroid and blender

[@judaslink3588]

Its me Maldy Android currently doesn’t have Face ID which is what these programs use for detailed face tracking. Like winking and accurate mouth movement

[@fraizineberry7623]

If I'm poor what hand tracking/ finger tracking should I use? Anything as long as the hands move it's fine.

[@MrRourk]

Find a Xbox kinect and a usb cable.

[@queeffersthrlnd1620]

@@MrRourk do you need any program or does it just work with vseeface? Or do you need the vmc thing from patreon?

[@MrRourk]

@@queeffersthrlnd1620 not sure what software you will want to use. The kinect has been around and used for a long time kzbin.info/www/bejne/fJjIpZifh9GXiq8

[@MrRourk]

@@queeffersthrlnd1620 kinect 360's are super cheap. But have small usable field of view.

[@KaiTheBreadBoi]

@@MrRourk Dies the software you use allow imported avatars and stuff like vseeface

[@nyuchuarchive]

VSeeFace always been here for us! 👀

[@emi_vt]

Thank you for making this guide! 4:14 If this only happens when any blendshapes are activated, this issue is actually caused by Unity calculating incorrect blendshape normals when importing the FBX. It can be fixed by changing "Blendshape Normals" to "None" in the import options of the FBX before exporting it to VRM.

[@RiyaCelesiar]

program: free requirements: not free

[@KomsaVT]

Looking to get into vtubing and this channel has been the most helpful source thanks so much! Subbed and liked!

[@MadHouse_Eboy]

I AM TOTALLY GOING TO DO THIS. :)thank youuuuuu

[@MyHeartsKingdom]

I have a question, when trying to test my facial/body/simple movement tracking the fps says it’s around 9 or 10 and I move around slowly. Is there any way to bring the fps up back to 30? I set up my camera to 30 fps

[@gabriellecomeau2338]

I love your channel ! Good job on the tutorials and fofamit ...she's very cute ! ❤❤❤

[@v3x796]

oml ur the funniest vtuber ive ever seen on yt

[@fish1798]

Omg thank you so much!

[@IBubblesTV]

Bruh I same to the video thinking I didn't need leap motion 😭

[@charlizelima4565]

same here lmao

[@Assortment54321]

Thanks for this video! You are an adorbs person. :)

[@oalexo]

freEEeEeEeeEeEeEeeeeeeEeEeE

[@telstraroenssel3701]

is the leap motion camera thingie a must for better eye track and hand track? they r now working for with only webcame while the rest works quite nicely :)

[@interlewdcreations5604]

It looks like you are using a neck mount for the Leap Motion controller. Can you recommend where I can buy a good neck mount?

[@elsemusic3550]

Is there any full facial tracking option for non-iphone users? I'm having so much trouble finding anything!!

[@kwekdc9006]

Ikr me too! i only have android

[@mcmeadow8591]

I came here to ask this question as well

[@thefoxmoonlight]

VUP on steam, I cam here to attempt to spread the word of this app (since its much easier to setup) but this girl/boy (sorry the voice is hard to tell T_T) deserves recognition not hate comments.

[@gonasf]

Lol android

[@MrFelblood]

Anything that works with the VMC protocol should be fine. We'll probably see a lot of new contenders on the marketplace now that VSeeFace has thrown their weight behind VMC.

[@Karlobster923]

I'm wondering where exactly you position your iphone for this? since I can't put it straight in front of me (bec I won't see my PC screen lol), its angled upward on the desk and my avatar keeps looking up and all over the place lol How is yours looking straight at the cam?

[@EvanthePixelPusher]

For some reason this does not work for me. My computer is on the same network, the port numbers are the same. Am I supposed to change it from what it started with to a different number? For some reason my webcam pick me up but I get nothing from my phone and I am using an Iphone XR. Also Im not sure if you knew at the time but newer versions of VCface do not have the VCM recieve feature I had to back track to the same version you used in this video (1.13.34c) in order to find this feature in the options.

[@slaveNo-4028]

can you leave the Iphone part out? Like will a webcam suffice in producing some kind of face tracking? Or lets put it this way, if you save up 150 bucks, should you invest it into a leapmotion or a used, broken-display-ed Iphone X?

[@3v1l73ddy]

Any way to track arms and hands without leap motion? Even just basic tracking?

[@CrownPrincessofLunaria]

Can you use the quest two for hand tracking? Or do you need a special hardware

[@estellastardust1159]

I cant figure out how to send the info from waidayo to vseeface and there's only this tutorial to base it of. I tried googling how to co nect everything to the same network and i am not having much luck. I dont know what else to do

[@mediaisekai2935]

got a problem, everything is setup with my avatar on the app and i put in the setting but none of the iphone info is applying to vseeface

[@shugooii1434]

I have a question what about the battery live of your iPhone?

[@ako5481]

At home I have the Xbox Kinect, does it work with the application?

[@therigwinmivtuber4372]

Thank you. Just found your channel. I have been trying to use Luppet with a leap motion for some things, other things I use my HTC Vive with Eye Tracking in Virtual Cast. But this looks incredible. I am going to have to give this a try later today. Do I need to do anything if I made my avatar in VRIOD studio or should the VRM file just work?

[@Fofamit]

For use in VSeeFace you can only use VRM. So since you already have one you should be good to go :).

[@therigwinmivtuber4372]

Got it installed. Here is a nice thing about waidayo it allows you to load your characters from VROID that you upload to pixiv. So I got the face tracking via my iPad Pro perfectly with my avatar. Still trying to figure out the leap motion. Rewatching the video to figure that out. So here are some questions, I have Vive Trackers and a HTC Vive Pro with eye tracking. So to do the full body tracking, I don’t wear my headset do I? Cause that would mess up the iPhone face tracking. So do I put a tracker on my head? I have 5 of them, originally for waist, foot, foot, elbow, elbow. (For Virtual Cast). Thanks for your time and have you thought of doing a Collab with PhiaBunny over on the Virtual Reality Show?

[@Fofamit]

Yes you can use a vive tracker on your head instead of using a headset. Make sure you are using Virtual Motion Capture (VMC) to send the motion data or another application which can send OSC motion data to VSeeFace for full body tracking. I would love to do something with PhiaBunny in the future~

[@JackieJKENVtuber]

What if I just want to use VR? Can I use VR in conjunction with VSee face? My idea is just to have my stream on one side and on the other side my model with both the head and the hands tracked in order to interact better with my public

[@chubsnbiggie]

i tried these steps track my body but it doesnt read my arms, is the leap motion controller necessary?

[@ThatHippyDuck]

HAHA. *FREEEEE^ that was adorable

[@baehaqi30vf]

I like your tutorial

[@SargHWP]

Hey what the best software for hp laptops and not sketch

[@RichDiamond11]

so is there a way to do this with just the pc and webcam

[@doomblackdragon]

Is there not a program that tracks hands using your web camera like normal cam streamers.

[@turtleeatstomato8572]

is there any free way to get some sort of scuffed webcam powered body tracking in vseeface?

[@rezacahyaramdana7901]

If i dont have leap motion, can i move hand tracking? If use app it ?

[@EmulationGuru]

Your tuts are the best. I set this up in minutes yesterday

[@animatoonstv.2216]

Hi, does anybody know why the smart phone has to be connected? She said the web cam is the one tracking the face I do not understand. 🙄🙄😨😨

[@AzWild9020]

For the vsee face thingy application, I cannot open it for some reason, it says its not supported for macos. I think it may be because its a swf file, and I can never open swf files for some reason, can you help?

[@Weredah]

Something about one of your timestamps tells me you like to watch VCR Repair Men

[@watson9692]

So I want to be just hand-tracking, how can I do it? Not a whole body just a hand. It for my school project.

[@md278x]

Amazing Intro

[@Kareszkoma]

I love you, thank you.

[@call_me_hoja]

When i move, it takes 3 or 5 secs to move the avatar, its annoying, how do u fix it

[@atombomb54847]

idp.generic is apparently one of the files in veeseeface which my antivirus and google identifies as a virus. Could it be a false positive?

[@Sofia-hp8gc]

Hello, I am using a translator so you know, but why when I activate the tracking of the hands, does it not capture the movements? the same thing happens with my fingers because I have used a very good webcam and it still does not capture my fingers, what should I do? I was a little late

[@annat.500]

I'm on mac and the only program I found was 3tene and it doesn't even really track my lips :,( Does anyone know a better program for mac?

[@Will_Is_Coming]

Will it work with Kinect Xbox one?

[@AlyCatVR]

Man, I'm glad I've been pushing VSF for people new to the VTubing game. Shame that the Face ID app is exclusively for Apple (I'm an Android only user), but all in all, wonderful information!

[@ShonenVR]

I don't understand the role of the phone in all that we have to place our phone in front of us the whole time ?

[@AlyCatVR]

@@ShonenVR I think it's facial tracking data?? I've never seen or used an app like that, so I couldn't tell you unfortunately. ;(

[@emi_vt]

@@ShonenVR By using Waidayo on an iPhone, you can get some more detailed face tracking and do things like stick your tongue out, as long as your model has the necessary blendshapes. Android's face tracking SDK is not quite as advanced and the regular camera tracking probably already gives better results. Of course you can also use VSeeFace without an iPhone and track your face using a regular webcam.

[@elsemusic3550]

I'm on android too, have you found a way to do full facial tracking without the iphone?

[@emi_vt]

@@elsemusic3550 Android's ARCore face tracking is sadly not that impressive, so just it won't really give better results than just using the regular webcam tracking.

[@rosabellewakefield5444]

Is there a method for those of us with good phones?

[@Konfettimusic17]

im poor and don't want to use a device for hand tracking is there any way i can do it without a leap motion controller?

[@underbelly69]

tried Luppet demo (has leap + iphone face tracking but USD$60), and then tried 3tene (it's free + has leap, but no iphone integration) - then BOOM this! best of all worlds - and very responsive = LOVE IT

[@meowmeowm30w]

When using vseeface and I have issues with the eye tracking. Everytime I turn the motion/sensitivity up the pupils shoot up and to the right in its own so now idk what to do :(

[@telstraroenssel3701]

same problem here. and it doesnt track my shoulders, hand and fingers at all X"D

[@bettyyeet2453]

Will a iPhone 12 Pro max work as well?

[@josephpalmer691]

is there a way to do it without i phone?

[@aquapunkvods]

My face is a lil twitchy on it but it's such a good application!

[@QueenMsp]

Can I use my iphone with face id for a webcam if I dont have one

[@overlord_ora7648]

So you can use the vive trackers for full body here too O: ? I see it in the description. At 4:00 it's probably one of those check box options to turn it on?

[@Fofamit]

Yes if you use Virtual Motion Capture to pair with it~

[@connor-sw1hl]

Does it work with 2d models? or just 3d?

[@neonrainbow9033]

can this be done with an Xbox360 camera?

[@starii2173]

Me on pc: *Fades away*

[@Nekotico]

can i use kinect to track my hands? im a digial artist and i starting to sell avatars and i need to be sure it wokrs propertly before haha....ty for share this info

[@underbelly69]

leap motion is best bet

[@zukio6088]

omg why didnt i think of this

[@redjack2629]

You missed one. Threedposetracker is a nice compliment for Vseeface, too. :P

[@warlockeshane]

Love your videos! I want to try this app but it doesn't look like it works for Mac. Do you have any software you can recommend that is compatible with Mac?

[@Fofamit]

Unfortunately the only Mac Vtubing application requires the use of an ipad or iphone with faceID. So like an iPad Pro, or a iPhone X or higher. If you do have an iPhone/iPad with FaceID then this video will help: kzbin.info/www/bejne/apKpgKV-gbdsZ7s

[@cppgamingchannel3915]

Hi, @UCFFFT9fux06NW1Q9QeccJlA. Thank you for the tutorial. I wonder I have to get the Leap Motion Controller in order to get the hand and arms tracking, is that right? thank you

[@amberabby7502]

Do you know more about using tobii eye tracker with vrm? I have one for suva, but I'm having difficulties using the eye tracker with vrm. Thanks.

[@Fofamit]

You would have to use it with virtual motion capture

[@theOneJello]

can i use this for steam vr

[@lazyt3ddy482]

Are these custom or there own design they want

[@tokyohanamura]

That last part though... god that was the hand tracking problem I had 😂 Only thing is... is there a way to hide the Leap Motion 3D model in VSeeFace after correctly positioning it? I’m worried that it’ll look awkward with the 3D model hovering around the neck or chest area on stream

[@Fofamit]

Yes you just close the settings menu for Leap Motion Settings.

[@tokyohanamura]

Everything seems to function fine now, but occasionally the hand tracking will get wonky like the arms popping into place where my actual hands are or just floating. Pretty sure I gotta just mess with the reaction time settings some more, cause the visualizer shows it’s able to track the hands perfectly even when switched to head mounted mode

[@pogdog8655]

Can we do it with mobile camera?

[@lpspatronus9536]

Hey fo. Is there one that can track facial emotions with just a webcam?

[@Vayriah]

no other alternative to use for the Iphone bit? many of us dont use iphones

[@sheepishcactus9451]

can you use this with mmd models?

[@misticnight6118]

Will it work with glasses

[@Awesomwott]

Asking for a friend, they don't own an iphone. So I was wondering if you have other videos for face tracking without iPhone

[@iSahphie]

you said for more info at 4:08 check out this video but i can't find that video omg :(

[@battychaos4169]

What if I don't have an IPhone? Is there other apps for Android phones?

[@shaper_i-o]

Is there an android substitute for waidayo?

[@Lxcid279]

Anything I can use for face tracking and hand/finger tracking for free without leapmotion

[@menjivarTv]

Does this work for 2D models?

[@dingodooo1882]

and can it be done with android?

[@thisisaname3447]

Question, with this set up, will I be able to pick stuff up/interact with items for example in VR chat?

[@Fofamit]

no

[@Nekotico]

wonder how to connect the htc vive tracker to vsee face....im about to buy that thing but not sure how to do it, or to make an interface on unity? i see most ppl use unity cuz u can customize a lot....any clue of that? i work with unity to make avatars already but i wanna do the next big step with unity interface and shaders and stuff

[@Fofamit]

Check out my other videos~

[@Nekotico]

@@Fofamit okay i will n.n ty

[@thewaba5167]

Is there a android Waidayo equivalent? Cause my phone does have facial reconnection but I cant find the app there so, any ideas of other good apps?

[@SG7Music]

Meow Face

[@hiyokosaionji4910]

verseeface doesn't work on pc and why in waidayo I see no character on pc like how do I import it?

[@underbelly69]

do you mean VSEEFACE ? it's working for me on PC Omen laptop, with leap motion, iphone waidayo app = magic

[@MadiiiWorld]

i clicked this video so fast soon i saw free

[@leoace5614]

what about for iPad?

[@rikuobito3653]

Where do you get the attachments for your body that allow you to use the body tracking? Do you use strap-on stuff, or the thing you set down?

[@slaveNo-4028]

maybe hot glue a clip onto it.. seriously I don't understand why they designed it this way, like, no holes for screws, no clips, no nothing, all smooth, no way of pinning it anywhere/DIYing anything, except you have a 3d printer.

[@vieatiran2703]

What are you using right now

[@Sasha-lc8hv]

Can you get one for android?

[@taylatsiaousis2213]

for some reason when i download Vseeface it downloads as a file and not an app, do you know how to help me? T^T I've been tryna fix it for about an hour now T^T I'm crying on the inside

[@akiki_.]

so you have to go to it and unzip it, then if you have a admin pass, enter it and it installs an app (sorry i’m so late)

[@GregJoiner]

And if you are not an iPhone user and use Android instead?

[@zafuanadi]

how dose one stream with other people and their avatars with transparent background ???

[@Fofamit]

You can use a virtual camera and use the virtual camera in Discord.

[@summer8810]

What if your phone has face id but you don't have an iPhone?

[@Fofamit]

Needs to be an iPhone since Apple is the only one who has developed an SDK which people can access for free.

[@cafe8162]

Fofmait can u do A vtuber App with no leap just webcam?

[@kr1s_d3ltarun3]

At first i thought he was doing ASL while talking because he was moving his hands so much.

[@st4rxs_]

my veeseeface tracking is not working

[@hashbrowns.1]

Dang i dont have money to get leap motion

[@annetterains7927]

video idea: How to dowload a avatar from fiverr to vseeface

[@aethert0ast]

thanks, i make yt videos n stream and hate how i look so i turned to mocap.