Vtuber Lipsync Showdown! Simple vs Advanced Lipsync vs VBridger!

Pandoramonium.com for my Social Media, Twitch, and Stream Schedule!
In this video, I'll be comparing and contrasting the three major lipsync methods for Vtubers today, including the brand new option from Vtube Studio 5.0, Advanced Lipsync!
Our goal is to highlight the strengths and opportunities of each method, to give you a clear picture of each, and help you to pick which one is right for you and your vtubing journey!
Scene Transition Music: Fun in Kyoto by www.FesliyanStudios.com (Steve Oxen)
#vtuber #vtubestudio #vbridger #vteacher #vtubing #virtualyoutuber

[@Jiyuna]

I was here when it was posted :)

[@CosmindArt]

Im kinda confused as to why VTS is marked as "no cellphone". I've used my phone for the tracking since forever. The phone version of Vtube Studio has the "Stream to PC" option. (IDK if it's a pro-only setting or not, I've had it for so long I cant remember)

[@pandoramonium]

Hi @CosmindArt! At it's basic level, VTS uses a webcam. Getting the phone up and working is an advanced feature with a greater range of tracking. I did the list using the minimum bar for entry, with new people in mind. I've only used an iPhone, but as an Android user that was an extra expense for me. I don't want people thinking they need to buy stuff when it's optional, or a later upgrade. Hope that makes sense, and thanks for watching!

[@grey1350]

9:07 Looks like you didn't set up Simple Lipsync properly. Some of the boxes should be red or black, so different sounds could make diferent mouth shapes. All of them is green, this is why your mouth keep the same shape.

[@MuxiCatVT3865]

Can you use vbridger if your modal isn't ridged for it

[@pandoramonium]

You can use it, but you wouldn't be getting 100% out of the software. I used it before I rigged the vbridger mouth for my model!

[@vampiragoddess]

I decided to use a webcam for my facial tracking because vts on my phone kept eating at my charge. I could only use it for an hour and fourty eight mintues before it decided nope i need to charge. I figured with the webcam i would get more ease without my phone heating up on me

[@pandoramonium]

Hi @VampiraGoddess! What I do is have the phone plugged into charge while I use it and that works well for me with no heat issues and I typically stream 3hrs+. My phones tendeded to heat up the older they got. It may help to close out all other apps, dim the screen as much as possible, and maybe a cooler if things are too bad. I hope that helps, good luck, and have fun streaming! ♥️💙💛💚🦊🥷🏼

[@Poorfanqiang]

It can work normally, but really, but I really can't imagine that 50RMB can solve it. You actually need so much money there, which is only suitable for those with poor cameras to use. However, this microphone must have a low background noise to have good results. However, those who don't buy a good camera will definitely not have a very good microphone, and there will be current or noise, so they should install RTX Voice, But if his graphics card is too old or his graphics memory is too small to use the program, then I don't know what kind of person this feature is suitable for. Anyway, if you have money, you can't use it, and if you don't have money, you can't use it. It's a very awkward feature, good or bad

[@pandoramonium]

Hi @Poorfanqiang! Thank you for commenting! ❤💙💛💚 The prices I displayed are USD, so 50USD would end up about 360RMB (Chinese Yuan). The webcam I have is very basic, 1080p Logitech and it works well, so nothing fancy is required there. You are right about the noise levels, if your mic picks up any other sounds it will be translated into a mouth movement. However, sometimes this is fixed (or at least made better) with settings on the computer. If you have to have loud things near you and your mic like a fan, then you'll most likely need a better mic that can cut that noise out more easily. I'm not familiar with RTX Voice at all, but it may be helpful/good info for some viewers that need it. I don't agree with the feature being awkward or unusable though. All three I featured have an audience that will prefer to use them or need to use them due to limitations of some type. But can strive to use other methods as time goes on. Let me know your thoughts!

[@Poorfanqiang]

Of course, I agree with your point of view, but for most Chinese people, a 20RMB microphone is enough for them to use. The microphone I use is worth 50RMB, but its effect is very poor and there is a lot of background noise, even if I am in a very quiet late night. If I don't use RTX voice, these noises cannot be ignored, but the graphics card performance required for this software is not low, But if a person is willing to create a Vtube, then the money is not much for them. I have overlooked who is using this feature, and I am very sorry

[@DrackiVTuber]

Hi @pandoramonium I've recently got a new VTuber model and now trying to increase the lip sync as much as possible. It is okish at this point but I'm trying to get it as good as possible before I make content Is it enough to just connect an iPhone to VTube Studio via "VTube Studio" setting in VBridger or is it recommend to use an app like iFacialMoCap? What did you use in the video with VBridger? The best results I have found is with NVIDIA Broadcast and making my powerful webcam take as many frames as possible - 720p 90fps or 1080p 60fps. Are you aware of Nvidia Broadcast?

[@pandoramonium]

Hello @DrackiVTuber! I have always used VBridger with an iPhone as the camera, so I'm not sure about just the iPhone and VTube Studio by themselves. I'm still fine tuning my mouth movements so it does take a while to get just right. And I haven't used iFacialMoCap either. I have a help section on my Discord (link in video description) where I can help you more directly. Especially if you had a video to share about where things aren't working as you'd like or expect them too. What I've learned so far is the Input and Output values make a difference. Such as, if your mouth takes too much effort to open, you may need to adjust how soon VTube Studio considers your mouth to be open by moving the input value. I've heard of NVIDIA Broadcast, but haven't tried it out for myself. I get more range of motion and options from using the iPhone, so I just haven't tried it as a result. But I may end up doing so for a future video just to understand it better and help others do the same. Another thing to consider is pairing both visual input (webcam, iPhone, etc.) with audio capture. This can get things a bit more precise because it's more than just one type of data input, but would require more set up and tinkering to get it right. Feel free to drop by the Discord for some more direct help! 🦊🥷🏻❤💙💛💚 ~ Pandora

[@DrackiVTuber]

@@pandoramoniumThanks for the reply. Sorry for the delay - progress has been slow for me recently. Thanks for the huge detail - much appreciated I have liked NVidia Broadcast since having a central webcam giving the video feed to VTube Studio is very convenient. I have set up the Mouth Smile and Mouth Open parameters to be not too sensitive and making sure the mouth can stay closed when needed. I have also tweaked other parameters in VTube Studio. The only issue is that just having MouthOpen and MouthSmile can be restrictive. Sometimes Nvidia Broadcast can look pretty good but when you talk fast or do certain utterances, they are not picked up in a good way. I quickly realised with your video that VBridger can work with additional parameters and settings to help make lip sync more "human like". My rig on the model however does not have that. The model creators that made mine have brought up a VBridger add on but I don't know how advanced they can make it for the price they quoted. It is possible I may have to get them to reanimate the mouth entirely with a new rig - or even invest at a later date with someone that is better quality. It is hard getting it right whilst trying not to waste money. I have an iPhone now so I am trialling that with VTube Studio app just to see if I can get better standard capture and if it can perform better in a greater amount of talking scenarios. I only use the video input into VTube Studio because I prefer it only looking at my mouth state. If you have audio input too, I find that if you make a loud sound with your mouth not that open, the model will then have a more open mouth due to the compounded effect of the two inputs. I will definitely check your Discord thank you!

[@pandoramonium]

@@DrackiVTuber Welcome and no worries! I'll eventually try out and make a video on NVidia Broadcast, but so far I haven't used it. And as a fast talker myself, I ran into that too. I tinkered with the weights a lot, and eventually the way I approached it was just enough to keep the mouth closed, but I gave the open less and less weight until it could keep up with me. But sometimes it's also how well your face can be seen, so lighting is also important. VBridger basically adds an additional 2 sets of variables you can use to further refine things. The latest update added even more, including the use of audio. Adding features afterwards can be a bit tricky. Usually you're locked into who originally rigged your model unless you got the Live2D files so anyone else can edit it later. And depending on how much time has passed, they may not have kept your file so you're stuck with a full re-rigging. From what you said, I'd want to see some examples from other models they've rigged before to see if it's worth the cost. Otherwise, see how much it'd cost to get your original file so you can take it to someone else instead. So far the iPhone's been great so I think you'll like it versus the webcam! And you're absolutely right, inputs will compound, I found that out while making my videos and my model went full on eldritch horror lol Looking forward to having you on Discord! ❤💙💛💚🦊🥷🏻

[@DrackiVTuber]

​@@pandoramonium Yes I agree, the iPhone when playing with it is much better. It is even better now that I have tweaked it further and can catch up a bit better when talking fast without VBridger. I don't know how much extra rigging I need but I certainly won't rush the decision I also think that in VTube Studio, getting the talking right is kinda different for masculine models when compared with feminine ones - the mouths tend to be bigger and of course majority of the time the lips are less defined.

[@pandoramonium]

@@DrackiVTuber I'm glad to hear the iPhone is working out well! I can't say I'm familiar with a male model, but as an artist and a beginner rigger, the art, shapes and rigging methods are all the same. It should come down to the art prep and the rigging having the needed details.