Unity VRoid Eye Tracking and Blinking in 6 Minutes or Less

[@ReForgeMode]

Didn't like the video? Comment down here to tell everyone if there's something wrong in the video!

[@gugukuri1267]

This is the best strict to point tutorial on VRM eye tracking= ) Thanks!

[@ReForgeMode]

Thank you! Glad I could help! Also, nice profile picture!

[@ReForgeMode]

Now with a better animation, thumbnail, and GitHub download page! I've been kinda busy for the last 2.5 months with my main channel and a broken computer. But now it's fixed and I'm back! The promised facial expressions tutorial will require more time to make. I figured you guys deserved a good and clear tutorial and not some half-baked instructions!

[@Shihiko]

Actually looked at your channel today to see if you updated earlier, but I guess here we are. Let's see what this one is about.

[@ReForgeMode]

Hello, there! I hope this didn't disappoint!

[@Shihiko]

@@ReForgeMode looking forward to the next part. ;)

[@EnderGamer402]

dude, i've gotta say: you make excellent tutorials. instant subscribe. :)

[@ReForgeMode]

Hehe, thank you. Glad I could help you!

[@lucynavt]

amazing video as always!

[@ReForgeMode]

Thank you!

[@FabianMerks]

That is so much value. thank u so much!

[@ReForgeMode]

Glad I could help!

[@Mikes-Code]

Hey, someone knows how to deal with black flickering on the model when it has an animation. they appear on the neck, sometimes at the edge of clothing. But only when there is an animation playing.

[@ReForgeMode]

Ohhhh, yeah that is a shader error I think. Let's talk about it on our discord server! (Link in the description)

[@RalfZzZ]

The newer version of VRM doesnt allow me to override the eye close blendshape. I can disable the VRMInstance component but I also want to use it to make the character look at a transform. What workarounds are there to still use the Eye Blinking script but with the VRMInstance?

[@ReForgeMode]

Yes, it kinda sucks that they combine LookAt, Boobs and Hair physics, and Eye Blinker. I do have a tutorial for eye blinker that disables VRMInstance: kzbin.info/www/bejne/hpbPeoSIj5VrsLs But right now, I'm looking for a solution. VRMInstance also messed up the hair and clothes physics when your character move around. I'll make a video if I find the solution.

[@RalfZzZ]

@@ReForgeMode Thanks for the reply! I hope you can find a solution. Your tutorials are amazing!

[@ReForgeMode]

Hehe, I'm just someone who likes to help people and get paid by youtube moneh! 😂😂

[@Massi_]

This is amazing, could you please explain how to turn the face towards the object? I need it urgently...

[@ReForgeMode]

It's not easy to explain it in a comment section because you need to add new code into the plugin. The video should be up later this week

[@ReForgeMode]

The video that explain this is up now right here! kzbin.info/www/bejne/nnK0g55uqJWCpZo

[@Massi_]

@@ReForgeMode Thank you!!

[@zaichenhu679]

I've also noticed that many of your tutorials use plugins for VRM 0.0 instead of 1.0. For example, I can find 'blinker.cs' in VRM 0.0 but not in 1.0.

[@ReForgeMode]

Yes, because back when this video was released, there is no VRM 1. It's a new addition last year.

[@zaichenhu679]

@@ReForgeMode Previously, I mentioned an issue where I used VRM 1.0, exported to FBX, uploaded to Mixamo, and then imported the animation into Unity. The upper body of the animation is distorted. What could be the possible reasons for this?

[@ReForgeMode]

@@zaichenhu679 That's what generally happen when you uploaded VRoid models to Mixamo. Just skip the step where you upload to Mixamo and just download their model. Then, use their animation and import setting as instructed in the video.

[@ComfyKashi]

Great tutorial, thanks And I have a small question, How can I change my default face expression in unity?

[@ReForgeMode]

You can open the Blendshape of your character from the MeshRenderer component. There will be a lot of sliders there to control every expressions 😃😃

[@ComfyKashi]

@@ReForgeMode but how can I for example put the joy expression at start without sing the slider

[@FeyHunt]

@@ComfyKashi You can make an animation to change the values, and play it in the animator component. I will make this tutorial soon.

[@moonlight7844]

I fixed every problem I had but how do I make these models quest compatible

[@ReForgeMode]

Quest? As in Oculus Quest? Or quests in RPG game genre? I'm afraid I don't have any VR headset to test VRoid models right now. But if I do get a VR headset in the future, I'll be sure to make tutorials for it in this channel. If you mean quests in RPG, I think I can do that. But it would take quite a while before we get there, since we've only just covered basic movement in Unity.

[@xurvis2789]

I'm assuming you are trying to port it over to VrChat for quest users to see. For that you'd need to make a new project with the same avatar ID but when launching the project make sure you do it in the android version. Afterwards it should be simple as long as your polygon and dynamic bone count are good.

[@Rioni]

In my case the head doesn't rotate towareds the target but it rotates in vseeface. Why do you want to make it rotate in unity too?

[@ReForgeMode]

My channel is leaning more on game development using VRoid models. In fact, I don't even cover usage in vtubing softwares. This look at function is a neat feature to make your character feels more in tune with their environment. See Genshin Impact for example.

[@bini7203]

Awesome !!

[@ReForgeMode]

Thank you!

[@hysyoi]

Thank you very much!

[@ReForgeMode]

Glad I could help!

[@march9004]

Hi ive exported a character from Vroid and imported it into Unity but when I try to preview the blendshapes the avatar doesnt do them. I clicked on blendhape avatar and select blink. sad etc. but nothing happens on my avatars face. Is there something I did wrong in the export or import? Im using the new version of Vroid if that makes a difference

[@ReForgeMode]

You can't preview the blendshapes from the blendshape files, but you can preview it from the Skinned Mesh Renderer component at the Blendshape section.

[@FabianMerks]

You are a damn genius

[@ReForgeMode]

Haha, so I guess that works for your use case then?

[@FabianMerks]

@@ReForgeMode yes :)

[@yongha1273]

Thank you for great video! However, my avatar's left eye moves well, but right eye doesn't move. Could you tell me why this problem happened? I checked both look at bone applyer's left eye and right eye.

[@ReForgeMode]

Hmm... I could think of two possibilities: 1. Your right eye actually moves, but it doesn't move that much because of the camera angles. Try to move the Target object very far left and right to see if the right eye actually moves or not. I could bet that this is the most probable cause. 2. Check the LookAtBoneApplyer component. You did say that you assigned the Left and Right eye, but is it actually the right eye? Maybe you accidentally picked the Right eyebrow or another "Right"-named object instead. It happened to me before... Let me know if this helps!

[@what8020]

is there a way to coordinate the blend shapes in a unity timeline

[@ReForgeMode]

Yes. You can technically build individual expressions as animations and put them in a timeline. The next video is all about that, but not yet into Timelines.

[@lithmariel]

I have no movement at all, any idea how to fix it? if I edit the eyes manually they move, but outside that nothing. Both eyes are listed exactly as you show. Look up/down etc are empty in blendshapes.

[@ReForgeMode]

Hmmm... I assume you mean Head movements? Have you tried changing the Update Type just like in the video? When we edit the script so that now the head moves, the eyes won't move to follow the object because we're always looking straight into the object.

[@lithmariel]

@@ReForgeMode Appreciate the response! :3 Writing with little time, if you need more details I can record a video/take some screenshots later! But what I mean is eyeball movement. I exported from Vroid Editor, and inside vroid editor I can see it moves. But in Unity, there is nothing on LookUp/down blendshapes, and following this tutorial [including the part about changing Update type to Late update], nothing moves at all. I''m using an external tracker that also doesn''t recognize any blendshapes for eye movement, I''m not sure where the problem is.

[@lithmariel]

@@ReForgeMode Welll... this does not solve the unity issue, but I was able to fix the issue on the tracker. Since this was my main goal feel free to disregard this. Appreciate your time

[@ReForgeMode]

I see... Did you use UniVRM plugin to import your VRM directly to Unity? This can get quite complicated if you convert VRM files to FBX. Some data from VRoid Studio might get lost in the conversion, including blendshapes If this doesn't solve the problem, do send your screenshots to my email (in my about page) or comment down here.

[@ReForgeMode]

Anytime, friend. I'm happy to help!

[@zaichenhu679]

After downloading animations from Mixamo and importing them into Unity, when I copy them, the animations are crooked. Why is this happening?

[@ReForgeMode]

This is unrelated to this video. Watch this one instead about using Mixamo Animations with VRoid models: kzbin.info/www/bejne/apq0eqR4asqqhdk

[@zaichenhu679]

@@ReForgeMode I just watched this video, thank you!

[@ReForgeMode]

@@zaichenhu679 No problem! Let me know if you need further help!

[@kilowattsup3885]

can this work for fbx models, or is it only vrm?

[@ReForgeMode]

The principle should work on all FBX model, but ofc the script we're using is provided for VRoid models only. It may or may not work on a random 3D model

[@echothorns]

How do I add my own model for this to work?

[@ReForgeMode]

I have made the tutorial right here: kzbin.info/www/bejne/f6Osn2yth76ki6s

[@piotrkrupczynski6054]

how to setup mouse position as target?

[@ReForgeMode]

It depends on your specific use case. It's not straightforward to just translate your mouse position from 2D plane (x, y in your monitor) in the 3D game world (x, y, z). It will be a variation of using mainCamera.ScreenToWorldPoint(Input.mousePosition). This assumes your current scene camera as the reference point of your 2D plane.

[@piotrkrupczynski6054]

i tried that but i have few cameras witch are swapping as main and this not work then, is it possible to folow object example prefab that will be sprite of mouse to follow? @@ReForgeMode

[@ReForgeMode]

@@piotrkrupczynski6054 Everything is possible with coding. If you have multiple cameras switching between each other, of course they will give out different values for the mouse cursor. My prior comment is not a silver bullet solution. It's a starting point. Without knowing full well what you're trying to achieve, and what you don't want to happen in your game. All I can do is to give suggestions. This time, I would recommend you to look up ScreenToWorld function earlier and see if you can manipulate the values to achieve your desired goal. Or maybe you need to change how your game design work.

[@piotrkrupczynski6054]

@@ReForgeMode ty you man for fast response i made special canvas with transparent sprite that follows mouse on active camera then i added this sprite as parent to follow eyes too works nice

[@Adrienaline-Rushed-Art]

I just can't figure out how to move the eyes in blender, is there no way?

[@ReForgeMode]

Well, you can do two things: - Rotate the eyes's mesh - Use some UV mapping to move the textures of the eyes And for advanced use, you can set up a rig that does the same thing I did in this tutorial. Very useful if you want to create an animation movie in Blender. Then again, it all depends what you're trying to accomplish. If you just want to make a playable character in Unity, the method I taught in this video should suffice.

[@dequentinmiller9086]

is there a way to do eye tracking WITHOUT pressing the play button?

[@ReForgeMode]

Well, there is. But what is it for? I can't think of a use case for Editor-only eye tracking. If any chance you're just trying to get your model working for vtubing, you don't need this tutorial. Eye tracking is natively supported by your vtubing application. You don't need to do anything except uploading your model to it.

[@dequentinmiller9086]

animated film and Blendshapes didnt work after pressing play button. (Face would go back to normal) I'm guessing it's a bug in VRM1 because that's what I was using. As soon as I switch to VRM0 the blend shaped worked. I was just going to record each scene in the editor instead of hitting play but I needed the eye tracking lol . But now it works perfectly with VRM0.

[@ReForgeMode]

@@dequentinmiller9086 Yeah, I can confirm there are some bugs with VRM 1. Do note that this eye tracking only work if you're animating in Unity. I doubt it will work if it's exported to other examples.