Track laughter and simple expressions with ARKit

Ever wanted to have laughing expressions etc on your avatar when using ARKit tracking? Today I'll show how to do this with VNyan's Expression Mapper without using additional webcamera.
Important links:
- VNyan: suvidriel.itch.io/vnyan
- Discord for support: / discord
𝐒𝐎𝐂𝐈𝐀𝐋𝐒 𝐓𝐎 𝐅𝐎𝐋𝐋𝐎𝐖:
🐦 Twitter: / suvidriel
🎮 Twitch: / suvidriel
💬 Discord: / discord
#vtuber #vtubertutorial #vnyan #envtuber
Intro:
Track: Phantom Sage - Our Lives Past (feat. Emily Stiles) [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Phantom Sage - Our Liv...
Free Download / Stream: ncs.io/OurLivesPastYO

[@Katubug]

This is incredible! You are a true innovator, and I am so grateful for your expertise!

[@CuteWaterBeary]

This is awesome! I love it! 💜

[@-.-._]

Good job, you are an incredibly skilled dev

[@Axo_Vods]

Miss Suvidriel you are so awesomme!!! I was thinking about this exact problem two days ago wondering if VNyan would be able to do dynamic expressions and here it is!!! Amazing work!!

[@ripkaylizzle2170]

Yass!! I was JUST going over a combination use of vseeface and vnyan for expressions since I just re configured my avatar yesterday lol this will be so helpful in so many ways!

[@Louberries]

Thank you so much for this!!! I personally don't use ARKit but my vtuber kids do and I'll be able to help them configure their models to be even more expressive now

[@SheepyShiro]

Thanks so much for this i was able to add so much more *character* to my model

[@artartzy]

Yay!!! You're amazing! 👏

[@grim_vestige]

Ohhh this is awesome!!! I might actually be able to save resources and just run vnyan with this!

[@luissantiago5163]

Super interesting! Much appreciated

[@sekinimation_play]

Great app 😊 I think it will more easy to create custom expression with slider like VUP app. Thankyou ❤

[@azurthedragon]

Thank you for your videos. I have a separate question about what kind of hardware/software you're using for the upper body tracking in your videos right now so you can move your avatar away from the virtual camera ? Is that the combination of an iPhone & leapmotion or just a webcam ?

[@Suvidriel]

I use either vr tracking or rokoko mocap suit. Depending on the mood

[@BunnyButtGaming]

You are so incredible!! Out of curiosity, do you take commissions on custom faces and adding this kind of stuff? I have watched all your tutorials, but I am still struggling to even add my own custom faces. When you explain it, it seems so simple, but when I go to do it, I can't seem to get anything to work.

[@Suvidriel]

I'm not an artist tbh so nope

[@siamsiam1205]

Is it possible to make an avatar that does not support ARkit work with an ARkit compatible application such as Meowface using the "I" and "U" expressions with parameters from 0-100?

[@tepafray]

Neato, I was doing this by hand using nodes before. As a request, could we get option to show the blend shape monitor with red to green numbers, and/or freeze the numbers?

[@Suvidriel]

Hm.. I'll have to think of this. Some indicator could be useful yes

[@nox_rayne]

Incredible as always Suvi! Just one question. Can I add a gif to my expressions instead of a static image? In some anime or games, some characters get a mess of squiggly lines above their heads when they're irritated or upset. Also, when you adjust the weights and values for jawopem and other expressions, do you need to do the same thing in MeowFace,?

[@Suvidriel]

You can add gifs as props in vnyan and make node graph show them when necessary

[@nox_rayne]

@@Suvidriel thank you!! ♡

[@Nani_wkwk]

after seeing the vnyan 1.1.4 update I'm excited to try the gamepad button/stick detection. but I don't know how to make it work. can you make some guide how to use the gamepad detection? is it gamepad detection like in vmagicmirror?

[@Suvidriel]

It's just a node for detecting the buttons and joystick states so it'll be up to the users to decide what to do with them. I know couple users already have plans for making similar gamepad "tracking" as in vmagicmirror so stay tuned in case those graphs get posted in the discord in the coming days

[@AMonstHer]

do you think you could do a tutorial on 3d rooms in VNyan?

[@Suvidriel]

Yea it's on my list. Sadly been too busy recently to record and edit tutorials

[@AMonstHer]

@@Suvidriel all good, take as much time as you need, I got it figured out. I hope that you are still taking time for yourself even with the workload! 💖

[@8l00my]

Can u use vnyan with iphone??

[@Guapitudo]

Sí, de hecho vnyan casi casi reemplaza a vsf la conexión es directa

[@8l00my]

@@Guapitudo cool! Gracias!

[@vinicastro9292]

I'm still blinking while joy tracking, idk why

[@Suvidriel]

You'd need to disable blinks in the expression menu. VNyan comes with some expression examples in the examples-folder under vnyan's installation folder that you could use as base. However, if you use ARKit blendshapes then you'll need to obviously change the blendshape names to be the arkit blink blendshapes

[@pastelartboy]

Do you need an iPhone. Can it be an iPad? I don't have a iPhone.

[@Suvidriel]

If it supports FaceID. VNyan can also do ARKit tracking from webcamera though