Creating Real-time Lip-sync in Unity Using Code

[@ArtellBlender]

Tested and works, huge thanks! Very decent quality considering the bare requirements. Tip: change the ThreeLS script to "LateUpdate()" instead of "Update()" if you need to apply it over existing animated blend shapes (FBX files may have animation tracks zeroed out for all blend shapes)

[@ritamr2690]

do you have the unity project? if so, would I be able to use it?

[@ChristopherYabsley]

Hats off to you man, this is such a simple solution. Thanks for sharing!

[@hyperdevil9997]

Thank you for this great tutorial! How are you importing blend shapes for this model? I'm using the FBX downloaded from CGTrader but blend shapes are not available in the skinned mesh renderer comp.

[@MuxavirGee]

Thank you very much, it worked.

[@codedesigner]

Glad it helped

[@АлександрЭльмуратов]

What exactly is coroutine used for in the Start method in the ThreeLSControl class? Is it possible not to use a coroutine here?

[@mikeb3172]

The glow in the hair is caused by "preserve specular lighting" and/or depth prepass. You can set alpha clipping off and just use shadow threshhold

[@codedesigner]

Thank you for your feedback, but I just used the model out of the box to show the lip-sync script.

[@leelavinodh]

Its working in unity editor but not working in webgl build. Can you please help me with this

[@leelavinodh]

The teeth is not moving along with mouth open. How to get this addressed

[@duaneamos619]

Very good tutorial. No animation yet on my model, but I will play around with it. Instead of a microphone, I want to add ChatGPT Unity SDK. Can you tell how to add AI? Thank you!

[@codedesigner]

You need to be sure the blandshapes belong to the children of the gameobject with threeLS script(or change the script as your need). I haven't worked with Chat GPT in Unity yet, but if the sdk generates/produces an AudioClip, which can be added to an audiosouce component, then you can use this method. If not, you need to use a text2speech asset to create the audioclip at runtime using Chat gpt response.

[@Leyverse]

GREAT +1 SUB!

[@du.4875]

can someone share the script? thanks

[@mediaexposed6256]

you have to zoom in your video as your screen inches are high and thats why its hard to clearly see small things

[@codedesigner]

The videos are 4K if you open them fullscreen, it is clear to see. By the way, I will zoom in on the code next time

[@gleetch_yt]

Hello, pretty useful and simple script, but it doesnt works on a WebGL build, do you know how that can be fixed?

[@codedesigner]

Hi, I have never tested it on webGL, but probably the issue could be related to the limitations of audio in webGl. You can read more here: docs.unity3d.com/2018.3/Documentation/Manual/webgl-audio.html

[@sev8798]

Ehi, I also have the same problem, have you managed to solve it?

[@woodsmanthepeasant]

👍thank you

[@biocronic2986]

music in the background

[@ПрапорщикЗадов-т7д]

Hello, is it intended for UnityEditor only or it is possible to build it?

[@codedesigner]

Hi, Yes this script works fine after building and on the real devices too, including mobile devices. It uses blendshapes to simulate the lipsync

[@josepalemany]

I can’t download the script now, it is still available?