Making a Realistic MetaHuman in Unreal Engine | Realtime lip sync with MetaHumanSDK

MetaHuman SDK is an innovative tool for facial character animation, powered by ML. This plugin for Unreal Engine streamlines the animation process, ensuring real-time facial expressions and lip-syncing thanks to our cloud service. Bring your projects to life with ease and high precision with MetaHuman SDK.
Use MetaHuman SDK for your creative projects and enjoy a variety of functions:
• Text to Lip Sync
• Audio to Lip Sync
• Language Independence
• Custom Model
• Chatbot Integration
Create facial animations in seconds and build interactive avatars with MetaHuman SDK!
Get the plugin for free from here: www.unrealengine.com/marketpl...
#MetaHumanSDK #lipSyncTech #InteractiveAvatars #gamedevelopment #artificialintelligence #generativeart #aicharacters #chatbot #chatbotdevelopment #unrealengine #avatar #virtualreality #madeinunreal #naturallanguageprocessing #DigitalMetaverse #Web3Avatars #CloudRendering #AIIntegration #VoiceAssistant #MicrosoftAzure #GoogleCloud #ConversationalAI #TTS #ATL #ChatBotIntegration #11labs #VirtualCharacters #AvatarCreation #MetaHumanTools #VirtualRealityTech #AvatarDevelopment #DigitalAssistant #Web3Development #CloudRenderingTech #InteractiveAI

[@jpegodzilla]

Absolutely stunning!

[@victoraddiamaadorortega7121]

What is the difference between audio-to-lipsync and the real-time-lipsync service that is listed in the available services in the chatbot subscription?

[@metahumansdk]

Hi! The chatbot can use streaming requests that generate animations in chunks, so the generated animation runs faster than a normal ATL request.

[@Vegaart777]

It’s driving me crazy , I have two avatars in the scene , ( with face animation created from audio and I have body animations as well, one avatar is perfect the other one has his had inside his shoulders when I apply the face animation

[@Vegaart777]

Ok it was because I had two metahumans in the scene , and bc I’m not too advanced I was using the same face blueprint for both , I just did them in separate projects. Not a solution but hey 👁️

[@metahumansdk]

Hi! We use same animation blueprints on the different metahumans and they works fine. If it possible please send us more details about this issue to the discord discord.gg/MJmAaqtdN8 or e-mail [e-mail] we will try to help you.

[@bruninhohenrri]

@@metahumansdk I have kinda the same problem. This plugin doesnt work with animation Blueprints for the face

[@metahumansdk]

@bruninhohenrri This video was created with modified facial animation blueprint. We also share it to users who ask for it in our discord, you can download Facial animation blueprint that we usied for UE 5.3 from this message discord.com/channels/1010548957258186792/1210158174133551174/1211563094808203274

[@bruninhohenrri]

Why the heck my code stopped working ???????????? The GetChunk function is returning invalid, and the code was like, WORKING, a few weeks ago, today a opened the same build and the speech doesnt work anymore ? Did you guys changed anything ? I'm not even using any AI services of ai, just the standard audio-to-lipsync conversion. What the heel happened ? I have a deadline on this project, imagine for my surprise, when i open it and the code doesn't work anymore....

[@metahumansdk]

Hi! We didn't change anything on our side, all depends on your tariff plan only. For more information please visit our discord discord.com/invite/kubCAZh37D or send an e-mail to the [e-mail]

[@sinaasadiyan]

hello I have developed a STT+chatgpt+TTS in UE5 , how can I use your plugin to lipsync ??

[@metahumansdk]

Hi! Sounds great! You can send your sound as 16-bit PCM wave to the Audio To Lipsync/Combo nodes and it should work fine to you. To generate animation MetahumanSDK plugin just need a sound and you already have TTS so it should work 😉

[@sinaasadiyan]

@@metahumansdk Great, As I have developed these parts separately, does it still require any token or API to use Metahuman SDK for "audio to lipsync" part?

[@metahumansdk]

@sinaasadiyan yes you need a token

[@stephanedavi]

Hi. Can I add an audio file to have lips-sync ?

[@metahumansdk]

Yes. We recommend you to use 16-bit PCM wave for it.

[@GrandePlansDigitalSoluti-rv3gp]

It shows as unavailable in Unreal Engine Marketplace.

[@metahumansdk]

Hi! Please reopen EGS launcher. It should fix a bug with unavailable content.

[@SK-hj1xh]

RVC models (Retrieval-based-Voice-Conversion) support?

[@metahumansdk]

Hi! You can use any sound source that you want to use if you can convert it to a 16-bit PCM wave. Our nodes should sork correctly if they receive correct sound data.

[@SK-hj1xh]

@@metahumansdk i mean that can i put my own pre training rvc voice model into metahuman sdk and after this my rvc will be working by text to voice in the project?

[@metahumansdk]

@SK-hj1xh we have no option to integrate external APIs into our plugin by users but if you alreadi have your TTS in Unreal Engine you can use it as source of wave files and send it to MetahumanSDK. Also you can try to modify cpp files of plugin which you can find in the Unreal Engine plugin's folder to integrate your own TTS directly in plugin.

[@hernanmartz]

Hey, what about Mac? The SDK is not available for it. ☹

[@metahumansdk]

At this time we have removed the Mac version due to a virus found for that platform in one of the third-party libraries we used for the plugin on macOS.

[@hernanmartz]

@@metahumansdk Virus on Mac? Hmm… ok. Hope it can be fixed asap. Thanks. 😃

[@abelboban4557]

i am not able to register at the site for the token

[@metahumansdk]

Hi! Please contact us through the [e-mail] We will help you with the token while our site on reconstruction 🛠️

[@Djonsing]

Realism doesn't even smell like it

[@metahumansdk]

We are sorry to hear about this. Could you please send us an email to [e-mail] about what you expect from a service like ours? It would help us to be better!

[@Djonsing]

@metahumansdk I don’t mean the lip synchronization to the speech (it’s really at a high level), but the appearance of the characters themselves, the title of the video clearly states: “Creating a *REALISTIC* metahuman”, so where is the realism here? Even a schoolchild will immediately understand that these are computer characters... Watch videos about Metahumans by blogger JSFILMZ or others like him, that’s what I understand *REALISM!*