Adapting characters to use real-time facial capture | Unreal Engine

[@bev.tarzan]

Everyday, I get more and more scared. Unreal has meerkats now... and they can talk 😳

[@Jsfilmz]

hakuna matata

[@MagicSwordFilms]

This meerkat demo has consecutively solved all the problems I've been looking for solutions for

[@rob99201]

Quick tip - if you can't see the Live Link Debugger at 5:50, it is because you didn't toggle on the Live Link Curve UI plugin. ;)

[@borakian]

Great tutorial. Very clear and easily understood. Very creative.

[@TheAmoscokkie]

so far this is UE4 best features that do not need to rely on other external tools. This is then the right concept of selling UE4 apart from the recent short film event in Australia.

[@IRONFRIDGE]

Very easy to understand. Good Tutorial 👍🏼

[@KriGeta]

but how can we control the eyes? can we see and rotate our eye balls?

[@xbandpixels]

14:45 hahahaha its good

[@Wistbacka]

If only Epic put like 1% of this effort into improving their game store, it too would be amazing. Now it's a clunky mess. Seriously impressive stuff shown here.

[@rob99201]

To save people time creating the head rotation, I've added the following pastebin: blueprintue.com/blueprint/3_8k5j18/ Please note, you'll still need to make the actual live link node and create the anim curves - but this should save someone a few minutes at least.

[@erickmoralvocal8586]

Muchas gracias Christian! Eres muy inteligente y muy bien explicado, ojala despues subas un tutorial para animar los ojos tambien! Saludos!!!

[@DabblerDotUK]

I don't understand why Unreal Engine would push tech onto Apple platforms. What if Epic and Apple fall out?

[@inikko]

Apple and their products from Iphone X and up feature the Tech that works perfectly with face capture... Android is a mixed bag some may have some functionality some not, why would they release face tracking software that doesn't support 90% of the Android phones, And also every phone camera can be quite different so if the software was writen for specific camera it would't work on phones that don't have it and that's why iPhones are the best pick as they all ( from X and up ) have the Face ID sensor / True Depth sensor thing

[@MagicSwordFilms]

When I add the new curves to rotate the head they have X's next to them instead of a check, and the rotation doesnt work, even though I have it turned on in the app.

[@ankur1503]

Many thanks, Can we deploy Meerkat on iOS app with fur ?

[@gamedevstanislove]

Wow! Super-cool! Thank you!=)

[@ChiakaChima3D]

Very cool info and funny 😆😆

[@PolygonHive]

Unreal... making meerkats talk! 😅

[@FeedingWolves]

This is awesome!!! I just followed along. Is there going to be a continuation of this?

[@ApexArtistX]

How to steam audio ?

[@goll4m]

By the time you retarget the blendshapes/morph targets to the character, it looks like you get 5-10% movement. In order to fully take advantage of this tech, you need to blow out the blendshapes to 200% or more (play with it). So by the time you have it retargeted, there's a closer 1:1 ratio of movement on the character.

[@lpnp9477]

That's the motion smoothing. You can reduce this in unreal at the expense of a little jitter.

[@Andorade]

For sure. There are many things that can be done, including adding other blendshapes that use the same curve as another. And yes, all curves can all be calibrated or overexposed manually. But in the video it was tried to keep everything simple to be able to comply with the time, but it can definitely be explored in more detail on twitter or another medium in the future.

[@goll4m]

@@lpnp9477 I've tried to look up motion smoothing, for UE4 and nothing comes up. Is it called something else? Theres a motion smoothing SteamVR but thats it and it looks like its for cameras.

[@sk.mahdeemahbubsamy2857]

Can we get an app on Android please?

[@zorakbane]

Does this work with a webcam?

[@borakian]

No you need a iPhone X or later.

[@phantomgaming5199]

i dont have an iphone :v I guess my game will feature expressionless dialogues :3

[@YT-CesarB]

Lips do not match the words

[@Andorade]

As mentioned in the video, in this specific mesh, facial blend shapes are not compatible with the requirements of the 51 facial curves required by Arkit SDK. This is something that could be a limitation since there are many character assets that do not have more than a limited number of blends shapes. However, this is not a limitation as there are ways to take advantage of some of these deformers and make the mesh move. And that was what was done in this video. If you are looking to have much more control of the movement, even at the lip sync level, I would recommend trying the Face Ar sample or adding the blendshapes in an external 3D DDC tools

[@heidenburg5445]

awwwowoo thats great

[@innerguardianXIII]

ComparetheMeerkat. Now made even more Simples.

[@tnctr948]

very difficult and inconvenient to use

[@tutur42ful]

Do you know something more efficient ? @tnctr948

[@Gaamer-r9b]

my phone wasnt recognized in ue T_T

[@jeffrey5955]

Are you sure this tutorial is right?

[@huangti490]

when I create a human character to animated with blendshape and Niagara grooms，and the model are poke，grooms cannot drive correctly with blendshape, how could I fix this issue？

[@TURGiy]

Ue5 pls

[@FUNBBTV]

waow..i'm impressed :) too bad i don't have the latest iphone :(

[@fredrikekholm3718]

You don't need the latest iphone, it works with iphone X and later.

[@ManBeyondWorld]

hahaha it's really good

[@uja1096]

Do you have to have iphone 12 or can you do this with iphone 10 also?

[@fracturedfantasy]

iPhone x works... general rule of thumb is any iPhone without the "thumb button". I also have this working with iPad Pro 3 & 4. Demo here if you're interested: kzbin.info/www/bejne/iZmvYpZrbL-pb5I&

[@sigrid714]

Didn't Epic Games acquire Hypersense? Wasn't Hypersense working on something like this that worked on Android?

[@ogradus]

You should've included that it used an iPhone in the title. Hype == LetDown = true

[@borakian]

He did tell us in the beginning of the tutorial.

[@koko-nl5tp]

HAy.. Thank you. thank you.

[@Braindrain85]

Well, I don't have an iPhone :-/

[@reginaldarmah2381]

The instructor has a funny accent.

[@lpnp9477]

I'm not sure where he's from, but I would guess Mexico. I love the trill on his Rs.

[@lpnp9477]

@@christianandrade1162 i think you did a great job. I am spanish so it may have been easier for me, but i think you were clear. I learned a lot from this presentation.

[@marcfree2855]

I made a new meercat video in UE5:-)

[@WibuSableng]

this is What happen To Vtuber Code Miko and AI Angel

[@ALYTV13]

CodeMiko used a whole suit for about 30k

[@strikeforcealpha9343]

lol

[@MCJams]

Wtf is he saying? 😛

[@CHITUS]

Wow! Super-cool! Thank you!=)