The Future of Facial Animation with AI | Unreal Engine Metahuman

[@Unrealhan]

my 60s ARkit workflow here: kzbin.infoXCImJLD9FA4?feature=share

[@joshuacameron2008]

Excellent, comprehensive, brilliant, thank you

[@StephanBuchin]

4:17 Super creepy 😎 Great effect to show a robot freaking out and starting to lose control.

[@AnnisNaeemOfficial]

Thanks for this comprehensive video, Han.

[@workflowinmind]

Glad I discovered this channel

[@trickdiggidy]

🔥exciting series, looking forward to the next one!

[@gamedevsteve7388]

Aye Cory!!!

[@MortalShores]

iClone Acculips, with adjustments, layered in UE with ARkit passes, with adjustments, have gotten me the best results by far.

[@SaguinMedia]

Do you have a workflow tutorial on that?

[@MortalShores]

@@SaguinMedia I will probably do it after my next video release!

[@dan323609]

Ahahah

[@lionroot_tv]

Great stuff Unreal Han! Thank you for your suggestions!

[@davidjiang7021]

superb video man

[@nuzzmusic9859]

Learn something new from master Han today✌🏻

[@mm-hl7gh]

may i ask, what did you learn from this video ?

[@ryanwong1620]

Awesome content Han! subscribed and liked!

[@apixel.content]

Thank you for sharing your insights, looking forward to learn more from this site! Thank you!

[@jieda7303]

Awesome stuff! Very useful sum up for facial production.

[@kingsleyadu9289]

Awesome man thanks you so much, am waiting for more. 💕💕💕💕

[@jbach]

Really great content. Thanks for sharing.

[@rorozhy3480]

感谢分享，干货中的干货👍

[@samgoldwater]

Fascinating Han thank you so much for sharing! Brilliant deep dive 🙏

[@Silpheedx]

Thank you SO SO SO much for this research. I know it was a mountain. I definantly will stick with ARKit and additional post animation where needed.

[@beeceelad]

And now MetaHuman Animator is dropping soon. Exciting times!

[@kompst_tu]

I feel like with this ai matching system, the animations look a lot more emotional and believable!

[@ThomasMusings]

What an awesome video!

[@NetanelMaor]

Great video! Link to face good?

[@ohheyvoid]

Very interesting. Thanks for sharing. Great video ❤😮😊

[@naq_montages]

Love your content! Keep it up!

[@3dchick]

Really interesting!!! Thanks. Sticking to ARKit for now. 👍

[@DirectionGaming]

Is there any option for android alternative for ARkit

[@StephanBuchin]

11:09 Respect.

[@gladiyit6771]

What do you think about the new animation in UE 5.2 they have greatly improved it, please record a video!

[@artrush3603]

Sir can you make a full tutorial about that

[@CaptainPrice360]

Youre the man Han. Show us the ways!

[@Nickknows00]

Would love a tutorial on editing a base meta human (like heavily edit, turn the human into a orc or dwarf) in blender or zbrush then importing back to unreal with all the facial rigging working, or even how to edit the blend shapes of the new edited character in unreal

[@Unrealhan]

It's coming for sure. Still need a bit time but def on the list!

[@N4S.M]

@Dahoony thank you!

[@ibingeyt6077]

Hey, have you seen Unreal Engine's 5.2 Realtime Facial Tracking Animation Demo? Are you using the same tech/approach in the video?

[@TheAF01]

Amazing explanation and sharing of knowledge! Insta sub!

[@3djimmy]

Great stuff keep going

[@echoperformances]

Excellent breakdown! Thanks for putting it together! I’m glad to see Faceware Portal get a quick mention at the end. As a long-time user of Faceware, I know the barrier of entry is a bit steep for indy devs and beginners, but I believe Portal eliminates a huge portion of that front end, especially when it comes to building a tracking model. The fact that Portal is a cloud-based neural net solver, I expect it to continue to evolve and improve over time, and hopefully become more accessible to a wider audience.

[@Unrealhan]

Totally agree. It's not easy to get Faceware for sure. Love to hear your thoughts on it in terms of quality tho. Especially the new portal.

[@echoperformances]

@@Unrealhan I only briefly touched on this subject in my last webinar, but I hope I’ll have some more substantial feedback to share in the near future. Here’s a clip to preview those tracking results on static cam footage: kzbin.infoUgkxZbu8aJEZfmg7oBM01iaEBaBayFtcPRNO

[@dan323609]

What about combine and retarget 3d scanned per frame facial animation and morph animation? Maybe is not good and flexible for unreal but must be better result

[@GISxRayzen]

you deserve your name ! +1 sub

[@studio85amsterdam]

Does it run on an apple m1 chip. I heard that its complicated?

[@baseerfarooqui168]

what software or tool u use to track AI generated faical capture

[@AparnaModou]

Hi thanks for this! Can I get your opinion on upcoming image generators? Specifically Bluewillow as they are still in beta testing phase? Are they too late?

[@cucciolo182]

Facil ? What about body track to convert the data into a avatar ?

[@cirquare]

hi good stuff, which program you use to do the machine learning from tracking data to retarget data? and how did you put it back for auto retargeting

[@_vigosword2050]

您好，杨老师。我是一名正在学习ue的学生。想请问一下您的lip sync 中，手指涂抹嘴唇那个部分是怎么做的，可以简单帮我指个方向吗。非常感谢

[@davekite5690]

How about looking at iClone 2 Unreal?

[@Ronin2079]

Can I ask you where you got the assets you used to spice up your scene for your last video?? I thought they were assets scattered aroung the city already, but I cant find the movie theater entrance and storefront entrance anywhere....

[@Unrealhan]

kitbash3d.com/collections

[@Ronin2079]

@@Unrealhan awesome, thank you!

[@MR3DDev]

IMO I think I will be sticking to live link face app for a while and then manually correct in Unreal.

[@Unrealhan]

Same here.

[@yearight1205]

I have an android, not an iphone. But I'm working on getting an iphone (not to have as a phone but just for facial motion capture). Could you please put together a tutorial at some point on facial motion capture in Unreal Engine 5 please?

[@sfkim44]

Now Unreal 5.2 is out. What do you think about Metehuman Animator?

[@Unrealhan]

Game changer, great for creators and sucks for other companies. Can’t wait for it to come out.

[@sfkim44]

@Unreal Han I really hope they start enabling the LiDAR in the rear camera soon. The demo appears to be using the front cam which is SL based with low resolution

[@JeffreyChery]

I think having the unreal engine options completely work on a iOS or android device is truly being accessible

[@Picturevizcom]

Ar core supported android phone works here?

[@EGYPTIANKING24]

Do those metahumans' skins naturally deform based on collisions like a real human? I'm really wondering if this is happening.

[@A8EL]

this just got a huge update

[@Unrealhan]

Ya crazy stuff right? They cracked the code

[@Pauliotoshi]

For me, a big drawback of ARKit is that certain facial expressions aren't recorded into the blendshapes at all. For example you can hardly do non-symmetric brow up movement (🤨), worried face (😟) and sad face with hanging lower lip. I assume this can be solved by software in the future, but so far I haven't seen much progress in ARKit facial tracking last few years unfortunately.

[@Unrealhan]

good point. the hardware ability is there, just need someone to design the app specifically for us animators. I hope something is coming tho

[@Pauliotoshi]

@@Unrealhan yeah I hope so too. Although it is currently limited, it is very accessible both technically and financially compared to alternatives.

[@GaradrielS]

What about faceware?

[@Unrealhan]

Should be similar idea. I tested their lite version before, similar to ARKit. The new portal looks like an ai driven solution as well.

[@studio85amsterdam]

New sub here looking for good content on unreal for cinematic purposes.

[@pelomundo2080]

does it work on mac m1 max ?

[@sergeigourski]

What is your opinion of the new Faceware Portal solution?

[@bryansteagall]

Really interesting breakdown, and matches exactly my experience, although, the more you feed the FG AI, the better it becomes, so those places where it misses become less and less

[@Jsfilmz]

hey i know you!

[@jelowang4941]

@@Jsfilmz me too

[@dissonanceparadiddle]

I want to see AI interpolated in-betweens for hand-drawn animation that use a timing chart that you give it so it understands which friend a favor because right now that text just looks mushy cuz they're straight in-betweens

[@lolmao500]

We really need an AI that analyses like 10 minutes of videos of someones face and can use that data to make near-perfect copy of all the emotions of that person in metahumans.

[@vi4dofficial]

Give alternative we dont have i phone

[@Dohpamine_Design]

Me crying in 3DS MAX rip me, honestly i might aswell start some sorta RND for facial tracking for 3ds max, i need courage!!!!

[@agentdeluxe]

The ghetto camera rig was like $4k lol!!

[@josevieira3367]

top

[@jasonwong4996]

我也在期待那一天的到来，去释放arkit或者普通摄像头的最高的能力，AiGC的时代终将来临！！

[@mn04147]

if there's someone who looking for lipsync animation for virtual human's speeking here are some alternative ai animation solutions speechgraphics is amazing OVR is free Metahuman SDK Plugin is free

[@mn04147]

man....metahuman animator is comming...it use iphone depth sensor. wow.. what a fast

[@anandvijay6389]

🦒

[@soncho.editz1]

That video of the robot girl and the girl kissing looks inappropriate honestly

[@garethde-witt6433]

I’d rather it didn’t it’s bad enough without getting it to do that

[@mikeb3172]

Dude AI is useless. It's guessing + using more power. And people growling isn't a sellable product in entertainment. Entertainment is s break from that.

[@feli6821]

嗨，大佬，有办法把metahuman performance里面的voice combine apple arkit吗，实时live link这种，