Metahuman Animator Tutorial

[@Jsfilmz]

ITS ON LINKEY DONKEYYYYY! Here is video of the animator app studio.kzbin.infoiXH79mrKADM/edit First rap test kzbin.info/www/bejne/o3K3Yotqrcd4mqM

[@jimmwagner]

Do you need the depth capture in Live Link or can you just use video?

[@lukewilliams7020]

Yes so glad to hear you’ll be doing more tests with this. Bro just realised I’ve been on this unreal journey with you since your channel was a baby. You’re literally the don in this field. Keep em coming

[@MyChannel-l4w]

Bruh! It is nuts how accurate this thing is.

[@JarppaGuru]

track with python send to unreal.iphone xD.even creality 3d scanner is cheaper and would do better job.realtime.if someone actually make it work there is no SDK. it can make 3d scxan sure it can track

[@leonaraya2149]

Bro, you are a master! And the way that Unreal it's improving this technology day by day it's amazing.

[@Jsfilmz]

thx dude!

[@jimmwagner]

Knew this was coming the second I saw the Unreal post about this. Excited to try this out today.

[@PrintThatThing]

Incredible! You're so ON IT. Thanks for the walk-thru. Very helpful!!!

[@IDAVFX]

Thanks Lord Helmet!!!

[@JoshPurple]

LUV'd your last vid with your little girl (all of your vids ❤ ) 🏆😁👍!

[@Jsfilmz]

hahah thx man it got 20k views on twitter hahah

[@artofjhill]

yeeeeaaaaaaa 🔥

[@Jsfilmz]

hey J can i borrow 10k subs so i can hit 100k? thanks mang

[@user-ct8my8rv9c]

Good and fast tutorial, nice man

[@Jsfilmz]

tx

[@MiguePizar]

Thanks for the tutorial, I can't wait to try it, although it will take hours since the shorts that I'm doing have a lot of dialogue, but it will look a lot more realistic. Best

[@JDRos]

So how do we connect this to a body so we can add animations to it? 😅

[@lukewilliams7020]

The accuracy looks insane

[@Jsfilmz]

yea

[@fotenks]

I Hope that behind the scenes they are working on full body mocap

[@kool-movies]

just done my first quick test following your tuts to guide me through the process, got to say wowzer animator is amazing. so much cleaner then i expected. well worth the iphone rental :}. keep the vids coming :}

[@Jsfilmz]

oof someones bout to buy an iphone 😂

[@moneybagvr2113]

You the man bro

[@2beornot2bable]

Great job!

[@DLVRYDRYVR]

7:00 I'm listening to this video driving around, windows down, level 20 on Bluetooth and at a Bus Stop with little old ladies 🤦‍♂️

[@Jsfilmz]

lol

[@MANIAKRA]

BIG! I was wanting to explore this, thanks for the tutorials!

[@TheSabanrab]

Was about to say the same thing about your teeth JS 😂

[@ohheyvoid]

You're amazing. Thank you for staying on the pulse!

[@Jsfilmz]

thanks for being here!

[@Stonefieldmedia]

...and off we go!

[@sakibkhondaker]

Some day i think you will make tutorials before release. haha. So quick!!!!!!!!!!!

[@Jsfilmz]

hahaha yea they didnt select me for beta access unfortunately i know some did so im gonna have to catch up to them

[@christiandebney1989]

Awesome.. Randomly woke up at 4am knew there was a reason... thank you!

[@Jsfilmz]

hahaha

[@StyleMarshall]

sooo quick , Dude ! you are fast as hell .... 😁

[@Jsfilmz]

i woke up 5 am today haha

[@veith3dclub]

I need to buy iPhone 12 + first hahaha

[@Jsfilmz]

looks like 11 works

[@aaagaming2023]

@@Jsfilmz In their docs they group the X, 11 and 12 together and then 13 and 14 together. Then they give a caveat about the X, saying that its not capable of capturing more than a few sec, but they dont say that about the 11, so yeah, I think the 11 is capable of capturing longer form content like the 12.

[@andrewwelch5017]

@@aaagaming2023I tested the iPhone 11 Pro last night and it works fine, zero issues.

[@oldmatttv]

This really is such a big step forward. Very exciting. The only problem is the iPhone only requirement. That makes no sense for something like this, but let's hope it changes sooner or later.

[@Jsfilmz]

beats buying a real mocap system 😂

[@oldmatttv]

@@Jsfilmz True I suppose :)

[@guilhermenunes7075]

4:43 "Iden-TITTIES" hahahahahaha Great video man! Thank you for the content.

[@pokusalisobaki]

Thank you so much, its amazing

[@TetraVaalBioSecurity]

A lost detail in Hideo Kojima's DS2 trailer, was the end saying the performance-capture is powered by Metahuman. That game is going to be insane levels of detail.

[@Jsfilmz]

yea hideo was lurking around on my channel when ue5 first came out i was one of the first ones to cover it

[@natanaelgauna3600]

Amazing content as always! Could you please make a video on troubleshooting these three issues: - “Promote Frame” randomly jumping to a different frame than the selected one. - Metahuman Identity Solve not accurate result. - “Add Teeth Pose” breaking the Identity Solve even more. Thanks a lot!!!!!

[@3DComparison]

Great stuff!

[@Jsfilmz]

Thanks!

[@TheChrisNong]

I get the "assertion failed" crash every time I promote the first frame... what should I do?

[@DariuszMakowski]

Have u done any vids how to take that head with animation and apply to another metahuman ?

[@OfficialCloverPie]

my head keeps detaching from the body when i attach the animation to the face and play it in sequencer

[@jaykeastle8804]

BOSS!

[@D3Z_animations]

Awesome video! Thanks for the quick setup explanation! But I have to point out that MetaHuman Animator already uses AI for its solves. When you hit the "Prepare for performance" button, it trains a model on your face to later mimic the way it moves so it can animate other metahuman characters to that likeness. Thats why this step took 8-10 minutes : )

[@JoshuaLundquist]

Epic is great but they gotta give us some options with the markers, like maybe don't make the ones that go on teeth straight up yellow / green lol? Great video though.

[@tomhalpin8]

Were you able to figure out how to connect the iPhone to wirelessly be triggered to record from Unreal?

[@Jsfilmz]

just the regular livelink way?

[@tomhalpin8]

@@Jsfilmz With the new MetaHuman Animator version of LiveLinkFace, can't seem to connect. Trying to match my body mocap with the face mocap.

[@Jsfilmz]

cant do it live animator is offline

[@tomhalpin8]

@@Jsfilmz I wonder if I can I can use OSC to sync up the recording process on the phone with the mocap

[@binyaminbass]

when doing facial mocap, what do you do for a mic? do you have a little one that you attach to your helmet? what kind of mic is goof for that?

[@Jsfilmz]

just my good ole senheisser g2

[@binyaminbass]

@@Jsfilmz do you attach it to the arm of your helmet?

[@DLVRYDRYVR]

Hope my XS works. Don't want any more Apple pradux

[@IamSH1VA]

Please try it & please report if it works, I have same iPhone XS. I am not gonna have access to windows system for at least 15 days, but I am dying to test this feature.

[@sibinaayagam1838]

If I'm not wrong, you should be turning off the neck solving on a headmounted camera. And use it only for static cameras.

[@Jsfilmz]

in my case ill use neck movements from my mocap the exported sequence doesnt come with neck check out new rap video i uploaded

[@ArabicTechAILab]

when I play the level in a new window (PIE) the Metahuman character is moving his face with my motions very good, but when I click in any window other than the PIE window it starts to lag. and when I click again on the PIE window, the character moves normally ! what should I do ?

[@Griff_The_5th]

Do you think it's worth shelling out a bit more money for the iPhone mini 13 over 12?

[@Jsfilmz]

im broke so ur askin wrong person

[@omri1324]

What about body movement and hands?

[@benblaumentalism6245]

I wonder how long a given take can be. I have to make some EDU products with this, and they might need to be on the long-ish side.

[@DominicRyanOsborne]

When the teeth are facing you but the face isn't, must be metahuman, or a horrible childhood accident you didn't wanna talk about

[@kool-movies]

epic insane , looks much better then faceware or livelink .like you say the curves look smooth and no jitter . and to think i spent 4 hours last night doing 30 sec of manual facial animation that looks rubbish. so if i get a friend who has iphone they can send me clips ? .

[@Jsfilmz]

yes

[@kool-movies]

@@Jsfilmz awesome ,, is there any reason not to get the iphone 12 mini ?

[@Jsfilmz]

@@kool-movies i havent tested longer takes with 12 mini yet but it used to overheat on me alot hahaha

[@kool-movies]

@@Jsfilmz your rap video is a long take so hopefully will be good , i just ordered/renting a cheap refurbished one. hopefully arrives tomorrow. :} the 3lateral video that just released is mind blowing.assume they used a stereo camera ?,

[@HQLNOH]

Hey! I just wanted to say thank you for your videos! If it wasn't for you I couldn't have created animation for my metahumans. ❤ Thank you

[@Jsfilmz]

That is awesome!

[@HQLNOH]

Omg yes! My video will never make it if it wasn't for you! I even mentioned you in appreciation in the description ❤️

[@Jsfilmz]

​@@HQLNOHthanks man not manu give credit back to me i appreciate it

[@supermattis108]

So, whenever I want to make a facial animation (of myself doing it with live link) I need to go through these steps? But they can be used on every metahuman?

[@davidMRZ]

So strange, in Capture source whatever I put( live link or archive) it doesn't find it. It gets green in Capture manager but nothing shows up🤔

[@Jsfilmz]

hey man im uploading another tutorial stand by maybe u missed something

[@davidMRZ]

@@Jsfilmz great, thank you so much

[@sajans3998]

@@davidMRZ same thing happen to me. any solution?

[@dmingod999]

Did you see issues of floating head? Face animates and body remains still.. if neck rotation is disabled it's fixed but the neck rotation part of the capture is lost.. Do you know how to avoid that? Thanks! 🙂🙏🏻

[@AllanMcKay]

Great video - you mentioned about when MHA will later use AI. Just a heads up it is AI driven currently. Pixel tracking is only a small part of the foundation. Great video!

[@Jsfilmz]

wait like its doing ai pose estimations already?

[@AllanMcKay]

@@Jsfilmz yeah there’s a lot under the hood that’s ML driven already. Facial tracking doesn’t account for wrinkles or much else other than eye and mouth shapes, everything else is interpolated with AI. there’s more coming but the foundation is already utilizing AI in a lot of areas.The second pass animation is still being improved on, so it’ll continue to get better. But it’s a training model based on a lot of human facial animation, to know what to do when cheeks are raised, nostrils flared, eyebrow wrinkles etc Night and day different to something like live face or other tools which purely track eye and mouth shapes and don’t leverage any AI to them interpolate wrinkles and pseudo face muscles into the animation

[@Jsfilmz]

​@@AllanMcKayoh wow hahaha crazy stuff being 1.0 its not bad oh btw for iphone it can only output 30 even when recording 60 right? Thanks i love knowing about the tech

[@ResonanceRebirth]

JS can that be done with DAZ characters too?

[@Blairjones3d]

I may be a little late to the party, but im confused why you choose in CAPTURE SOURCE LivelLink Archives (which is uploading footage etc from PC drive?) then when you go to import you do it from the iPhone in CAPTURE MANAGER?

[@Jsfilmz]

i made tutorial both ways my usb transfers faster for bigger files

[@Blairjones3d]

@@Jsfilmz oh sweet as no worries bro. Any tips on uploading from USB then using archives? Seems more complicated to setup the Performance that way...

[@juanmaliceras]

it´s here!!!!! downloading plugin!!!!!

[@stevenbaker153]

Nice tutorial but can you please show how to get the iPhone to talk with Unreal engine online

[@Jsfilmz]

wdym

[@stevenbaker153]

@@Jsfilmz what I mean is how do I get my live link app to sync with Unreal Engine?

[@Jsfilmz]

​@@stevenbaker153this is only offline broski cant do it live

[@stevenbaker153]

@@Jsfilmz All good. I figured out my problem. I had the wrong IPv4 number in Unreal. Problem solved

[@michaelb1099]

is there a way to create the captures from a webcam?

[@jerrtinlsc]

So right now its only recording, but no livestream yet?

[@Jsfilmz]

live? nah facegood for that

[@sybexstudio]

Did everything to the t and my animation sequence doesn't show up in the Face animation menu. Any tips?

[@alexdjulin]

Thanks for the tutorial! Do you have a recorded take file I can use as a test? I dont have an iphone. Thanks

[@LucasGVoices_]

I’m using a D2 helmet Is there a way to do this without a smartphone app?? Sorry if you’ve covered this already, I’m new to face capture

[@Aragao95]

does someone got it working good with iphone 11? the new epic post says it needs aleast iphone 12..

[@Jsfilmz]

try it broski

[@cubiclulz]

can it be used not with my face but with the one created in metahuman creator? and how to do it?

[@Jsfilmz]

stay tuned

[@PecoraSpec]

can i move the mocap data to another software, like blender?

[@MichaelHurdleStudio]

This was jammed packed with so much information. I have to pause this video and slowly follow all of the steps. This is amazing! I'd like to purchase the headset for my iPhone. Did you make it yourself? or did you get it from a website?

[@Jsfilmz]

home made bro with my sweat and blood its not $129 anymore though its $169 now

[@itsyigitersoy]

Hi, i asked this to a lot person but couldn't get appropriate answer. After creating identity and perform with it, my Metahuman Character's default lips and teeth are changing, it's because of getting my identity. Is it possible to keep animation exactly same but with character's default facial features?

[@JonCG]

Hello bro.I just went back to this video ,becasue for me when update to 5.2.1 its says Preparation for Performance Fialed,any tips on it bro.

[@jonos138]

I have a problem that the character shows more bottom teeth than top. I dont speak like that andclive link doesnt do it. I tried re-tracking face markers but still had the same problem.

[@lilylikesherpotato3461]

add get hangs when I try to bake the animation....???

[@michaeleansan2730]

crazy times!

[@juanleche5382]

Idk what's going on its says that's its no longer available 😕 I been trying to this for weeks now

[@reubencf]

3:52 it's says iPhone 11 here But my phone is also 11 and over there on live link for metahuman animator I get that your device model is not supported you can continue to use but results won't be that good

[@Jsfilmz]

yea ive had people here try it

[@reubencf]

@@Jsfilmz I just went back to the site and it seems that iPhone X and 11 are removed

[@Jsfilmz]

wtfff on the docs?

[@reubencf]

@@Jsfilmz yes

[@RandallsDiversion]

bro mine keep crashin when loading trackers. Any suggestions?

[@Jsfilmz]

check hardware specs? not sure mine just takes a min

[@RandallsDiversion]

@@Jsfilmz I fixed it. Apparently if you create a duplicate project from an older version of unreal engine, it won't work. Thanks though.

[@mt_gox]

His next epic: "Get Shorty"

[@humbertomoli99]

Isn't it necessary to have an iPhone with this? I don't have an iphone sorry

[@ComanderJTC]

I thought when you're calibrating you need a minimum of 3 the front, left and right, and that the teeth only need one in front?

[@Jsfilmz]

turn ur head with helmet on

[@ComanderJTC]

@@Jsfilmz ah I see your point but wouldn't it be better for calibration to do it without the helmet to get the three angles then so the depth part of the camera recordings with the helmet on make it more accurate for certain things like creases on face and stuff similar to that when making facial expressions?

[@Jsfilmz]

​​@@ComanderJTCar as i know u gotta calibrate like how ur gonna record

[@ComanderJTC]

@@Jsfilmz ok makes sense I just figured it would read your face better if it had an initial depth analysis from multiple angles was my thinking

[@JoshuaLundquist]

Also hey I'd like one of those helmets, but I do have an iphone 11, will that work? It's bigger than the mini, of course.

[@Jsfilmz]

i know some people whos tried 11 with MHA and they said it works i havent tested it myself

[@JoshuaLundquist]

Gonna buy an iphone 12 mini like you have, can you tell me which mount you use so I can buy that and yr helmet? @@Jsfilmz

[@AbhinavVerma-l6v]

how to add my facial animation to different character

[@grossimatte]

Hey J, for some reason the result of your test is not very good, compared to other tests I have seen online. Do you reckon there was something in the configuration/shooting conditions that interfeered? Or perhaps the other tests used other types of cameras, such as stereo cams, rather than iPhone? Thanks for the tut though.

[@Jsfilmz]

i think the demo videos that came out were done with stereo cams not sure

[@grossimatte]

@@Jsfilmz The rig used when they announced MHA wasn't a metahuman rig, in fact they only showcased how the new system works on metahuman rigs for like, 5" at the end of the presentation.

[@Jsfilmz]

@@grossimatte hahaha thats cheating then right lol

[@prasoondhapola2875]

I don't have an iPhone. Will it work with my 2021 iPad pro ?

[@marcusjordan8036]

This is awesome! Would I be able to take the metahuman/face mocap and export it into blender? I don't need any of the textures or materials. Thanks!

[@Va.bene.]

hey bro! awesome! I have a little issue, when I track my face with the animation sequence it's like nothing happend in my sequencer, but it exported correctly because when I open it alone the animation is fine. any idea? Thank you!

[@Jsfilmz]

i dont understand :( maybe join unreal discord and post pics and issue there

[@RemDovans]

Maybe in a year or 2 Epic will give us a way to import our videos for mocap, I’ve tried so many apps and don’t like the results, only one left to try is supposedly the best one, move ai.

[@Jsfilmz]

yea its good man

[@inrapture]

is pasible only with iphone?

[@privet20005]

I was trying to make it work all night and it keeps crashing after first stage of tracking😭 I can make it work only when I select like 150 frames. I bought laptop in january and its already not enough😭

[@Jsfilmz]

yea its cpu intensive bro

[@privet20005]

@@Jsfilmz I hope upgrading ram to 64gb would help

[@Jsfilmz]

gl man

[@prodev4012]

Do you think there will be a marketplace to buy mesh to meta human data scans eventually? For example I don't know any korean girls to run this on with an iphone like that company did but i'd be willing to pay people who know attractive people from every race if they did

[@kameroco]

I've done it but how can I render it... help me pls

[@chipcode5538]

On the metahuman video they use 4 calibration images, is there a reason why you did not use side views?

[@Jsfilmz]

i have helmet on bro hahahaha they on a tripod if i move my head the camera wil move too

[@chipcode5538]

@@Jsfilmz You could make the calibration video with a tripod and use the helmet for the capture. I don’t know if it will improve the calibration, the metahuman seems a little off especially the nose.

[@Pauliotoshi]

Great showcase! In case you plan to make more test video's, can you show expressions that are hard to do with Apple Arkit? I'm curious how it compares. For example a sad face with hanging lower lip, asymmetric brow movement🤨, worried face 😟 or any interaction between teeth, lips and tongue.

[@Jsfilmz]

man im a terrible actor but ill try

[@Pauliotoshi]

@@Jsfilmz Thanks a lot!

[@Persianprograph]

Incredible! Is there a way to export Animation data to Maya to have more freedom for further tweaks?

[@Jsfilmz]

that would be amazing but i dont think thats possible yet

[@Luke_wp]

This guy made a script to do it kzbin.info/www/bejne/m5S8gGhjast_Zrc

[@Amelia_PC]

Could you provide me with the information about your iPhone 12 model so that I can search for a similar one to buy?

[@Jsfilmz]

12 mini broski

[@Amelia_PC]

@@Jsfilmz Thanks!

[@terry9183]

You're absolutely smashing it man 👏🏽 Thank you so much for the awesome content! I do have one slight issue though!.. For the life of me, I cannot get the Livelink Face mocap to work with separate body mocap. The head/chest just detaches itself and they are both independent. It's driving me insane. I've tried following some advice on the UE forums to no avail 😭 Have you experienced this yet? any tips? Thank you

[@Impaczus43]

Hold up! Did it just create a Metahuman face with your facial proportions just based on the iPhone calibration!? that's crazy!

[@Jsfilmz]

yes

[@Impaczus43]

@@Jsfilmz that's awesome! Glad you were able to share this! Awesome video

[@Jsfilmz]

@@Impaczus43 it made a fatter version of me which is effed up lol

[@soncho.editz1]

@@JsfilmzI mean honestly it does have your facial features but do u think you can adjust that manually ???

[@RikkTheGaijin]

is there a way to export the head movement? it looks unnatural to have just the facial expression with a still head.

[@Jsfilmz]

yes watch my tripod tutorial i uploaded today

[@RikkTheGaijin]

@@Jsfilmz it looks like that if you use a Metahuman that you already had before the latest update, it won't export the head movement, but if you download a Metahuman now, it will. I guess they have updated all metahumans in the Bridge catalog. I still don't know how to attach it to the body.

[@RikkTheGaijin]

@@Jsfilmz I watched it but you are not showing how to attach the body, you just say "I will import a body animation"

[@muviing5427]

Wow. I have the same issue. How to attach the body with proper movement...? I think there is the way to solve it with BP... I checked the Unreal forum but I can't find the proper way.

[@SkyHandOneTen]

Can you do a video of how to warp a metahuman to look like a custom character? RS3D Zwrap is a good wrapper. I have this model of NAS I want to put to the test, as well as Michael Jordan and the Rock.

[@Jsfilmz]

just mesh to metahuman it mang save u the headache hee hee

[@SkyHandOneTen]

@@Jsfilmz Hmmm, I guess I'm a bit behind. Not sure what that is or where it is. I'll look it up. Thanks

[@Jsfilmz]

​@@SkyHandOneTenoh yea man mesh to metahuman jsfilmz look it up its easy

[@SkyHandOneTen]

Seems that info is everywhere I was just looking up the wrong terminology all this time. Thanks.

[@sahinerdem5496]

can i use android video record of my face?

[@naytbreeze]

Having a huge problem when I try to add the animation to my metahuman in the sequencer it becomes detached from the body from around the shoulders area. Was it because I may have moved too much in the capture? Can’t seem to get the body and the head attached

[@UnrealEnginecode]

after ubdate iphone app It gives a warning for the animation part of the application live link face app saying your device model is unsupported you may continue but your results could be affcted"for meta human animator capture ." and its work but i wonder My economic situation is not good and I will develop games. How much performance difference does the iPhone 11 make?

[@andrewwelch5017]

The software requires an iPhone with a “True Depth” (LiDAR) sensor because it needs depth data to accurately track your face. You can always borrow an iPhone 11 (or newer) to do the tracking and then transfer the file to your computer.

[@UnrealEnginecode]

thank you for your answer, what are they doing this warning for, do you think there will be a noticeable quality difference?

[@andrewwelch5017]

@@UnrealEnginecode The quality I got was excellent so I'm not worried about it.

[@burov.a.4467]

Hi! Thanks for the cool video. Listen to the question, will it work if you upload a regular video shot on a camera?) just no iphone))

[@Jsfilmz]

dont think so iphones are precalibrated like i showed in the performance editor

[@andrewwelch5017]

No, the software requires depth information which a regular camera can’t provide. Borrow an iPhone 11 or newer.