MetaHuman - Real-Time Facial Model Animation Demo | State of Unreal 2023

[@Tony-qe3fl]

I’m no game developer, but the fact that it took less than 5 minutes to do this is mind blowing!

[@ForDadsTV]

But was this really captured on iPhone ?

[@coresnap]

@@ForDadsTV yeah. That's not what they would use in professional settings, but with lighting adapters and the iphone's 3d scan ability, you can do amazing things with it. Like, your iphone can literally scan your whole house and render it out in real time. It's fkn crazy!

[@FantasyNero]

Don't judge yet, we need to see this real-time when released, so everyone will make a video about it if it's true or lie.

[@ABrokenTV]

@@FantasyNero It's not a matter of if the phone is capable, as doing research shows that it can. It's a matter of whether or not Epic can make the software to do it.

[@Rompodable]

Yup UE5 is quite à technological milestone for the industry. A lot is going to change in the background. People might not see it through games and shows/movies but it will be used in a lot of production pipeline now.

[@jesterspawn]

It's hard to overstate just how incredible it is to get to 85% finished facial animation (especially since it is now easily editable with animatable controls) integrated into the pipeline in around 3 minutes of real time using a freely-available engine and only a phone camera for the capture. They're playing it cool, but this is a serious flex.

[@ThatGuyMata]

3 min for 85% of a 9 second shot.

[@owenb6499]

honestly id say its above 85 the movements almost perfect, but it’s rendering in real time. absolutely insane.

[@yeetbro3659]

@@ThatGuyMata Its free

[@Midnight_Gust]

@@ThatGuyMata yea its great

[@josephmarch7142]

That cleanup though!

[@raihidara]

It's not perfect but this is incredible for how much time this saves and how little needs to be adjusted

[@dougr8646]

What nickpick do you have clown

[@korinogaro]

True, but imagine that you can have high quality close-up while you talk with literally every NPC in some RPG... Insanity.

[@Ricolaaaaaaaaaaaaaaaaa]

No worries AI will fix it by next week.

[@BladeValant546]

Well, they did do this with a cell phone.

[@神林しマイケル]

@@korinogaro the only thing really missing for NPCs are human like behaviour. Imagine if NPC act like humans? Hopefully as advancement in AI, we would see it applied in game where there are a billion parameters the NPC can act like at all times.

[@maspleben]

That deserved a standing ovation. This is so impressive to be able to render this level of quality animation in only 5 minutes.

[@jmasked5082]

Everyone is overwhelmed with all the innovations happening all the time. We can't keep acting surprised and blown away every 5 minutes.

[@Zicolie]

@@jmasked5082 bro is not excited 💀

[@jogoegel1279]

​@@jmasked5082 because it doesn't look like your as passionate about it as we are

[@xaby996]

Metahumans are not game-ready. And they will not be game ready for a few years at least. They TANK the performance of your project. And that is before optimizing with LOD, adding facial animations or adding hair physics. Yeah its cool technology. But Unreal Engine 4.26 without Metahumans is the meta for indie developers. Haha did you see the pun there.

[@leigel3]

And using a phone as the capture device.

[@AsuraDandy]

This is quite amazing. You know what they need to do? Go to visually pleasing places and take a stroll and record it, take a recording of people talking, record a couple. Then recreate the scenery and model and animate them in a similar fashion. Then put them in any order and see how many people can spot which one is the real and which one is the Unreal.

[@remyd8767]

That idea is unreal

[@guillermogutierrez710]

TM

[@Arrradiance]

you're hired!

[@kneelbeforezak]

Really excited to see what independent filmmakers do with this tech

[@jiwik731]

One man movie... you can create a whole movie alone. Or from clips on the internet. :D

[@badpuppy3]

You won't watch it, even when they do. There's so much content out there now, no one has the time.

[@grav4261]

Zumeckis enters the chat

[@_Cetarial]

They don’t need to hire actors.

[@him2674]

@@badpuppy3 you're boring

[@darron13]

This is gonna be huuuuuge for low budget projects, amazing

[@rokpepeshogun]

which will never come out in the frst place

[@TheOneLilo]

​@@rokpepeshogun Spiderman lotus reference

[@whoareyou3781]

So now we will have better corn animation?

[@raduromanesti6408]

For everyone really

[@silvernoob1603]

neil breen might be able to make even more amazing movies

[@muhammadnabil3390]

Eerily scary to see that from just a short period of time,from 1990s' onwards,technology has progressed immensely.

[@Ep0xy]

Right? 50 years ago was when Pong arrived.

[@jojothepro15]

Why is everything going so fast

[@elegantfeet6733]

@@jojothepro15 Technology will keep expanding exponentially. Especially now A.I is becomming to become a thing. It will actually go faster.

[@ToneyCrimson]

@@jojothepro15 Gotta go fast!

[@leoleo1035]

@@jojothepro15 That's how technology works. But still... not fast enough.

[@AutTheWizard]

This brings AAA animation and graphics to the hands of indie game developers. The future is looking so exciting!

[@jhinabloomingflower807]

Or boring and repetitive

[@fluge_]

@@jhinabloomingflower807 will see tho

[@gregory7406]

Its looks dystopian and evil tbh

[@MuromachiLines]

You miss the point. This technology is boring and old without AI-NPCs

[@DDumbrille]

Wait till AI takes away your writing job too...then AI will do all the 'developing'. No humans required.

[@HelplessTeno]

It's absolutely insane how quick that process is. With maybe 10-20 more minutes you could pretty much completely clean up the animations and smooth everything out.

[@Grievousss]

Holy crap! From months to minutes… Games in the coming years are going to be absolutely insane

[@Rapunzel879]

Let's hope that games will come out faster because nowadays it takes them forever.

[@willuigi64]

@@Rapunzel879 that's the aim of many of the improvements made to unreal. hoping for the best but even with all these advancements, I'm unsure games will come out much quicker.

[@TheBearWithOrangeFur]

@@Rapunzel879I’d rather have them take forever and make a masterpiece, than have it sooner and be terrible.

[@moxifloxacinsalsa]

graphics aren't what makes the games fun, as we have witnessed in the last... 7-10 years

[@rionix88]

Lets hope the file size is smaller.. current game at least 20gb

[@larsmuldjord9907]

I've been impressed with motion capture and performance capture for many years. But it's the first time I've seen it done in a way that doesn't require a separate room with trackers, many cameras and an entire team to make it go prrrrr. This is beyond impressive and should help even the tiniest teams to get some great performances in their games. Incredible!

[@dafonsecaalexis5355]

That’s MIND BLOWING… even the smalles of studios have facial motion capture within reach now… what a time to be alive… Unreal is just giving the best tools to developpers right now.

[@flipnap2112]

2 minute papers ha ha!!

[@farithadnan]

Wow, 2023 is off to a great start with all the incredible advancements in tech! From AI to Unreal Engine 5.2, this year is already shaping up to be a wild ride.

[@civil_fellow2985]

4 minutes for that level of quality animation is absolutely insane. Can't wait for this

[@KryptsCorner]

The future is bright for Game development

[@Irrelavant__]

@@Siranoxz Unity just added chatGPT into the engine

[@ibringthelastwords1358]

Scary 😱

[@twentytwo138]

Sorry to be pessimistic, but we've been saying that for years and the quality of games has decreased. Everyone was saying ''future looks bright for gaming'' but then we just got disappointments.. The only game that truly reached the expectations from 10-15 years ago is RDR2. Most gamers expected that every game will have the same quality as RDR2 by now, but that's not the case. Even the most advanced technology is useless if the companies are greedy and the developers are lazy and have no passion..

[@mclarenrob2]

@@Irrelavant__ is that just for NPC dialog creation? Im most excited for when AI can actually create game worlds itself

[@Irrelavant__]

@@mclarenrob2 thats Ubisoft. The unity one is implemented in the engine. so if you write create 100 cubes it will do that or any other task that is not too complicated. Yet

[@TrueDiox]

Happy to see Melina is basically the face of a bright-looking future for performance capture.

[@frijolitodr]

Following their whole process from making Hellblade, she became my favorite person with no prior cinematic experience be able to elevate the character they way she did it is exceptional.

[@darcksonrabelo1866]

She's gorgeous

[@stormbreaker82]

​@@darcksonrabelo1866 no cap

[@yesyouareright9800]

who is melina

[@AynenMakino]

It's still in the uncanny valley, but just the amount of workload that's being cut out is amazing!

[@Kyuubi840]

True. Climbing that valley!

[@iSOBigD]

For sure, but this can be hand tweaked to fix a lot of that as far as animation. However, think back just a few years to high budget CG movies like Beowulf, Final Fantasy Spirits Within or even Avatar and how close this gets to that in real time without armies of artists and with almost no budget. It's a huge leap.

[@offlinegamer6756]

the eyes still looks dead !

[@bulletsmichael]

​@@offlinegamer6756 cry to mommy

[@libertas5005]

@@bulletsmichael Why are you butthurt?

[@crablord7934]

This is a crazy game changer for cutscene animation. Traditional motion tracking is still using reference to start from scratch, while this goes through 80 percent of the process and leaves only a bit of rendering left for the animators. Crazy.

[@hermano8160]

The mouth looks off but with a bit time for adjusting it should be convincing. Pretty incredible for small studios who want to tell character-driven stories.

[@eastons_1015]

Insane how far our technology has come in the last 50 years. Even in an early state this looks phenomenal!

[@M.Evra91]

Imagine what the next 50 years will be like.

[@arkhe1n107]

50? Try 10.

[@SassInYourClass]

50 years? Try 15 days.

[@sandprince4165]

​@@SassInYourClass 15 days? Try 5

[@valoriant]

@@sandprince4165 5 days? Try 5 minutes

[@regiluthfi]

metahuman looks more promising than metaverse

[@AlmiranteLobo]

Different technologys.

[@raeji]

@@AlmiranteLobo yeah no s**t sherlock ☠

[@user-su5jg7cg9x]

You do know that unreal engine 5 will be used for the metaverse and there is a tech called metahero that scans items.

[@Qual_]

metahuman looked like Mark Zukerberg tho' :v

[@rasmuspedersen3563]

WTF is "metaverse"??

[@kylerobinson6102]

Amazing work guys!

[@colonizingpuertoricoisunde2971]

-Phillipa Soo “Exposing yourself to many kinds of art can only lead to amazing things. It helps you learn about your own art, your own taste, what kind of art you want to create for yourself.”

[@funkyswamprat]

This could be useful for artists too, you could take a picture of an expression and showcase it from different angles and use it as a reference

[@taja1976]

Uncanny-valley still evident but things keep on improving and it is highly impressive. There are nuances, particularly in the stop/start movements that look a little bit like the old days of stop-motion animations.

[@duck_time]

honestly, the slight smile seriously impressed me. That part looked the most human to me

[@mahirmuhammad2325]

Well to be honest She herself was uncanny in the capture athe very first place

[@mrdngm]

This is a game changer for creatives everywhere. This technology is going to unlock new levels of innovation for artists! Wow!

[@YuniorGamboa]

This saves a lot of production time and is efficient. The devs just need to focus on the finishing touches

[@LHK-art]

Or focus on looking for new jobs or start their own projects

[@K0vasz]

Finally. It's just crazy how game developing time is 2...7 years nowadays

[@LimitedWard]

IDK why people are saying the end result looks "rough". The result was on par with my many AAA titles today, and yet it was done in just a couple minutes using an iPhone. To get comparative performance with tradtional methods would take way more time, money, and skill.

[@SantiagoVelezRestrepo]

Since the last December, everything in IA is going ultra mega fast

[@Leo-yn5fx]

we are near the peak of graphics. What more could you possibly want more than this.

[@fernandogoerl5476]

This is insane!!!!!

[@ChengTeoh]

David Cage : "Emotion is something complex and challenging to capture" MetaHuman Team : "All we need is a phone a PC and a couple of minutes"

[@litjellyfish]

And a cup of coffee (not included for free in Unreal) Engine

[@MildMisanthropeMaybeMassive]

A lot of complex things are being calculated by the phone and PC.

[@LightsJusticeZ]

It is crazy how far we keep advancing in mo-cap! Truly impressive!

[@Homemade-Blurb]

Thanks to Jordan Mechner who started the motion capture way back in 1989, in a game "Prince Of Persia"

[@icantchosemyname]

Great work. It gets me woried about actor and actresses's future. i'm waiting for the combination of Metahuman + chatgpt + whisper + text to speech.

[@xs1egex999]

Imagine one day we have the opportunity to "upload" our face to a character customization process, rather than trying to re-create ourselves... 🤯

[@patrickkiwan741]

Combine that with the programs that can recreate your voice from just a couple of words and people could be playing as themselves.

[@swishahoops2103]

You can already do it on sports games now at a lower level. Games like NBA2K allow you to scan your face with your phone and upload it into the game. Not nearly as detailed as this, but the idea is def out there.

[@DazHotep6EQUJ5]

The whole point of gaming is being someone else no?

[@geelemo]

So we can have better avatars for social media than the ones on our phones

[@SuperVegotta]

​@@DazHotep6EQUJ5No

[@FR-oz9px]

Happy to see Melina again.

[@thekingchem8800]

If this tech was used in an online VR setting with more processing power you could have real time facial capture animated on your avatar. Getting closer to ready player one every year.

[@joelface]

This was my thought as well. You've got to think that's next.

[@lgolem09l]

That would be a online game with quiet some minimum requirements, just for one feature. Unlikely financially feasible in the coming decade, because those games want a huge userbase, i.e. let it run on a potato

[@jase276]

Sounds like you'd need a NASA computer for that

[@technus147]

​@@lgolem09l you could just turn it on or off

[@aidenarkham]

Whoa that's crazy. No dots on the face with just a phone. Insane.

[@l-y-d-s]

Now that we have soundbites from people with the exact facial movements when they said them, it seems like in the future if we have vocal chord mapping, we should be able to also take quotes from people and change the voice to any person we want the same way they effortlessly changed the model as shown at the end of the video.

[@misterteee9471]

Its rough for like a finished game but big budget games that will be using this stuff already has people that tweak little facial movements after they were recorded so this would DRASTICALLY shorten the process its actually insane Insane how every year some tech company unveils their new stuff and each time I'm blown away

[@alexdoan273]

have you seen Ubisoft games? This is in no way rough! This level of facial performance is already better than many AAA games' already

[@averydee5328]

@@alexdoan273 It's not perfect, but they didn't really take their time to make any adjustments, they did it in under 3 minutes, imagine if they took 10 minutes to tweak it

[@misterteee9471]

@@alexdoan273 Keyword "finished" no Ubisoft game is finished lmao

[@jagotiberan2181]

@@misterteee9471 LOL truth about Ubisoft 😂

[@brandonh2690]

Not sure if you watched to the end but after the raw capture they cut to a polished clip. The capture isn't lacking its the environment/lighting/Vfx work that puts it over the top.

[@agentys3544]

What a time to be alive guys! Now imagine old games like PoP with this features..

[@pallavrajpatra7227]

Mentions PoP, deserves a like. My man.

[@Nom1fan]

Prince of Persia?

[@12stepsbeyondtheeventhorizon]

This is insane... it's absolutely crazy to think how much this type of technology has developed. What a great time to become a game dev.

[@bella_ciao_8]

Awesome! So glad she ended up playing Senua.

[@dito7347]

We’re getting closer to the simulation

[@jerbsherb4391]

Imagine video game performers doing their jobs without a mocap

[@HafidzMurshidie]

This is still MoCap..

[@trmbne]

You mean without the mocap suit

[@ryndaman619]

@@trmbne yup its using Iphone to detect our facial structure, so webcam or android phone cant be use since the technology is better on Iphone

[@subsume7904]

You know what Motion Capture is.. right?..

[@brolys992]

01:23 Angry while smile 😁 ❤

[@dreamzdziner8484]

That rig generation was super fast and the quality looks pretty amazing. Metahuman is truly a game changer the moment it arrived. Always happy to see it progressing beyond limits.

[@Mychannelbandung]

The future of gaming is really bright

[@rpgamer2944]

for a process that would take weeks to get to this stage, forming almost instantly, it's impressive.

[@maizeman90]

This is incredible tech that is only going to improve over time. Indie devs and filmakers are going to benefit tremendously from this

[@roger4845]

Competition will skyrocket.

[@KloneStudios]

This is absolutely AMAZING. Gawdamn!!!!!!!

[@oldmatttv]

This is pretty incredible for devs and creators, small especially in terms of accessibility etc, but large too for time and budget and such. Really looking forward to this tech being used more widely.

[@karmah88]

everything ive been learning about animation and modelling over the last 5 years? defeated lol. the rate of development at the moment is really incredible.

[@AyeJVinci]

This looks so cool. If I’m not mistaken the little kid model that they showed on the screen looks like it can be a character in the movie Monster House

[@tochukwuudu7763]

Epics killing it.

[@EdsonJuniordj]

Minha cabeça explodiu com isso, meses de trabalho em minutos! Eu aplaudi junto com o público.

[@Eizenz]

This has terrifying uses. Awesome, but also oh no what have we done.

[@dturkiali123]

it seems the cooperation with alliance is done perfectly

[@JESSROCKEDChannel]

This would forever keep people alive no matter what age

[@LouSipher]

Politics getting wild in 10 years

[@devinsidell]

@@LouSipher It's wild enough now ffs

[@LewisNakao]

Nice. Pet Cemetery not required.

[@yahiiia9269]

@@devinsidell It was wilder before. Someone legit made a horse a politician in the past.

[@cartoonman884]

Deepfake grandma doing her own funeral service.

[@kiix]

Wow. Extremely impressive.

[@davidzimmerly4973]

The automatic rotoscoping alone deserves recognition.

[@ozarcnetwork]

there is no rotoscoping.... the tracking data its being placed on a 3d character with a 3d background.

[@keshi5541]

Its not rotoscoping. That is different. Rotoscoping is 2d animation technique and this is a 3d model.

[@ttydusfilho]

O que ainda não conseguiram chegar a "perfeição"...é de fato o movimento dos olhos!

[@JaroAtry]

It's funny that they chose an actress who herself looks like motion capture when she moves :D

[@vincentkentermusic]

At some point they will release a video like this and reveal the presenters were actually the MetaHumans.

[@darrinstokke3839]

Looks like a big step towards getting past the Uncanny Valley problem. Though the different faces at the end were weird. That is to be expected from different facial structure.

[@purplcyanide]

So, it's not perfect, but the fact that this could be produced so quickly is mind-blowing, especially considering this was captured on a phone.

[@universalmonkey7059]

This will change the digital industry forever.

[@equalgeek]

This is absolutely insane!!!! Congrats to all the devs that developed this tech as you are the real hero's here... seriously!!!!! Thank You!!!!

[@Coindrop-Tim]

This is seriously impressive tech.

[@ayayron8221]

they did all this in less than 5 minutes on a phone. Imagine what game studios can do with even better equipment and time to properly fine tune the facial expressions.

[@cptLeviAckerman]

WOW!!! Just think what this will be after 10 years from now?

[@fawwazthalibthalib7180]

Imagine mortal kombat with this face tech.

[@joelface]

Real-time virtual reality avatars that perfectly mimic everything we do.

[@flaisee]

All the motion capture things just in 5 mins. That's really saved a big time.

[@savagekid94]

Feels like we're in the future. Gaming went from lovable pixel art to "life could be a simulation" in the span of

[@TheLYagAmi]

Damn! Still needs a bit of tweaking but this definitely pushes models over the uncanny valley. Ps: all animator’s who painstakingly hand key facial animation are probably mad af right now that this wasn’t available earlier lol

[@Isa13]

amazing technology

[@Fantomfields]

There literally cutting days, possibly MONTHS of work out of the process with this and the other procedural stuff they're doing! This is insane!

[@5amuk]

Not insane, UNREAL.

[@ramabg2]

When every game is AAA game there is no AAA game. It Is great for Unreal to level the playing field for indie studio. Going forward, studio will focus more to artistic than tech.

[@Thomas890s]

It is really fast to finish the capture…Awesome!

[@kbrodeur]

it's not there yet, but it sure is perfect for a quick entry, it's like 80% there. but at low budgets it's 100% there.

[@tticusFinch]

This tech makes independent and amateur filmmaking so much more accessible and high-quality

[@swizard0922]

Love it!!!!! Astounding!!!

[@AngryBulldogGaming]

that angry grunt was wholesome

[@Lord_of_Dread]

*sexy

[@ferreroman2913]

the last few seconds of the capture looked the best of it all

[@tunelyricgameon]

This is easily the biggest benefit for me as an independent filmmaker in Unreal Engine

[@highmm2696]

Can't wait for the Dark Pictures series or another bigger game from Super Massive Games to use this tech.

[@SmoothKenny]

I'm genuinely impressed. Bravo 👏

[@mousmnwltr]

Wow!! This is insane

[@MrHappymanx]

This is incredible technology. Wow

[@VPopkins]

why is it you can almost immediately see computer generated mimics at 3:40 even though the render itself is close to perfect? can notice but can not explain what is missing

[@skexone]

Congrats to melina on being the face of unreal's metahuman

[@alphach1mp]

The amount of developers that will be layed out because of this technology will be immense!!!

[@wiseman4953]

That is likely to happen some places. But, if Unreal is mostly unlocked to low budget teams, this could lead to more small team devs and more games released. In essence spreading the wealth versus a few companies monopolizing everything.

[@YaretziaGarcia]

That’s pretty impressive

[@fernandofaria2872]

Melina is so beautiful :)

[@Enoc63]

Gonna make my first AAA on lunch break now :)

[@Dzeividz]

Wow this will make game development much faster

[@cdonnelly5566]

What would really blow my mind would be if right at the end, it was revealed that the people on stage were actually just a sim running in MetaHuman

[@neilkmps]

Too soon. Maybe in 5 to 10 years

[@HafidzMurshidie]

We still don’t have the technology for IRL 3D hologram… All we have right now is a 2D projection on a glass..

[@phoenix9531]

Has nothing to do with this. Thats like saing it would have been impressive if the technician did a backflip or the stage theyre on build itself like a transformer...

[@mukhlisz]

keep the magic flowing guys! to infinity and beyond!

[@Kryujiin]

Baffled me wordless. The time savings let alone the worktime-flow! Impressive! Thats gamechanging

[@kaminobatto]

Picture this, combined with an AI like ChatGPT and you got yourself a... I don't know what... This is getting better by the day, and I'm excited to see where it goes in a few years! Absolutely amazing!

[@getyournekoshere9824]

until you ask chatgpt to do anything above a 5th grade level and it completely breaks down

[@vladimirvladimirovich-l1y]

Surprised no one mentioned Melina's acting at 1:18. She had to perform multiple emotions very quickly, yet it looked so believable. Especially when she's acting scared. Shows what a great actress she really is.