MetaHuman - Real-Time Facial Model Animation Demo | State of Unreal 2023

Unreal Engine 5's MetaHuman is getting a new update called Animator to create new fully animated facial models in real time with just a phone camera. The team from Ninja Theory, including Senua actor Melina Juergens shows this process in real time.
#IGN #Gaming #UnrealEngine5

[@Tony-qe3fl]

I’m no game developer, but the fact that it took less than 5 minutes to do this is mind blowing!

[@Bananaz9xx]

But was this really captured on iPhone ?

[@coresnap]

@@Bananaz9xx yeah. That's not what they would use in professional settings, but with lighting adapters and the iphone's 3d scan ability, you can do amazing things with it. Like, your iphone can literally scan your whole house and render it out in real time. It's fkn crazy!

[@FantasyNero]

Don't judge yet, we need to see this real-time when released, so everyone will make a video about it if it's true or lie.

[@ABrokenTV]

@@FantasyNero It's not a matter of if the phone is capable, as doing research shows that it can. It's a matter of whether or not Epic can make the software to do it.

[@Rompodable]

Yup UE5 is quite à technological milestone for the industry. A lot is going to change in the background. People might not see it through games and shows/movies but it will be used in a lot of production pipeline now.

[@jesterspawn]

It's hard to overstate just how incredible it is to get to 85% finished facial animation (especially since it is now easily editable with animatable controls) integrated into the pipeline in around 3 minutes of real time using a freely-available engine and only a phone camera for the capture. They're playing it cool, but this is a serious flex.

[@ThatGuyMata]

3 min for 85% of a 9 second shot.

[@owenb6499]

honestly id say its above 85 the movements almost perfect, but it’s rendering in real time. absolutely insane.

[@yeetbro3659]

@@ThatGuyMata Its free

[@Midnight_Gust]

@@ThatGuyMata yea its great

[@josephmarch7142]

That cleanup though!

[@maspleben]

That deserved a standing ovation. This is so impressive to be able to render this level of quality animation in only 5 minutes.

[@jmasked5082]

Everyone is overwhelmed with all the innovations happening all the time. We can't keep acting surprised and blown away every 5 minutes.

[@Zicolie]

@@jmasked5082 bro is not excited 💀

[@jogoegel1279]

​@@jmasked5082 because it doesn't look like your as passionate about it as we are

[@xaby996]

Metahumans are not game-ready. And they will not be game ready for a few years at least. They TANK the performance of your project. And that is before optimizing with LOD, adding facial animations or adding hair physics. Yeah its cool technology. But Unreal Engine 4.26 without Metahumans is the meta for indie developers. Haha did you see the pun there.

[@leigel3]

And using a phone as the capture device.

[@AsuraDandy]

This is quite amazing. You know what they need to do? Go to visually pleasing places and take a stroll and record it, take a recording of people talking, record a couple. Then recreate the scenery and model and animate them in a similar fashion. Then put them in any order and see how many people can spot which one is the real and which one is the Unreal.

[@remyd8767]

That idea is unreal

[@guillermogutierrez710]

TM

[@Arrradiance]

you're hired!

[@raihidara]

It's not perfect but this is incredible for how much time this saves and how little needs to be adjusted

[@dougr8646]

What nickpick do you have clown

[@korinogaro]

True, but imagine that you can have high quality close-up while you talk with literally every NPC in some RPG... Insanity.

[@Ricolaaaaaaaaaaaaaaaaa]

No worries AI will fix it by next week.

[@BladeValant546]

Well, they did do this with a cell phone.

[@user-mp3eh1vb9w]

@@korinogaro the only thing really missing for NPCs are human like behaviour. Imagine if NPC act like humans? Hopefully as advancement in AI, we would see it applied in game where there are a billion parameters the NPC can act like at all times.

[@darron13]

This is gonna be huuuuuge for low budget projects, amazing

[@rokpepeshogun]

which will never come out in the frst place

[@TheOneLilo]

​@@rokpepeshogun Spiderman lotus reference

[@whoareyou3781]

So now we will have better corn animation?

[@raduromanesti6408]

For everyone really

[@silvernoob1603]

neil breen might be able to make even more amazing movies

[@AutTheWizard]

This brings AAA animation and graphics to the hands of indie game developers. The future is looking so exciting!

[@jhinabloomingflower807]

Or boring and repetitive

[@fluge_]

@@jhinabloomingflower807 will see tho

[@gregory7406]

Its looks dystopian and evil tbh

[@MuromachiLines]

You miss the point. This technology is boring and old without AI-NPCs

[@DDumbrille]

Wait till AI takes away your writing job too...then AI will do all the 'developing'. No humans required.

[@HelplessTeno]

It's absolutely insane how quick that process is. With maybe 10-20 more minutes you could pretty much completely clean up the animations and smooth everything out.

[@kneelbeforezak]

Really excited to see what independent filmmakers do with this tech

[@jiwik731]

One man movie... you can create a whole movie alone. Or from clips on the internet. :D

[@badpuppy3]

You won't watch it, even when they do. There's so much content out there now, no one has the time.

[@grav4261]

Zumeckis enters the chat

[@_Cetarial]

They don’t need to hire actors.

[@him2674]

@@badpuppy3 you're boring

[@muhammadnabil3390]

Eerily scary to see that from just a short period of time,from 1990s' onwards,technology has progressed immensely.

[@Ep0xy]

Right? 50 years ago was when Pong arrived.

[@jojothepro15]

Why is everything going so fast

[@elegantfeet6733]

@@jojothepro15 Technology will keep expanding exponentially. Especially now A.I is becomming to become a thing. It will actually go faster.

[@ToneyCrimson]

@@jojothepro15 Gotta go fast!

[@leoleo1035]

@@jojothepro15 That's how technology works. But still... not fast enough.

[@civil_fellow2985]

4 minutes for that level of quality animation is absolutely insane. Can't wait for this

[@farithadnan]

Wow, 2023 is off to a great start with all the incredible advancements in tech! From AI to Unreal Engine 5.2, this year is already shaping up to be a wild ride.

[@Grievousss]

Holy crap! From months to minutes… Games in the coming years are going to be absolutely insane

[@Rapunzel879]

Let's hope that games will come out faster because nowadays it takes them forever.

[@willuigi64]

@@Rapunzel879 that's the aim of many of the improvements made to unreal. hoping for the best but even with all these advancements, I'm unsure games will come out much quicker.

[@TheBearWithOrangeFur]

@@Rapunzel879I’d rather have them take forever and make a masterpiece, than have it sooner and be terrible.

[@moxifloxacinsalsa]

graphics aren't what makes the games fun, as we have witnessed in the last... 7-10 years

[@rionix88]

Lets hope the file size is smaller.. current game at least 20gb

[@TrueDiox]

Happy to see Melina is basically the face of a bright-looking future for performance capture.

[@frijolitodr]

Following their whole process from making Hellblade, she became my favorite person with no prior cinematic experience be able to elevate the character they way she did it is exceptional.

[@darcksonrabelo1866]

She's gorgeous

[@stormbreaker82]

​@@darcksonrabelo1866 no cap

[@yesyouareright9800]

who is melina

[@crablord7934]

This is a crazy game changer for cutscene animation. Traditional motion tracking is still using reference to start from scratch, while this goes through 80 percent of the process and leaves only a bit of rendering left for the animators. Crazy.

[@Leo-yn5fx]

we are near the peak of graphics. What more could you possibly want more than this.

[@dafonsecaalexis5355]

That’s MIND BLOWING… even the smalles of studios have facial motion capture within reach now… what a time to be alive… Unreal is just giving the best tools to developpers right now.

[@flipnap2112]

2 minute papers ha ha!!

[@larsmuldjord9907]

I've been impressed with motion capture and performance capture for many years. But it's the first time I've seen it done in a way that doesn't require a separate room with trackers, many cameras and an entire team to make it go prrrrr. This is beyond impressive and should help even the tiniest teams to get some great performances in their games. Incredible!

[@mybrotherinchris]

This could be useful for artists too, you could take a picture of an expression and showcase it from different angles and use it as a reference

[@l-y-d-s]

Now that we have soundbites from people with the exact facial movements when they said them, it seems like in the future if we have vocal chord mapping, we should be able to also take quotes from people and change the voice to any person we want the same way they effortlessly changed the model as shown at the end of the video.

[@AynenMakino]

It's still in the uncanny valley, but just the amount of workload that's being cut out is amazing!

[@Kyuubi840]

True. Climbing that valley!

[@iSOBigD]

For sure, but this can be hand tweaked to fix a lot of that as far as animation. However, think back just a few years to high budget CG movies like Beowulf, Final Fantasy Spirits Within or even Avatar and how close this gets to that in real time without armies of artists and with almost no budget. It's a huge leap.

[@offlinegamer6756]

the eyes still looks dead !

[@bulletsmichael]

​@@offlinegamer6756 cry to mommy

[@libertas5005]

@@bulletsmichael Why are you butthurt?

[@KryptsCorner]

The future is bright for Game development

[@Siranoxz]

Just wait until they implement AI into game engines.

[@Irrelavant__]

@@Siranoxz Unity just added chatGPT into the engine

[@Siranoxz]

@@Irrelavant__ There ya go, it already begun..

[@ibringthelastwords1358]

Scary 😱

[@twentytwo138]

Sorry to be pessimistic, but we've been saying that for years and the quality of games has decreased. Everyone was saying ''future looks bright for gaming'' but then we just got disappointments.. The only game that truly reached the expectations from 10-15 years ago is RDR2. Most gamers expected that every game will have the same quality as RDR2 by now, but that's not the case. Even the most advanced technology is useless if the companies are greedy and the developers are lazy and have no passion..

[@taja1976]

Uncanny-valley still evident but things keep on improving and it is highly impressive. There are nuances, particularly in the stop/start movements that look a little bit like the old days of stop-motion animations.

[@duck_time]

honestly, the slight smile seriously impressed me. That part looked the most human to me

[@mahirmuhammad2325]

Well to be honest She herself was uncanny in the capture athe very first place

[@rpgamer2944]

for a process that would take weeks to get to this stage, forming almost instantly, it's impressive.

[@ChengTeoh]

David Cage : "Emotion is something complex and challenging to capture" MetaHuman Team : "All we need is a phone a PC and a couple of minutes"

[@litjellyfish]

And a cup of coffee (not included for free in Unreal) Engine

[@MildMisanthropeMaybeMassive]

A lot of complex things are being calculated by the phone and PC.

[@regiluthfi]

metahuman looks more promising than metaverse

[@AlmiranteLobo]

Different technologys.

[@raeji]

@@AlmiranteLobo yeah no s**t sherlock ☠

[@user-su5jg7cg9x]

You do know that unreal engine 5 will be used for the metaverse and there is a tech called metahero that scans items.

[@Qual_]

metahuman looked like Mark Zukerberg tho' :v

[@rasmuspedersen3563]

WTF is "metaverse"??

[@ferreroman2913]

the last few seconds of the capture looked the best of it all

[@DrBrunoDzogovic]

Imagine for a moment you capture the facial expressions in 8K, 240fps... The level of detail in the animation though... Mind-blowing is an understatement!

[@eastons_1015]

Insane how far our technology has come in the last 50 years. Even in an early state this looks phenomenal!

[@M.Evra91]

Imagine what the next 50 years will be like.

[@arkhe1n107]

50? Try 10.

[@SassInYourClass]

50 years? Try 15 days.

[@sandprince4165]

​@@SassInYourClass 15 days? Try 5

[@valoriant]

@@sandprince4165 5 days? Try 5 minutes

[@xs1egex999]

Imagine one day we have the opportunity to "upload" our face to a character customization process, rather than trying to re-create ourselves... 🤯

[@patrickkiwan741]

Combine that with the programs that can recreate your voice from just a couple of words and people could be playing as themselves.

[@swishahoops2103]

You can already do it on sports games now at a lower level. Games like NBA2K allow you to scan your face with your phone and upload it into the game. Not nearly as detailed as this, but the idea is def out there.

[@dazshcan1224]

The whole point of gaming is being someone else no?

[@geelemo]

So we can have better avatars for social media than the ones on our phones

[@SuperVegotta]

​@@dazshcan1224No

[@Thomas890s]

It is really fast to finish the capture…Awesome!

[@Homemade-Blurb]

Thanks to Jordan Mechner who started the motion capture way back in 1989, in a game "Prince Of Persia"

[@YuniorGamboa]

This saves a lot of production time and is efficient. The devs just need to focus on the finishing touches

[@fkdump]

Or focus on looking for new jobs or start their own projects

[@K0vasz]

Finally. It's just crazy how game developing time is 2...7 years nowadays

[@kylerobinson6102]

Amazing work guys!

[@colonizingpuertoricoisunde2971]

-Phillipa Soo “Exposing yourself to many kinds of art can only lead to amazing things. It helps you learn about your own art, your own taste, what kind of art you want to create for yourself.”

[@liquidpza]

Now imagine this with a layer of ai diffusion on top while communicating through GPT-4. The near future is scary fascinating.

[@icantchosemyname]

Great work. It gets me woried about actor and actresses's future. i'm waiting for the combination of Metahuman + chatgpt + whisper + text to speech.

[@LimitedWard]

IDK why people are saying the end result looks "rough". The result was on par with my many AAA titles today, and yet it was done in just a couple minutes using an iPhone. To get comparative performance with tradtional methods would take way more time, money, and skill.

[@LightsJusticeZ]

It is crazy how far we keep advancing in mo-cap! Truly impressive!

[@ttydusfilho]

O que ainda não conseguiram chegar a "perfeição"...é de fato o movimento dos olhos!

[@Flopsaurus]

This is absolutely astonishing and is a massive breakthrough for animation technology.

[@fernandogoerl5476]

This is insane!!!!!

[@mrdngm]

This is a game changer for creatives everywhere. This technology is going to unlock new levels of innovation for artists! Wow!

[@aidenarkham]

Whoa that's crazy. No dots on the face with just a phone. Insane.

[@Cemovaran]

Baffled me wordless. The time savings let alone the worktime-flow! Impressive! Thats gamechanging

[@thekingchem8800]

If this tech was used in an online VR setting with more processing power you could have real time facial capture animated on your avatar. Getting closer to ready player one every year.

[@joelface]

This was my thought as well. You've got to think that's next.

[@lgolem09l]

That would be a online game with quiet some minimum requirements, just for one feature. Unlikely financially feasible in the coming decade, because those games want a huge userbase, i.e. let it run on a potato

[@jase276]

Sounds like you'd need a NASA computer for that

[@technus147]

​@@lgolem09l you could just turn it on or off

[@12stepsbeyondtheeventhorizon]

This is insane... it's absolutely crazy to think how much this type of technology has developed. What a great time to become a game dev.

[@recie29]

mind blown! from months to minutes! technology sure is fascinating!

[@SharifSourour]

It looks mind blowing on the stylized characters! Game changing.

[@FR-oz9px]

Happy to see Melina again.

[@dreamzdziner8484]

That rig generation was super fast and the quality looks pretty amazing. Metahuman is truly a game changer the moment it arrived. Always happy to see it progressing beyond limits.

[@savagekid94]

Feels like we're in the future. Gaming went from lovable pixel art to "life could be a simulation" in the span of

[@Tatsunero]

WOW this has improved SO MUCH

[@jerbsherb4391]

Imagine video game performers doing their jobs without a mocap

[@HafidzMurshidie]

This is still MoCap..

[@trmbne]

You mean without the mocap suit

[@ryndaman2855]

@@trmbne yup its using Iphone to detect our facial structure, so webcam or android phone cant be use since the technology is better on Iphone

[@subsume7904]

You know what Motion Capture is.. right?..

[@agentys3544]

What a time to be alive guys! Now imagine old games like PoP with this features..

[@pallavrajpatra7227]

Mentions PoP, deserves a like. My man.

[@Nom1fan]

Prince of Persia?

[@TeksTherapy]

Wow, that's incredible!

[@equalgeek]

This is absolutely insane!!!! Congrats to all the devs that developed this tech as you are the real hero's here... seriously!!!!! Thank You!!!!

[@misterteee9471]

Its rough for like a finished game but big budget games that will be using this stuff already has people that tweak little facial movements after they were recorded so this would DRASTICALLY shorten the process its actually insane Insane how every year some tech company unveils their new stuff and each time I'm blown away

[@alexdoan273]

have you seen Ubisoft games? This is in no way rough! This level of facial performance is already better than many AAA games' already

[@averydee5328]

@@alexdoan273 It's not perfect, but they didn't really take their time to make any adjustments, they did it in under 3 minutes, imagine if they took 10 minutes to tweak it

[@misterteee9471]

@@alexdoan273 Keyword "finished" no Ubisoft game is finished lmao

[@jagotiberan2181]

@@misterteee9471 LOL truth about Ubisoft 😂

[@brandonh2690]

Not sure if you watched to the end but after the raw capture they cut to a polished clip. The capture isn't lacking its the environment/lighting/Vfx work that puts it over the top.

[@oldmatttv]

This is pretty incredible for devs and creators, small especially in terms of accessibility etc, but large too for time and budget and such. Really looking forward to this tech being used more widely.

[@Delicate141]

This is truly amazing! Awesome job!

[@Eizenz]

This has terrifying uses. Awesome, but also oh no what have we done.

[@brolys992]

01:23 Angry while smile 😁 ❤

[@purplcyanide]

So, it's not perfect, but the fact that this could be produced so quickly is mind-blowing, especially considering this was captured on a phone.

[@KillerTacos54]

Absolutely OUSTANDING

[@creepyskull3834]

Imagine when it starts processing the facial actions realtime. It's going to speed up the conversion incredibly. 😮

[@kiix]

Wow. Extremely impressive.

[@KloneStudios]

This is absolutely AMAZING. Gawdamn!!!!!!!

[@Kysamael]

In five minutes to transferred a scene. It's amazing.

[@tochukwuudu7763]

Epics killing it.

[@EdsonJuniordj]

Minha cabeça explodiu com isso, meses de trabalho em minutos! Eu aplaudi junto com o público.

[@GotYourWallet]

Stunning demo!

[@mukhlisz]

keep the magic flowing guys! to infinity and beyond!

[@maizeman90]

This is incredible tech that is only going to improve over time. Indie devs and filmakers are going to benefit tremendously from this

[@roger4845]

Competition will skyrocket.

[@darrinstokke3839]

Looks like a big step towards getting past the Uncanny Valley problem. Though the different faces at the end were weird. That is to be expected from different facial structure.

[@MrHappymanx]

This is incredible technology. Wow

[@tunelyricgameon]

This is easily the biggest benefit for me as an independent filmmaker in Unreal Engine

[@higbeez]

This, along with the program that allows people to generate large simulated areas through AI will make it to where one person can make a full length animated movie or game by themselves.

[@SmoothKenny]

I'm genuinely impressed. Bravo 👏

[@ChaosUnfold]

This would not only revolutionize game design but pretty much anything else CGI related. Fantastic!

[@karmah88]

everything ive been learning about animation and modelling over the last 5 years? defeated lol. the rate of development at the moment is really incredible.

[@AyeJVinci]

This looks so cool. If I’m not mistaken the little kid model that they showed on the screen looks like it can be a character in the movie Monster House

[@mroldnewbie]

The last one is the kid model in "A Boy and His Kite" - a previous Unreal Engine demo.

[@Isa13]

amazing technology

[@illikaspain]

Yessir can’t wait to play with this!!

[@TrvvonxLeonn]

This is phenomenal wow !!

[@Superdazzu2]

melina juergens is such an actor, amazing performance, always!

[@vincentkentermusic]

At some point they will release a video like this and reveal the presenters were actually the MetaHumans.

[@michaelayoade1508]

I'm not in a field even remotely in gaming but this is just incredible work

[@emeraldskye4494]

Awesome! So glad she ended up playing Senua.

[@Coindrop-Tim]

This is seriously impressive tech.

[@mousmnwltr]

Wow!! This is insane

[@edongski]

wow! these are giving me goosebumps

[@Charitas25]

From a technology aspect, incredible. From a security perspective, frightening. This can be weaponized in many ways.

[@jjdizz1l]

That was incredible!!

[@ayayron8221]

they did all this in less than 5 minutes on a phone. Imagine what game studios can do with even better equipment and time to properly fine tune the facial expressions.

[@micblogs3629]

Wow the technology of what it can do now is freaking Amazing!

[@stonedgamer9432]

AWESOME !!! RESPECT FOR THE CREATORS OF THIS ENGINE

[@davidzimmerly4973]

The automatic rotoscoping alone deserves recognition.

[@ozarcnetwork]

there is no rotoscoping.... the tracking data its being placed on a 3d character with a 3d background.

[@keshi5541]

Its not rotoscoping. That is different. Rotoscoping is 2d animation technique and this is a 3d model.

[@splattv3173]

That's crazy!

[@solkalibri1376]

Just downloaded it from the App Store. It great!

[@jameshandley7197]

you guys witch extra time you saved you can tweek the animation better but out the box it looks amazing

[@mateusaraujo122]

Impressive!!