Skin Microstructure Deformation with Displacement Map Convolution

(Best Viewed at 1080p resolution)
We present a technique for synthesizing the effects of skin microstructure deformation by anisotropically convolving a highresolution displacement map to match normal distribution changes in measured skin samples. We use a 10-micron resolution scanning technique to measure several in vivo skin samples as they are stretched and compressed in different directions, quantifying how stretching smooths the skin and compression makes it rougher. We tabulate the resulting surface normal distributions, and show that convolving a neutral skin microstructure displacement map with blurring and sharpening filters can mimic normal distribution changes and microstructure deformations. We implement the spatially-varying displacement map filtering on the GPU to interactively render the effects of dynamic microgeometry on animated faces obtained from high-resolution facial scans.
gl.ict.usc.edu/Research/SkinSt...

[@MasterThief117]

I really wish they added eyes to the face when they showed the full simulation. Seeing the realistic face with voids for eyes is...really...creepy.

[@MakayMurray]

damn, CGI effects are gonna be impossible to differentiate soon.

[@AakashKalaria]

***** I have a movie for you... Life of Pi

[@drewid88]

***** The tiger especially is just a staggering piece of work.

[@RubbinRobbin]

***** Yes, but...Can they do a penis?

[@AakashKalaria]

Drew N Furred animal with photo-realism... R&H did beyond expectations work!

[@Yenkieldemente]

***** They are already.

[@TheoBrixtonTheKid]

I feel like I'm playing LA Noire again....he's lying I can tell by his microstructured skin.

[@tdogg06051991]

TheoBrixtonTheKid You wouldn't be able to tell they didn't have this tech.

[@raniaalazzazy890]

+Justin Eckert دي

[@GoldphishAnimation]

+Justin Eckert I'd beg to differ, a witness would be lying and I'd go to point it out, only to call their gender and existence into question, and everything short of opening fire.

[@zaparine]

lol. I got your joke man. Too bad these people have no sense of humour.

[@zaparine]

+Anthony Colon It's sarcasm dude. He just implied this microskin tech is useless if games can't get facial expression right in the first place. In La Noire, however, the characters can look real and life-like without this tech.

[@BekurahToshin]

I have no words... just... what a time to be alive.

[@harlequinne.]

Well...Thanks for that...Eyeball-less lady will likely be haunting my dreams for a while now.

[@barritoothy]

This is true

[@filipelucciano]

CG Geek, please, if there's a way to make it on Blender, please make tutorial video!

[@TheOneAndOnlySame]

Seeing all the steps used, all the video apparatus necessary I propose a revolutionnary way of doing perfect skin simulations : Discard usage of 3 mesh, make the video of the real human skin into a texture and project this texture to a flat IRL surface . Tada ! (and it works with water, fire etc too !) Seriously now, the results in the video are amazing !

[@minetruly]

VERY impressive! Fantastic modeling!

[@dimitreze]

so cool! congratulations to all! ;)

[@jddplays2757]

Getting pretty good, keep them coming.

[@bryansvt92]

It's Amazing and scary! soon we can see hyper realistic characters in movies and games! this would be great! In a near future, when I become a character artist I would like to apply this technique to my characters. Great work and many thanks for explaining!

[@M_k-zi3tn]

Are you a character artist yet?

[@absence9443]

Well regarding games many seem to do less of a good job to keep up with current technology resulting in mediocre character representations, but regardless, are you a character artist now?

[@bbqribz570]

one step closer to crossing the uncanny valley. i think all that's left is, recreating how pours look (as well as reacting under the compression and stretching) small hairs, and minute changes in color of the skin. however this is groundbreaking stuff.

[@aleksandarmilic4424]

***** This won't apply on robots; I mean it could, but it would have to be a 'physical' deformation. I thought uncanny valley was all about robots to begin with.

[@Arctorkovich]

Aleksandar Milic Uncanny valley phenomenon can be applicable to anything with aesthetic properties resembling natural beings.

[@aleksandarmilic4424]

Yes of course, but the origin of the term stems from the research of the phenomenon in regards to human-robot interaction; which is where, in my opinion, it matters the most.

[@Arctorkovich]

Aleksandar Milic I had to look that up because I was unaware that's where it originating. Fascinating. Oxford Dictionary: "1970s: from a translation of Japanese bukimi no tani, coined by the roboticist Masahiro Moti, who created a graph that plotted the emotional response of a human being to a robot against the increase in the perceived realism of a robot; the graph showed a significant dip at the point where the robot's resemblance to a human is perceived to be almost exact."

[@pantherof7]

absolutely amazing, the only part that looked unrealistic (and Im nitpicking here) was around the hair populated areas.

[@VFXKitchen]

insane details !

[@Curiomatic]

Pretty impressive stuff. I will say, however, that properly capturing the quality of human skin is only one of the massive roadblocks to overcome. One thing they have yet to even come close to conquering is the complexity of human body and facial movement, which is arguably more influential in affecting our perception of what is real, since textural quality is often lost even on real human actors due to motion blur, distance, etc. Even with motion capture, movement usually ends up being far too smooth between animated keyframes, and also too rigid looking. But nevertheless, this is a huge step in the right direction, and I hope to see some similarly big strides in capturing all of the nuances of the way humans move. I feel like that's the largest thing holding it back.

[@RaftProduction]

Artsy Omni Who do we have heeeere :3

[@zaparine]

Yeah. It's like making 3d human cells while can't get facial expression right.

[@jorgebernier5d]

This is amazing, but how do I use this on my own projects??

[@henkkramer8558]

If someone could make a Blender addon of this, I'm willing to pay $8.

[@MENIAC404]

Is there any?

[@yourcrush4889]

Sold!

[@Gokhan_B]

Awesome, congratulations

[@AlanResnicks]

fantastic

[@beavis8073]

How would you even do this in any program without making a different texture to apply on the model for every different frame?

[@Climberfx]

Amazing, when is this available on maya?

[@MiRaje8086]

This is amazing

[@Cr3pit0]

This is awesome :D

[@hiimgamerspruzzino5804]

2015: We created a CGI that simulates every part of the skin 2040: We created a CGI that simulates every atom in the body

[@ElectricDaisyify]

i have no idea what he is saying but this is awesome. . .

[@savysarah]

this helps with my drawing :)

[@SoThisOneDude]

So, it's like micro motion blur on parallax mapping..?

[@theplushchannels8531]

Why dose the terminator look so fake,when he was cutting his eye out

[@ColossalCake]

That is insane!!!!

[@brendawsouza6870]

WHERE I GET THAT BUMP TEXTURE

[@SuperGamer87]

I'd make a pretty good that companies like Pixologic are checking this out. And I bet you anything it'll be Pixologic to first put support for displacement map convolution in one of the iterations of ZBrush 5, or somewhere later down the line.

[@steves5476]

3:37 can't help but see the eye sockets as mouths. Creeeep-y!

[@ayakiri5098]

is it possible to get the dynamic displacement map ??

[@strob6946]

Wow!!! amazing

[@goodvibestv5380]

I feel like when these faces are rendered, devs make the faces way too shiny, to replicate the oils on the face. Sure our faces have oils on them, but damn it's not supposed to look like it was doused with gasoline. They could also add a little bit more folding on the skin, just a little bit. And add some tiny hairs on the skin. That's what's really giving it away. But something I've always noticed in video games is that in every facial animation, the eyebrows or lips are almost never captured right. Either the mouth looks way too animated or the eyebrows have little to no movement. The faces are emotionless except for a slight tweak like eyebrows moving slightly down unconvincingly. Or the teeth looking unconnected to the jaw with almost a completely black untextured internal mouth structure.

[@Doincus]

bro that's not the point of the video, they're showing the skin deformation technology, not the whole package of how to make a realistic face. Of course, when someone uses this tech in their project they'll add hair and more convincing animation, but the point in this video is to advertise all the fancy skin breakthroughs they've made.

[@Meteotrance]

my huge problem on any CG software, it's the lack of control of the specularity, even with SSS it does'nt look natural enough, it's just too sharp and not natural, also for non organic surface, we must improved the spec and shadow algorythm with better physical based approach, and a control of softness and anysotropy, with this new study of microstructure normal map.

[@spenceranderson4563]

Cool beans man.

[@innerguardianXIII]

PLEASE give these techniques to Hideo Kojima and his Fox Engine.

[@cavadorarpeiro7812]

Incredible, unbelieveble, awsome!!! I dream with a day where we can transfer anythng of real world to videogames... I think to myself it will be like to play to be God!!! hehehehe

[@barritoothy]

I got your nose *I’ll be havin this*

[@BLUEBLOOD219]

Digitized skin patch samples are available from the website: gl.ict.usc.edu/Research/SkinStretch/

[@Thealexcgart]

amazing

[@bobabooey8367]

So..how do I do it? any links to tutorials, preferably Blender or Max?

[@bobabooey8367]

thanx Test:)

[@PixelPulse168]

where can i find this article paper?

[@Ben3DMaster]

great

[@DutchDread]

How do I implement this!?

[@DavidRodriguezb3d]

awesome

[@dation0000]

wow just wow , this is absolutely amazing congrats to you and everyone one on the team :D this will literaly blow up VR :D , now imagine some how the motion cap was intergrated with the vr headset somehow so in essence while playing with a VR headset it could also capture your facial features . let say you were in a multiplay social VR game / simulation you could talk to and meeet people around the world , i know that is dificult , you could always just animate the facial features just based on how the face deforms when saying certain sound / words i guess it would be easir that way. am not done yet , now take it up a notch hehe VR porn , you and your team would absoluteley explode with recognition plus all the funds you would acqure for the industry :D , even better it means there would be porn that there would be no need for real physical organ interfacing with the actors involved :D literaly the safest porn ever best part the user could tailor to their own unique lets say interesting neeeds :D after all there really is no bad or good publicity ^_^ you really have nothing to lose but only a way to improve the industry :D am currious as to what you have to say about this :D

[@hageby123jean1]

for which program is it for?

[@Mouserzz]

i like this soundbite 2:15

[@Slateproc]

Welp, congratulations, you've tricked my primitive human brain. I can't wait to see this in videogames like TES VI and Fallout

[@beavis8073]

What software is this?

[@TheXeeman]

did they have to chose a creepy face with no eyeballs for this demo

[@jmez927]

OK.... Just a Photoshop Action for skin simulation using a displacement map,gaussian blurs, and sharpening techniques based on directional stretching. In 3d Realtime CGI. Makes sense.

[@obesefrogman]

why couldn't you give her eyes??

[@kuroiei]

This is amazing! Now, - how do I do that in 3ds Max? ;)

[@beavis8073]

What software?

[@pr_smart3486]

cool

[@DivineStride]

Neat! Although it looks like the main thing that was missed was that as skin wrinkles, it becomes more red.

[@houstonhelicoptertours1006]

Because it was not part of that paper, but others cover that.

[@muhamadabdrrozaqzulfa3495]

WOOOOOOOOOOOOOWWWWWWWWWWWWW

[@keithtam8859]

when can we have on Blender XD

[@placebo_yue]

impressive. Between this and virtual/expanded reality, we're not too far from the matrix

[@cpet1082]

Fuck! That’s good!! Already 4 years old?

[@markwincek6688]

in which programs did you make all of this?

[@JuusoAlasuutari]

Mark Wincek Ask Toolbar with QuickBasic

[@Orangoes]

Come on guys. It's RPG maker, Mark Wincek ;)

[@somethingthatpops]

Mark Wincek most likely custom written plugins for some available software, or entirely custom written

[@MuitoDaora]

Mark Wincek IE

[@BLUEBLOOD219]

Mark Wincek Some results are rendered by custom OpenGL/GLSL shaders (real-time), and others are V-ray (offline).

[@Left4TeamFortress2]

Still a bit uncanny but I'd say that must of took months to figure out!

[@user-ut5bi3qm4v]

wow

[@animatrix1490]

Gosh I never wanted to think about human flesh this much. I feel so cursed now

[@miscellaneous9506]

Naughty dog seem like they got the hang of it

[@Parhelion69]

The uncanny valley thing is real, I almost wanted to puke when watching the skin move like that. It was really close to the real thing, but not quite. That's the phenomenon that makes you uncomfortable, because your brain can tell it's not real, but it's so close... I need some rest after this xD

[@IvanF3_]

is good to have this realistic CGI, but is still better to have a good story to tell, even if it's told with pieces of paper. imho impressive, anyway!

[@Otrueniy]

+Ivan Ferrer really?

[@IvanF3_]

+John Doe you're right, but I'm sick of seeing poor stories told using those advanced and realistic tools. They focus on 'how' instead of on 'what'.

[@VillaCarrington]

+Ivan Ferrer This is the most ridiculous comment here. Are you seing a trailer for the last Crysis? This is a documentary, not a game/movie trailer. Dafuq?

[@emz_ow]

+Ivan Ferrer People making the story and people rigging character are not the same people. This is for people rigging characters.

[@johns3937]

+VillaCarrington but what does that have to do with the people doing this? They are two different groups and you are not telling it to the group responsible for stories.

[@ocklarock654]

whats the practical use?

[@kindeep]

gaming

[@ocklarock654]

all that money and research for games and movies? yea ok

[@gotDesign]

Finally they can make 3D realistic porn in 4K without the risk of STDs, extreme close up would look nice

[@bukachell]

Being the person to get scanned would probably feel uncomfortable

[@shaneoshea7991]

You know shits getting real when they have to caption an image 'photo' and 'Dynamic Mircrogeometry'..

[@DeneeHughes]

NOOOOOO why were there no eyes.

[@4relevants]

Having such realistic face in the game, how moral it will be to shoot somebody in the face?

[@campbelljohn]

yep one step closer to full on terminators,in fact I think that we as humans become so advanced in the future,that not only do we have humanoids ,but we also have time travel and their are these humanoids among us building themselves for the future! MINDFUCK!

[@christopher8116]

This is impressive, it's been tested on Vray any way someone interested can test ?

[@BLUEBLOOD219]

The Christopher Skin patch samples are released on website. gl.ict.usc.edu/Research/SkinStretch/ Please feel free to blur/sharpen for simulation.

[@YEASTY_COMMIE]

please add subsurface scattering

[@GUMMY_MKII]

I know it’s been 3 years since you asked but I’m not sure if it’s doable on a normal/bump map. As far as I know, subsurf scattering is only possible on geometry, so you would need an absurdly dense mesh and a very precise height map to notice the effect. However, if you manage to do do it on a smaller scale and bake (for example) an image sequence of the resulting, sub.scat., then you would be able to apply the same map everywhere just like you apply the normal map.

[@YEASTY_COMMIE]

@@GUMMY_MKII Maybe it would be possible to do it on the geometry only without taking the normal map into account ? I don't think the difference would be that noticeable and having both effects at the same time would give very realistic skin

[@GUMMY_MKII]

@@YEASTY_COMMIE I said that mainly because I think it is already applied to the model and I assumed you talked about applying the subsurface scattering to the new normals. 4:18 is a good example, you can see that crevasses appear less red than in the photo, but you can also see that the slight red over the eyebrow (where the geometry actually deforms) and the blue of the veins are present. Applying subsurface scattering really is one of the many secret ingredients of realistic human skin, but the thing is, it’s not really a secret. If you’re interested in making realistic skin and do research for about 30min, you *will* hear about it. There’s no way a team like this would omit such an "obvious" element, but they are researching on how to make a virtual human skin deform adequately, so they either chose or didn’t think about further effects.

[@YEASTY_COMMIE]

@@GUMMY_MKII You're right it does seem to be present at least on some examples, maybe not all of them ? I know it's not a secret, I just assumed they were focusing on one specific technique with basic lighting for the rest I was also 17 when I wrote this comment and had a tendency to give my opinion on topics that I don't know enough about lol

[@GUMMY_MKII]

@@YEASTY_COMMIE Indeed it’s not present on all of them. But yeah, I learned about subscat much later than you and I was very surprised to learn that even 3D non photorealistic animated movies used it. I was impressed and sad 🙃

[@ariamcmann6389]

This is very informative and impressive. Though, as an amateur animator, I need something a little more to-the-point, something that says "Do this. Click here. Do that. And, voilà; your 3D character has realistic skin." I liked the results demonstration. But, since I couldn't find such instructions here, I'm back to KZbin looking for such instructions.

[@Doincus]

I don't think this is meant as a tutorial. It's a tech demo, probably to garner interest from studios.

[@oshietkudafrickinsai1232]

Maybe we can give AI like Siri and Cleverbot faces to go with their speech

[@zac2013]

For those people who think this is going to be in soon a probz wrong... Not meant in a mean way xD the reason is because u can't have for example a Gta game with graphics like this cause the gore will be way to realistic and will be disturbing

[@Jaythakidd_02]

that's a good thing

[@Pertev777]

I bet, girls gone mistake this video as a beauty threading video, if they don't read the title xD

[@cheemsinu2147]

My computer said no and shutted down immediately.

[@ewerybody]

4:37 Whow! believable Angela Merkel close-up!

[@GD15555]

The only problem all cg skin looks too shiny and greasy.

[@myronlee93]

HoLisheet !!

[@sakupekonmaki8387]

too oily skin

[@Shroomsterdam]

Realease the Pascal !!!!

[@JorgeGamaliel]

Synthetic Humans. Artificial life. Living forever, living on video.

[@eulalio1]

Ya no sabemos diferenciar lo real de lo renderizado XD

[@Pinkyu14]

L \ London

[@SavvySteak]

They gotta work on ears and eye lids better.

[@m4r_art]

Ok so this is what Unreal Engine 5 is going for

[@Tribefull]

Techmology.

[@FernandoBaldrich]

jumpscare at 2:28 you're welcome.

[@WiseWik]

Half Life 3 confirmed!

[@IneptOrange]

Now try putting this on a console without turning it into a superheated bomb xD

[@jcuadrado9518]

😰

[@MauroMan_]

..Next : ... star trek holodeck..

[@Guss67T]

Really really good, but there are some small details that would make it more natural. For example at 3:49 "Too shiny" some facial areas aren't too much shining ( i understand that would not be easy to notice the details without too much glossiness), at 4:28 when the eye squeezes the skin around should go more "Red" from the pressure, at 5:05 when stretching the pores go too much shinny. Well those examples i gave are not the same on everyone's skin.