3D Face Reconstruction with Dense Landmarks

Landmarks often play a key role in face analysis, but many aspects of identity or expression cannot be represented by sparse landmarks alone. Thus, in order to reconstruct faces more accurately, landmarks are often combined with additional signals like depth images or techniques like differentiable rendering.
Can we keep things simple by just using more landmarks?
In answer, we present the first method that accurately predicts ten times as many landmarks as usual, covering the whole head, including the eyes and teeth. This is accomplished using synthetic training data, which guarantees perfect landmark annotations. By fitting a morphable model to these dense landmarks, we achieve state-of-the-art results for monocular 3D face reconstruction in the wild. We show that dense landmarks are an ideal signal for integrating face shape information across frames by demonstrating accurate and expressive facial performance capture in both monocular and multi-view scenarios. This approach is also highly efficient: we can predict dense landmarks and fit our 3D face model at over 150FPS on a single CPU thread.
Learn more about this work: www.microsoft.com/en-us/resea...
Microsoft Research's Mixed Reality & AI Lab - Cambridge: www.microsoft.com/en-us/resea...
Microsoft Research's Mixed Reality & AI Lab - Zurich: www.microsoft.com/en-us/resea...

[@ezioauditoredafirenze1108]

Looking forward to try it. Please release the code along with the model.

[@user-vk2yx2vd8p]

excellent work!

[@artofiran]

Great Work👌I am looking forward to releasing the AI model or library.

[@juanSpard1]

Would very appreciate learning about this and how you get there.

[@errorserver3000]

This is great. When can we expect the release?

[@calebweintraub1]

Excellent

[@ramgopalnalluri]

Thanks

[@michaellinner7772]

Cool, now we finally know how the 'Blue Man Group' came into being. 🥶🥶🥶

[@andrelecozvideographer9030]

any info on when this will become a product i can purchase? any clear date on the horizon?

[@integralogic]

Nice

[@SnrubSource]

this would be great for vr

[@kahmad3428]

Please release the code along with the model.

[@yuryeuceda8590]

Where are the repositories to help testing

[@eliu20050101]

are you going to open source it ?

[@JT-cs8wb]

Please release the code!

[@v.smourya8005]

Did they provide the code and the model yet?

[@bookcursory]

Future is here boyz

[@MiniloCrafting]

App name?

[@OriginRow]

Wow PSVR headest in Microsoft Research project 😅

[@DriesduPreez]

3:14 Ladies and gentlemen... The Blue Man Group

[@mattanimation]

blue man group is gonna be out of a job

[@JD-os2kr]

I had a similar thought. I appreciate your sense of humor!

[@eliu20050101]

amazing it could be done with laptop grade CPU ...

[@luisfelipeortega9411]

The Blue man group was created before.