Real-Time 3D Pose Estimation For Motion Capture With Camera | Game Futurology #20

This is episode #20 of the video series "Game Futurology" covering the paper "XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera" by Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll and Christian Theobalt.
PDF: arxiv.org/pdf/...
Authors' Video: • XNect: Real-time Multi...
Game Futurology:
This is a video series consisting of short 2-3 minute overview of research papers in the field of AI and Game Development. This series aims to ponder over what the future games might look like based on the latest academic research going on in the field today. Subscribe for more weekly videos!
Abstract:
We present a real-time approach for multi-person 3D motion capture at over 30 fps using a single RGB camera. It operates successfully in generic scenes which may contain occlusions by objects and by other people. Our method operates in subsequent stages. The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals.We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy. In the second stage, a fully connected neural network turns the possibly partial (on account of occlusion) 2Dpose and 3Dpose features for each subject into a complete 3Dpose estimate per individual. The third stage applies space-time skeletal model fitting to the predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose, and enforce temporal coherence. Our method returns the full skeletal pose in joint angles for each subject. This is a further key distinction from previous work that do not produce joint angle results of a coherent skeleton in real time for multi-person scenes. The proposed system runs on consumer hardware at a previously unseen speed of more than 30 fps given 512x320 images as input while achieving state-of-the-art accuracy, which we will demonstrate on a range of challenging real-world scenes.
Music Credits: www.fesliyanst...
----------------------------------------------------------------
• KZbin - / deepgaminga. .
• Twitter - / deepgamingai
• Medium - / chintan.t93
• GitHub - github.com/Chi...
--------------------------------------------------------------------
#DeepLearning #GameDesign #GameDevelopment #MotionCapture #PoseEstimation

[@DeepGamingAI]

This is episode #20 of the video series "Game Futurology" covering the paper "XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera" by Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll and Christian Theobalt. PDF: arxiv.org/pdf/1907.00837.pdf Authors' Video: kzbin.info/www/bejne/qpXbfqJnZrB9fc0

[@thomasgoodwin2648]

You seem to have a knack for finding the truly cool in the sea of meh. Always look forward to seeing you in the inbox. You hit auto-like status at least 10 videos ago. Thanks!

[@mssburr]

This would be over the top if it also captured the hands/fingers and facial expressions... Cool informative video.

[@Jennifer-yz6uq]

You are genius bro.

[@ymi_yugy3133]

I wonder how much better motion capture could be with a couple of Kinect cameras or iPhones with lidar. A full motion capture studio is obviously out of reach for smaller game studios and even bigger ones need to limit their use of it for budgetary concerns. Using a few consumer available cameras however seems totally doable. Sure a single RGB camera would be optimal, but that isn't really necessary and quality is probably the top priority.

[@Augmented_AI]

Do you know if it works in Unity?

[@DonaldG-qq4ol]

@1:04 he had jokes. People are uploading more of these on KZbin. I believe you'll do well.

[@nightmisterio]

Can I combine 2 angle videos of the same capture?

[@DeepakRavi93]

So Cool 😎 wanted to try 😈

[@HexOverride]

Can I use this?

[@THE-MNG]

nice

[@DP-ee6qv]

Software name?

[@zeeshanjillani]

Hey where can I find this code on your GitHub?

[@chitti1120]

Where can i find the code?

[@henrytheswimmer9453]

first