AI Face Body and Hand Pose Detection with Python and Mediapipe

Want to start building body pose based apps?
Maybe want to control your screen using nothing but gestures!
Well, Mediapipe and Python are the answer! In fact, in this video you'll learn the basics for getting started with Body pose detection, facial landmark estimation and hand pose detection using a single Mediapipe library and your webcam.
Behind the scenes, Mediapipe Holistic, the model shown in this video, uses deep learning models to be able to accurately detect keypoints. Using this model you can begin to prototype a whole bunch of different use cases like touchless gesture control, human sentiment analysis and could even build your own exercise counter!
In this video, you'll learn how to:
1. Install Mediapipe and setup Mediapipe Holistic for Python
2. Access a real time video feed from your webcam using OpenCV
3. Detect and visualise facial landmarks, body poses and hand poses
Get the Code: github.com/nicknochnack/Full-...
Chapters:
0:00​ - Start
2:49 - Installing Mediapipe and OpenCV
8:23 - Real Time Video Feed from Your Webcam using OpenCV
13:05 - Detecting Landmarks for Face, Body and Hand Poses
30:48 - Changing Landmark Colors and Styling
Oh, and don't forget to connect with me!
LinkedIn: / nicholasr...​
Facebook: / ​
GitHub: github.com/nicknochnack​
Patreon: / nicholasrenotte​
Join the Discussion on Discord: / discord
Happy coding!
Nick
P.s. Let me know how you go and drop a comment if you need a hand!

[@ionutos2]

This video keeps delivering even up to this day(2-3 years later)! Great!

[@PhilMcCarty]

Fantastic tutorial. Minor update: the mp_holistic model has changed the name for FACE_CONNECTIONS to FACEMESH_TESSELATION

[@Tigas4ever]

YOU ARE A LIFE SAVER MY FRIEND!

[@DaniELGKDG]

I LOVE YOU

[@tymoteuszjaworski5921]

THXXXXX

[@PhilMcCarty]

Your life needs to be lived @@Tigas4ever

[@PhilMcCarty]

It's so early in the relationship but yes I love you too @@DaniELGKDG

[@wgalloPT]

Best teaching ive seen on this subject so far in youtube (and Ive seen millions of them)...thank you for this...really...thank you!

[@hass89]

The quality of this and your other videos is outstanding. Great job Nicholas.

[@NicholasRenotte]

Thanks so much @Hassanin!

[@bibobo8276]

OMG big thanks Nicholas! I have gone thru so many overwhelming docs still couldn't understand how Mediapipe works until I watched your video. Thanks for making it easy and interesting🥰

[@user-if5gn5pb8y]

It's very cool! There are a lot of videos like that, but your videos are the most interesting and you explain it clearly! Thanks! All works correctly and without mistakes and bugs.

[@euyangchai9982]

This is a godsend and deserves more views dude. Fantastic job explaining and breaking it down individual lines of codes for beginners like me as I'm working on a Final Year Project that detects pose landmarks and using it to synthesise movements from images.

[@NicholasRenotte]

Thanks so much @Eu Yang Chai, gl with your capstone project!

[@euyangchai9982]

@@NicholasRenotte Hey Nick, I would like to request a tutorial video on motion transfer using pose estimation (i.e. detecting pose landmarks and using it to synthesise movement on images) using Python and if possible, using Mediapipe as well :) let me know if this interests you, thanks! P/S I've also left you a message on LinkedIn because I desperately need help on that..

[@NicholasRenotte]

@@euyangchai9982 definitely, working on it atm! Exploring how to do it with Tensorflow using the Barracuda framework!

[@euyangchai9982]

@@NicholasRenotte This is awesome! Thanks man, really appreciate it.

[@cong1393]

Yay so keen you just released this! I was working through how to do body language recognition for the TV show Frasier.

[@NicholasRenotte]

CONG!!! Thanks so much, awesome, quick note, you might need to render the lines as bolder if your image res is larger!

[@amishagangwar8686]

Explored your channel today! Amazing content ! Was waiting for such channel since way long back !❤️

[@NicholasRenotte]

Thanks soo much @Amisha! Glad you enjoyed it!

[@jiuqiangtang2227]

Great video! A small optimization is to set "image.flags.writeable" to False before sending the image object to Holistic and flip the writeable flag back to True after the process() call. By doing this, you will 1) avoid copying the image data by passing it by reference and ii) reuse the same image object for rendering ;)

[@NicholasRenotte]

HOLDDDD UPPPP, did I just have THE @Jiuqiang Tang comment on this!? Thank you sooo much, love your work!

[@moses5407]

FANTASTIC! YES, would love to see a continuation of this series, possibly including live display and minmax detection of joint angles at selected joints, velocity of motion for selected landmarks.

[@moses5407]

Would also be great to see an auto-reframe/dynamic crop of a wide angle view to do landmark-detection-driven "auto tracking".

[@moses5407]

Can the hand and face landmarks from the body pose set be dropped/made invisible selectively?

[@NicholasRenotte]

The auto reframing would be awesome! I saw a colleague do it recently, will add it to the list. Did you mean the invidual landmarks or the whole series e.g. drop all of the face landmarks?

[@raecanigiani2278]

This tuturial was really helpful, thank you for your work making it! I saw some other commenters mention it too, but using this for realtime 2D/3D avatar manipulation sounds like a dream come true. I use some of the programs vtubers use for training students on motion capture, so it'd be awesome if there was a way to use mediapipe to generate the vmc protocol for something like vSeeFace! Looking forward to seeing if it could be possible!

[@NicholasRenotte]

Ha, definitely going to dig into the linkage to vtubing software @Rachel, the ones that I've seen so far don't seem to have open APIs though. Just realised vseeface is in unity. Will dig into it some more!

[@nilutpolkashyap]

This project is really great. Thanks for it. Please make a full series for this project. Big fan of your work.

[@NicholasRenotte]

Definitely, I'm going to get onto it next @Nilutpol. Thanks sooo much!

[@alirezamortezaei5550]

What amazing dreams can come true with this tutorial 😍. Thanks Nicholas

[@sujaysaha6323]

Thanks a lot for listening to my request, Mate. Love Ya.

[@NicholasRenotte]

Anytime my man! Glad you're enjoying it @Sujay.

[@hsiaohsu4209]

Looking forward to the series on applications built using Mediapipe. Thank you!

[@NicholasRenotte]

Awesome! Anything in particular you'd like to see?

[@hsiaohsu4209]

@@NicholasRenotte Yes! Exercise counters perhaps? Like jumping jack counter or something of that sort? Thanks!

[@NicholasRenotte]

@@hsiaohsu4209 definitely, that's going to be first off the list!

[@Data360YP]

Great video Nicholas! Love the content!!!

[@NicholasRenotte]

Right back at ya @Data 360 YP!

[@richieithaca]

Nicholas this is exactly what I was hoping you would cover. Love the sound of some examples of rep counters (squats, press-ups, etc.) or body language detection which sounds really interesting. I know there is a lot of research on eye-tracking in psychology I'm sure people in that field would love to see some of that. I would really like to see some onscreen angular output or a way of outputting that data to a file while tracking. The media pipe documentation outlines a Z component for the model which would be very interesting to see in terms of what the model sees in that direction or the computation of angular change during a movement. Keep up the great work man I'm really enjoying the level you are pitching this at - the fact that I can follow along is a testament to your approach. :-)

[@NicholasRenotte]

Thank you so much @Richard, it means the world! Will be building on top of this, I really like the idea of bringing in the z axis into play. I'll brush up on my trig as well and get some angle calculations in there as well!

[@richieithaca]

@@NicholasRenotte I would really appreciate that thanks ;-)

[@NicholasRenotte]

@@richieithaca I'm on it!

[@maryogbukakenneth2465]

Wow!! I love this tutorial. It is very simplified and easy to follow and understand. I must say you are doing a great Job. Thank you for this

[@mohitkale6879]

Awesome explanation and demo.

[@tejaschoudhary3882]

Amazing work Nicholas! I'm waiting to see how you apply this- such as using gaze tracking to learn if someone is paying attention during zoom calls or learning about your morale by figuring your emotion through facial landmarks. Side note: I'm working on some of this stuff myself and would love to discuss it with you if you are interested

[@NicholasRenotte]

Awesome work @Tejas, got a bunch more stuff coming in this space particularly for body language detection.

[@user-ek9ho1ny8d]

Thanks for the content as always. Could you suggest some tutorials of docker and amazon aws? In general it would be interesting to see the whole picture as well (meaning not only model creation but its deploy/implementation to production)

[@bernardoolisan1010]

Dude this is fire, its amazing the stuff you can do with python, you are fire bro!

[@NicholasRenotte]

IKR, it's my absolute fav language! Thanks soo much @Bernado, plenty more to come!

[@SohamMangore]

Thankyou very much. Keep up the good work really helped a lot of us ♥

[@thirashapw]

Amazing content Nicholas. Thank you!

[@NicholasRenotte]

Thanks soo much @Thirasha!

[@AbdulGaffar-pu8mr]

Thanks for such explaining.

[@01bit]

Your Videos are excellent!

[@NicholasRenotte]

Thanks a ton @01bit!

[@prakhars962]

This is really cool. Also, Instead of copying the code, I wrote the code with the video, so that I can write my own comments.

[@NicholasRenotte]

Awesome work!

[@danieladama8105]

Can’t wait for the full project 🔥🔥🔥

[@NicholasRenotte]

Yessss, anything you'd like to see as part of it @Daniel?

[@danieladama8105]

@@NicholasRenotte yes, I was thinking if you could include an Iris detection using the Mediapipe package

[@NicholasRenotte]

@@danieladama8105 agreed! It's only available in C++ atm but I actually found a workaround.

[@danieladama8105]

@@NicholasRenotte Yayyyyyyy!

[@NicholasRenotte]

@@danieladama8105 ayyyyeeee!

[@abdullahe3674]

Your quality of videos is improving 😉 Awesome video :)

[@NicholasRenotte]

😅 phew! Hahaha thanks so much @Abdullah!

[@shankarkadam4184]

This is very helpful tutorial thanx for sharing brother.

[@buttscratcher4530]

thanks buddy , very helpful to understand

[@dmitryshkadarevich4020]

Man, thank you so much, you've done really great job! :))

[@NicholasRenotte]

Thanks sooo much @Dmitry! Glad you enjoyed it.

[@2498hellboy]

You, sir, have a really nice channel! I would love to see some tensorflow in react native, I really think that it has a lot of potential! Keep up the good work!

[@NicholasRenotte]

Thanks so much @kento, definitely still getting up to speed with React Native but it's definitely coming!

[@markusbuchholz3518]

It is very awesome and marvellous to watch you contribution in spreading the knowledge. Your current presentation (video) is an inspiring start for our own innovative projects (apps). Based on your contribution many different applications can be created (no limits). Big tomb up for you (unfortunately I can give only one) and Google team for the effort in creating and sharing state of the art work. Have a nice day.

[@NicholasRenotte]

Agreed, it's amazing what you can do with the models that are already out there! I've actually already built a pipeline on top of this code that allows you to apply a custom classifier from the keypoints, possibilities are endless!

[@khukuhprihatmikho7215]

Good tutorial video!! thumbs up for you

[@didunou4587]

You are the best Nicholas keep it going 😍

[@NicholasRenotte]

Thanks soo much @DidU!

[@mherrera66]

Great master @Nicholas! I had already seen some of #mediapipe but with your video I realized that I had only seen the tip of the iceberg, thank a lot.

[@NicholasRenotte]

Heyyyy, thanks for checking it out Marcelo!

[@nakulsaxena5138]

Thanks a lot, bro, I remember I said to you that please make a video on this topic and finally I got this. {Thanks a lot}😄 while True: print("Thanks a lot ")

[@NicholasRenotte]

youre_awesome=True if youre_awesome==True: print('Anytime my man, a promise it a promise ✌️')

[@andreoliveiracunha3084]

Fantastic! Congrats man! 👏

[@NicholasRenotte]

Thank you sooo much @Andre, glad you enjoyed it!

[@abhishek_maity]

INTERESTING!!! YOU ARE A GEM 💎 NICHOLAS !!!!! Waiting for next video now :)

[@NicholasRenotte]

Thanks so much @Abhishek! Definitley, plenty more to come.

[@NicholasRenotte]

Heya @Abhishek, pt 2 is out! Body language decoding! kzbin.info/www/bejne/jZaUpnVtbqicY9k

[@lorenzollamas9939]

Bro your the man and especially a genius 👏great work keep it up will love to see the end result

[@NicholasRenotte]

Ayyye, thanks so much @Lorenzo! Definitely, got some sweet stuff planned with it!

[@NicholasRenotte]

Heya Lorenzo, the follow up is out! kzbin.info/www/bejne/jZaUpnVtbqicY9k

[@g0gool_]

bro i'm a broke guy who cant afford rokoko and iphone x and you are my solution. love you

[@hagarsalpeter1288]

Hello Nicholas! Thank you very much for this helpful tutorial Do you have a nice solution to get the landmarks of the mouth and lips specifically?

[@priyambordoloi771]

Its amazing. Will always appreciate it.

[@NicholasRenotte]

Awesome stuff @Priyam!

[@dharmagurram958]

Without stopping the code and rerunning the release lines I think as we mentioned in starting code pressing 'q'(0xFF = 'q') in the pop up helps to do that. By the way Thanks, Great job! :)

[@NicholasRenotte]

Nice, I think sometimes it's a little glitchy on my PC. Looks like it works well when combined with cap.isOpened() as well.

[@infinitech8]

Thanks for this incredible tutorial I thought it can be something in healthcare as it is tracking the body movement (bones) but I am not sure yet. Anyway it's great

[@user-fp3qr3gb3l]

Great tutorial. Is there an example or tutorial on picking of object (=boxes/packages/parcel ) by hand, tracking the movement of the object and identifying the bin/shelf number in which it was dropped?

[@tarantborlase6440]

Great videos as always, I was just thinking how far away are we from using this in skill training? Could be something like record someone doing a task well, then compare a learner and calculate the distance from the 'ideal' as a metric of progress?

[@coolrock3733]

This is really informative!! Really helpful so here's a thanks from I.

[@NicholasRenotte]

Thanks so much @Cool Rock!

[@coolrock3733]

@@NicholasRenotte No u XD. Hopefully we can make proper use of this for school purposes.

[@NicholasRenotte]

@@coolrock3733 definitely, got a use case in mind?

[@codaemon375]

Enjoyed this video. I was able to keep up with the pace. However, I’m running this on a raspi pi and I have some latency issues. Will it be possible to target the GPU instead of the CPU?

[@kivenzamora7728]

Great video! A question: Can this method be used to convert head movement and facial expressions into keyboard input? If so, where can continue learning to develop this?

[@jiebao2223]

Amazing content!

[@NicholasRenotte]

Thanks soo much @Jie!

[@vietnguyenquoc6366]

Fantastic. Can you make a video of hand gestures?

[@RJ-jg3vl]

damn, thank you so much! that helped me out a LOT!!

[@RJ-jg3vl]

and really nice step by step explanations as well. love it!

[@NicholasRenotte]

So glad you enjoyed it @R J

[@adityatiwari3646]

Thankyou So much SIrr💌💌

[@akarsh008]

i had an idea using this for liveness detection

[@dave3083]

Wow, Great tutorial! wondering if this can be use for VTuber! they don't need to buy expensive tools for body & hand tracking

[@haziq7885]

Hey thanks so much for the vid! How would you implement this on recorded video (say mp4 that’s extracted from youtube) as opposed to live video?

[@muhammadsalik5501]

Thanks

[@jonnathanwilson5300]

Thank you very much for this great tutorial. Is there a software to make your own model (to use in game in unity)? Thank you very much for your help!

[@rekhytze781]

very good video, it helps a lot by the way is it possible to make a hairstyle filter using mediapipe? but wont looks like a wig?

[@henkhbit5748]

Great introduction to mediapipe! I suppose it can also detect multiple persons? How can u detect if a person is lying on the ground? More videos on mediapipe would be great.

[@NicholasRenotte]

Thanks Henk! I believe the Holistic model is a single person model but you can use the pose detection model for multiple people! For detecting those lying on the ground you could apply a secondary ML model to classify based on keypoints, I've got something in this space coming!

[@henkhbit5748]

In anticipation of your coming video! btw: the landmarks coordinates x,y,x are between [0,1]. Are this is in inches, cm or..? If not how can you map to cm for example?

[@amaurybonneau4602]

On a jupyter notebook, you can create a "key" variable with a value of 0 then make your while with a condition of key!=27. During the loop, you set the value of key to cv2.waitkey(10). It will escape the cam capture window when you press the escape key.

[@NicholasRenotte]

Sweet suggestion @Amaury!

[@alessiopasquini2183]

This video is great, but i have one question: is it possible to extract only one coordinates and print only that. For example, i want to print only the y value of my left hand. Thanks

[@user-of2mi7qy7g]

Thanks for nice tutorial! I have a question about mediapipe that if its possible to train the model on custom dataset provided by python language or just pre-trained model is available

[@NicholasRenotte]

The keypoint model, no. But you can use the mediapipe model as an input into another custom model!

[@hiteshvaidya3331]

Hi Nicholas, Thanks for doing this project. Extending this module to workout count would be really exciting. Looking forward to that video. Also, I tried the discord link but it seems the link is broken hence I couldn't join.

[@NicholasRenotte]

Thanks soo much @Hitesh. Oh noooo, for real it's not working?

[@IRaziel]

Thank you

[@NicholasRenotte]

Anytime @Kadir, glad you enjoyed it!

[@pfever]

Would be awesome to see action detection done on top of this! :D

[@NicholasRenotte]

Working on it as we speak!

[@rebeccaruben4121]

can you please do a video on Stroke detection using FAST (Facial drooping, Arm weakness, Speech difficulties and Time)

[@enasossama]

Thank you for sharing this tutorial. I tried to run the code but I am facing difficulties to make it work. The window with the camera opens, however, the outline for face, hands and body does not show. I am using PyCharm.

[@abdissagetachew5999]

Hi Nicolas NIce work

[@NicholasRenotte]

Thanks a bunch man!

[@saiganesh3693]

Bro, This is Awesome How about extending this project to create a sign language detection component that detects from the continuous video feed for which we don't need to label specifically for every gesture (like you did in the last live stream) or from a complex ready-made dataset from the internet that includes facial, hand and poses for recognition like British Sign Language, etc. It'll be lit.

[@saiganesh3693]

Can you guide me on how to build that sign language thing on top of this?

[@NicholasRenotte]

Heya @Sai, check this out: www.tensorflow.org/hub/tutorials/action_recognition_with_tf_hub

[@emmanuellaude9932]

Thank you very much Nicolas for this great video. Is it possible to follow the iris at the same time as the body? In the documentation we can read: "REFINE_FACE_LANDMARKS Whether to further refine the landmark coordinates around the eyes and lips, and output additional landmarks around the irises. Default to false."...

[@jinglebells6023]

Hey man, where do you come across this stuff? Are there any particular sites, channels, or subreddits you follow where these come up? Would be awesome to know! Great videos :)

[@NicholasRenotte]

A lot of it is through client conversations, reading reddit and seeing new projects on LinkedIn and Github!

[@fahadabdullah510]

Can you give little idea or make a separate video on how to apply clothes etc on human body using pose detection?

[@bernardoolisan1010]

Hey Nicholas, I’ve looking for voice classification tutorials but there is nothing!, it would be great if u teach us with your own database!!

[@NicholasRenotte]

You got it, I'll add it to the list @Bernardo. Also saw an awesome example of voice cloning the other day as well!

[@FaridSharby]

Hi Nicholas, can we use it to detect the outline of our body?

[@wgalloPT]

If you ever wanted to help, I actually was trying to get the angle of the wrist but it is confusing to me, because of the way it draws. All i need is the segment of the elbow connecting to the point in the wrist, and then from the wrist to the middle of the hand to get the angle I need, but it appears to me that there are more then one connecting point in the wrist...?

[@alessandrobeatini1882]

Hi Nicholas Do you think it is feasible to project over the forearm a semi-transparent image (or a grid)? I am doing research for my theses in the field of therapeutic FES , and part of the problems I am trying to address is an effective routine for stimulating electrodes placement. In order to work, they have to be near where the nerves ''get into'' the muscles. As of now electrodes are placed randomly and then the patient has to undergo a very lenghty process for mapping electrodes activations to hand/finger contraction/extension. So there is a lot room for improvement. The topology of those points is relatively invariant among individuals (both healthy and not) , so I think that being able to project a semi-transparent anatomic table over the forearm(even if it is not extremely precise), could significantly speed up the process for the subjects and enhance the efficiency of the stimulation, as electrodes are placed more easily near those ''motor points''

[@NicholasRenotte]

I believe you could. Presumably the grid would need to be in a 3 dimensional vector space to be able to place it somewhat accurately. I'm experimenting with overlays as we speak, I've had a bit of a backlog but have a lot more planned in this field!

[@ameerazam3269]

Thanks 😊

[@NicholasRenotte]

Anytime @Ameer, glad you liked it!

[@t-raystudios]

Thank you for the videos and this is real nice, qq, how could this be used as a blender addon to map this to a 3d model, for realtime mocap?

[@missty5290]

Hello this is my 1st time watching your video...is it possible for you to do multiple hands detection and counter?

[@thirashapw]

Nicholas, can you do a lesson about generative adversarial networks (GANs) using TensorFlow?

[@NicholasRenotte]

Yup! Definitely @Thirasha!

[@valentingarfi3021]

Hi Nicholas! Amazing tutorial!! However, I'm looking forward to apply this to an android app... how can I do that? Google's Mediapipe guide for Android gives me tons of headaches tho... thx!

[@salmidjedjiga609]

Thank you so much, you have done a good job. I have just a question how can we add the FPS measurement?

[@NicholasRenotte]

Should be able to add it! Check this out: learnopencv.com/how-to-find-frame-rate-or-frames-per-second-fps-in-opencv-python-cpp/

[@salmidjedjiga609]

thank you so much @@NicholasRenotte . This algorithm detects only one person, have you any idea how to detect more than one, please?

[@NicholasRenotte]

Heya @@salmidjedjiga609 I don't believe this model supports multi person tracking. You could take a look at OpenPose though for multi person capture!

[@rafaelbueno2889]

At first,your video is amazing,i liked a lot,and second I have tried to run the code ,but doesn't appear the markes in my webcam video.Is it because the cam quality?The program version?(I've the comunity version instaled) Or I need to impor something else to the project.Once again,your videos are sensational.

[@NicholasRenotte]

Try dropping the detection and tracking thresholds!

[@fadieid5638]

Hello and thanks for the valuable information and guidance you are putting on KZbin, I have no experience in Computer Vision. I am working on a software that can detect a body and get it's position stored in a variable so I can track it with a motorized camera mount. The project is working correctly using the Haar Classifier from openCV for face and pedestrian detection but I need more accurate results for accurate tracking and less false positives and this project seems to be what I need. Can you guide me on how to implement this program into a program that can run on the machine of my customers without them needing to download all those libraries and files with training data... How to implement those projects on a real product. Thank you again.

[@soothingrelaxation8725]

Amazing videos! I am novice in this field and need to use Unity VR and Blender. How can I use a neural network that feed with the images from the video stream to recognize the hand poses. Then you use a 3D model of the hands and esp. its skeleton to apply the recognized pose to the hands that represent the user’s hands in VR (Unity)?

[@NicholasRenotte]

I'm a bit of a rookie in Blender and Unity but I think you might be able to export the keypoints from these detections and use rigging to attach it to a hand model. Got it planned but haven't got the process fully thought out yet!

[@soothingrelaxation8725]

@@NicholasRenotte It will be great if you can show as this as I am really stuck on how to do it. Which hand model and dataset u recommend to use if we are looking to achieve a 60 to 90fps?

[@rrplaygames2883]

Does this library works for wrist detection as well or if there's any other model library for wrist detection then please let me know

[@firthmanee7735]

first of all I would like to TY for doing this video ,it amazing ! I have a few question,I am trying to do pose estimation correcting exercises posture it is possible that next video you can do this topic ? , or can you guide me how should I start ? , thanks in advance.

[@NicholasRenotte]

Would take a look at this, you could customize to train for good vs bad poses: kzbin.info/www/bejne/jZaUpnVtbqicY9k

[@firthmanee7735]

@@NicholasRenotte thanks ! ,really appreciate it. :)

[@PANDURANG99]

Instead of web cam I wants to use live desktop screen to detect objects, is it possible? And show in desktop rectangle itself?

[@user-bp6sv6or1p]

Very helpful and if I need to take the voice of a person what library it's the best to use.

[@NicholasRenotte]

Heya, try looking for voice cloner libraries. There's a few new ones out there!

[@user-bp6sv6or1p]

@@NicholasRenotte thanks

[@NicholasRenotte]

@@user-bp6sv6or1p anytime!

[@modyngs5985]

Hey nicholas, Thanks for the Video ... is there anyway to modify the keypoint? I mean, during the video there are both pose and hand gesture for hands ( they are overlapped) ... the same for Face ( face landmarks and pose landmarks overlapped each other for keypoints in face ) ... is there any option to modify the points? Thanks

[@sorgaxD]

Hey bro, i need exactly this, did you resolve the keypoints problem?

[@c.cassette6259]

i going to create a cinama 4d plugin to detect face mocap help plz

[@NicholasRenotte]

YESS! Sounds awesome! Planning on going some stuff in that space when I get some bandwidth!

[@MotuDaaduBhai]

Is there any way to connect this real time data in Unreal Engine model for Real Time animation? I have been looking for a good way to do real time animation production and your tutorial is very promising.

[@shwetapatil1726]

@Nicholas Renotte Great Video. How can we use this face and hand pose detection to build an attendance system?

[@NicholasRenotte]

In the works!

[@nikhilchhabra]

Questions: Can we access and modify the architecture ? Can we retrain on new data to make it work on new edge cases ?

[@NicholasRenotte]

1. For MP Holistic, no, but you can build custom models on top of it: kzbin.info/www/bejne/jZaUpnVtbqicY9k 2. Yep, see link above

[@ahmedwaly9073]

That's Amazing

[@NicholasRenotte]

Thanks so much @Ahmed!

[@Alex-gu4mi]

Hi! Thanks for this video, is very useful. I have a question, how can I access to the info? I mean, I want to know if I can somehow know the position of the fingers on each hand to use them for a project. I want to replicate the movement of the arm to move an arm robot, so I need the coordinates of the shoulder, elbow and wrist. Thanks!

[@NicholasRenotte]

Heya @AI, check this out, I show how to get the coordinates in here: kzbin.info/www/bejne/Zme3dpKLZ5Z5gZY