FreeMoCap Pre-recorded Videos Tutorial

[@ap320-u6t]

I am someone who recently started using this project. I needed this video, so thank you for uploading it.

[@fluxrenders]

I'm glad that it's useful. It just covers the basics so additional questions or comments can be solved in the server.

[@thecommentermaker]

Cool to see this project develop.

[@fluxrenders]

Yes, hopefully there will be new features this year (object/animal tracking, real-time, etc.)

[@sebastianluehring7488]

You’re amazing ❤

[@chillsoft]

Thanks so much for furthering our knowledge when it comes to this one! Free motion capture solutions need to be created and this one has so much promise! I was actually on my way to AI code a mediapipe to bvh thing but this seems to do it all! If we record from iPhones and start them manually (don't think there is automated way?) we have to cut the video to be exact same length, correct? Also, is there not more info about occlusion to extract for mediapipe not to fail? If we use, let say, 5 cameras instead? How to show the checkerboard to so many cams in a circle? Again, thanks so much for all of this, you are creating the best solution imho!

[@fluxrenders]

Yes, it certainly is a nice and promising free motion capture solution. Which is needed indeed. I usually record with phones, webcams (with OBS) and a gopro and then I synch the videos in DaVinci Resolve. The software has an import option where you can choose to auto sync the videos based on audio or on a brightness change. Have not tested that much though. Regarding occlusion, it affects the capture to some degree, but I discussed with the devs a possible option to discard the worst camera view in each frame or something similar. That way having 4 or more cameras gets really useful. The checkerboard needs to be seen simultaneously just by a pair of contigous cameras. In other words, each camera needs to share some checkerboard detected frames with at least one of the other cameras.

[@chillsoft]

Ah, yeah, that would be a smart solution, because right now it is averaging it? Would be cool if the bone could be recalculated from another view if the current view has a too high error threshold, but maybe that is too hard. Regarding the checkerboard, that sounds great, as long as two cams see it it works! Thanks so much for the detailed answer, really appreciate it!!!!

[@ruiqingfeng]

Hello, could you please share an example video with me? I can't find suitable input material

[@fluxrenders]

Hi, check the video at 0:20, it shows how to download a sample of 3 videos to do the calibration and processing in this tutorial.

[@ruiqingfeng]

@@fluxrenders Seems that it doesn't exit anymore so could you please help me with it😭

[@fluxrenders]

@@ruiqingfeng I checked version 1.2.2 and the option is still there. It's better that you go to the Discord server and ask so there is more people that could help.

[@ruiqingfeng]

@@fluxrenders Okay I tried again and it downloads properly,thank you very much!

[@rec-trick]

awesome but why cameras not working with wifi mobile cam and android cam for add more cameras without wires and usb ?

[@fluxrenders]

I haven't tried using smartphones as wifi webcams. A guy in the server mentioned that he uses an android app called Iriun webcam.

[@rec-trick]

Did this method work for your program?@@fluxrenders

[@ShacharYavnieli]

Hi, while trying to export, it keeps writing: "Blender file does not exist! Did something go wrong in the `export_to_blender` call above?." - How would I know what the Blender executable path is? or what else should I do? Thank you.

[@fluxrenders]

The more common reason for that is not having the Rigify addon enabled in Blender. The Blender should be detected automatically or you can put it in the right side export tab. In case the problem persists, is better to go to the D i s c o r d server for a detailed troubleshooting.

[@highlandsdweller]

thank you very much for the good video

[@recaia]

i have 4 cam with irun webcam this best wireless way with ur promgram but i Can I attach a helmet to the face camera so I can get high quality facial movement details?

[@fluxrenders]

Hi. Yes you can use a helmet with a camera but you won't be able to process it with the current FreeMoCap version. In the discord server there is a video of a test using the experimental face blendshapes tool that is ran in addition to the main process. Also you can record your face and then process that with other tools like Metahuman Animator like I did in this example: kzbin.info/www/bejne/r6CmpqifnJuUmqssi=5rrgx-KZuqt-sgYD just beware to not block much your face with the camera so the other cameras can detect your face.

[@recaia]

@@fluxrenders ty

[@hata_0]

After I reassigned the animation to rigify in blender, it did not capture the center of mass information. How can I solve the problem?

[@fluxrenders]

In case you used the freemocap_adapter addon first function called "adjust_empties", it might removed the center of mass sphere mesh. Maybe you are referring to that. Is going to fixed in the next release of the addon. If it is something different then you can ask in the d i s c o r d server.

[@hata_0]

@@fluxrenders I thought there was something wrong with my operation. This community cannot live without seniors like you, and I am very grateful to you for this.

[@RadimilRamirez]

¿Solamente se pueden poner las cámaras en ese ángulo o se puede hacer un círculo con las cámaras?

[@fluxrenders]

Se pueden poner en cualquier configuración que quieras. Eso sí para la calibración es requisito que cada cámara tenga frames en los cuales ve el tablero de calibración simultáneamente con otra cámara. Por lo que si pones las cámaras en ángulos muy separados puede no cumplirse ese requisito y falle la calibración.

[@RadimilRamirez]

@@fluxrenders thanks❤️

[@RadimilRamirez]

@fluxrenders ¿Y que pasa si primero calibro las cámaras y luego las muevo? O ¿podría colocarme un frame en la espalda y así la cámara de atrás lo capture? ¿Es eso posible? Y te agradezco que te tomes el tiempo de responder 😊

[@fluxrenders]

@@RadimilRamirez Una vez grabas la calibración no se deben mover las cámaras. Si mueves las cámaras luego la reconstrucción 3D tendrá errores. Debe ser un único cuadro de calibración.

[@zackel-baz9555]

hey Flux Renders! this video helped me so much, but i was wondering if you could possibly help me getting this to work with prerecorded one-camera footage? theres no charuco board in the videos either...would this even be possible?

[@fluxrenders]

Glad that it helped! With one camera you will only get a 2D capture, no depth. You just process the video with the 3D triangulation disabled.

[@zackel-baz9555]

@@fluxrenders thanks so much again! I'll give this a go and let you know if I can get it working! Btw this video is by far the best one out there!!!

[@zackel-baz9555]

@@fluxrenders so I've checked off run 3D triangulation, but it still says" ::: processing parameters are not valid for recording status". I have my video as an mp4., but when I have calibrate from active recording enabled it doesn't work...am I missing something else?

[@zackel-baz9555]

@@fluxrenders I have a pretty niche project I'm working on, could I email you about it?

[@fluxrenders]

@@zackel-baz9555 you don't use a calibration file when doing single camera captures. Better to go to the FreeMoCap D i s c o r d server to troubleshoot this problems.

[@lilien_rig]

does the format of charuco_board is important, if yes what format is right ?

[@fluxrenders]

The idea is to use the same charuco board. You can find the image here: github.com/freemocap/freemocap/blob/main/freemocap/assets/charuco/charuco_board_image.png The size of the board will depend on how far the capture space is from the cameras.

[@AlphaEdz]

how to install the freemocap ? I just saw the github and can't find the download link

[@fluxrenders]

Right now the installation method is through terminal commands. freemocap.github.io/documentation/installation.html

[@AlphaEdz]

@@fluxrenders we must install python ?

[@NobodyNo-Limit]

Whan i try to run calibration i got this input in terminal { 2024-01-23 23:48:54,365 - freemocap.data_layer.recording_models.recording_info_model - INFO - No calibration file found with session name, checking for other calibration files in recording session } I do exact same way you doing, can you please help me i am no familiar of the coding

[@fluxrenders]

When you want to run (or create) a calibration you press the button named "Run calibration..." like 2:44. Once you have the calibration file, you can process the videos with the button "Process Motion Capture Videos" before making sure that the correct calibration file is loaded (can load with the "Load TOML" button like in 3:42). In case you are having more problems, is easier to troubleshoot in the server.

[@NobodyNo-Limit]

@@fluxrenders thank you for your reply Can you please provide installation brief tutorial ? Because I see many error in terminal

[@fluxrenders]

@@NobodyNo-Limit the installation steps are described in the project documentation. But also the easiest way to check the errors is in the disc Ord server

[@jockastafernandesabbade]

When I try to press the ' Process Motion Capture Videos' botton, it gives an error related to the absence of a file called "mediapipe2dData_numCams_numFrames_numTrackedPoints_pixelXY.npy". Anyone know how to solve this? I'll let the full terminal promt message in reply to this comment.

[@fluxrenders]

It could be multiple reasons to get that error. Better to ask in the D i s c o r d server and check the whole terminal output there.

[@ok-hc4he]

Can I use one camera

[@fluxrenders]

Yes, but you will get a 2D capture.

[@hardway777]

Please use natural voice..... 😒

[@fluxrenders]

My first language is not English, so I've been using an AI voice because my pronunciation is not perfect and it would take me more time to create the videos. I would try in the future to use my voice and see how it results. Thanks!

[@Frostshocker]

@@fluxrenders Just wanted to say a lot of English as a first language users also have terrible pronunciation (and they'd agree they do, English is a crazy language). Just roll with it and try your best, Subtitles might be a decent option.

[@fluxrenders]

@@Frostshocker Thanks for the comment! I will try that in a future video for sure. Probably record all at once and then record separately the parts that were not so understandable.