Build a low-cost motion capture system

My experience in building a homemade low-cost motion capture system (best viewed in full-screen, full-hd).

[@OPUee]

thanks for the great introduction! Some colleagues and me are planning to develop a mocap system at university and this is a great start to get into the subject.

[@nishantgv6]

awesome work bro... thanks for bringing this to people

[@stephanegli3160]

Do you have any sample footage of what it looks like once your done recording and editing some of your own footage?

[@RaigyoEcU]

yes! finally i have my own MoCap after a year working on it, thank you so much, you and "Multiple View Geometry" book, both were completely helpful

[@thomascarstens2729]

Hi, can I dm you for a few questions about your process? I'm not sure where to start, what equipment to use

[@RaigyoEcU]

@@thomascarstens2729 yeah sure, i guess i could help you, by the way i'm not a pro at this, it took me so long that maybe it was better to just buy an already existing moCap suit

[@VisibleMRJ]

@@thomascarstens2729 you guys should start a discord server

[@fuivo3434]

@@RaigyoEcU Could you please tell me which software did you use for it?

[@RaigyoEcU]

Thanks, this was very helpful and professional, by the way perhaps You may know of there is any hybrid mocap? A mix between cameras and IMUs

[@GLITCHYSKY228]

such a great video, but can we use other camera systems? (ones that could sync obviously, maybe even security cameras or something?) I also think you should release some documentation on this too :D

[@nooblike7332]

hi, i was wondering how you did calculate the fundamental matrix what algorithm did you use to capture marker 2d positions in image ? did u do it manually ? and how did you convert from local coordinate system of the reference camera to the world coordinate system ? any alogirthm for that ?

[@michaeldunne5887]

Great Video ! Quick question tho..You mention a particular model of PS3 eye camera, can you be specific about which model to look for when buying online? Thanks

[@marknabil9978]

Do you think it could track a Quadcopter ? Its response time is quite good for such a purpose?

[@gregklein303]

Hey, Could you point me in the right direction for a good resource for bundle adjustment algorithm? I am having trouble getting the BA camera poses

[@ingenimundo]

Excelente Trabajo!

[@rgqric]

Yhea!! Great job stick it to the man.

[@ltd5352]

Complimenti Gran bel video! Dalla lingua di Windows ho notato che sei italiano. Io ho iniziato da poco con blender, e devo dire che mi piace molto. In ogni caso tienici aggiornati su questo progetto, veramente interessantissimo!

[@MINEventor]

Does this system work with VR and then play steam games?

[@GraysMediaLLC]

Nice work.

[@Fightarcade]

I love this video, and I DIY my own Motion Capture suit!

[@dowk100]

Which bundle adjustment library did you use?

[@AlmightyGauss]

Very informative video!

[@jordanafriend]

Which models or S/N are the good cameras? I am looking on ebay to find which are the best. Also could someone also use a Kinect with these cameras, or would the sensors conflict with each other? Is it possible to split the synchronization in half?

[@MrWhitenoise404]

awesome!

[@10xXxtailedxXxdemon]

Can you provide the source code you developed for this project? Thank you.

[@BayareaJay]

Awesome!

[@filipelucciano]

Can you post the sync program?

[@oylar3619]

thank you

[@akshaggarwal4092]

Can I use the cameras without ir filter or I have to choose with filter and remove manuy

[@mohammadzakeri5549]

very amazing!!! thanks for this video! Is the development of this project is finished?

[@agpxnet]

Thanks. Not yet, I do my experiments in my spare time.

[@fabiansuter5795]

I liked the music. But it made your voice a little hard to understand. ;)

[@alikocak3916]

you made my day :)

[@sewerynmalczewski7383]

hey. I want to make a mini mocap studio and I've a question. 1 What gives better quality Kinect ? or 6 cameras ? 2 Is it possible to record fingers too ?

[@marcdraco2189]

Both have their pros and cons. Kinect is probably cheaper since you don't have to look for the right one... Try iPiSoft.com for a pre-made solution that's guaranteed to work (in the right conditions) and won't break your bank.

[@xBattleSheep]

Can you make a tutorial how to build the clock?

[@beecee793]

I set up a basic Mocap system a couple years ago with an Xbox One Kinect, some adapters, and the Brekel software, but it (obviously) had some serious limitations since it only had the one camera. I'm looking to set up a newer, better one and stumbled on your video. I also own a Vive, but not extra trackers, so I don't know if I can use those Lighthouses in a helpful way or not. Are there any options for under 1,000USD today for full body Mocap? It doesn't need to be perfect by any means, as I am a 3D guy and can clean up any animation myself, and I am only using it for personal projects anyway. Project Neuron looks interesting, but it seems their prices have gone up and I can't find much documentation about it, and what I am finding is still dated in 2015 which makes me leery.

[@nathancreates]

Enflux www.getenflux.com/pages/product

[@hunkbol6154]

man man i made full setup that you say,but which software i use

[@cinamonbay]

Hi, really great work you have done, i am trying to do something like this in my students grad work. I am on Boundle Adjustment step now. Can you please help with code of Boundle Adjustment method? I’l be really appreciate your!)

[@tawnkramer]

It looks like you have made enough progress to start a nice open source project. I think many would help you finish. I would.

[@ingenimundo]

What is the name of the marker material?

[@ChinyONE]

Quite impressive, well done.

[@DorseyJackson]

What's the total cost?

[@CaptinJetmouse99]

84 euro 94 usd

[@AlexBradley123]

What kind of software are you use? Amazing video!

[@agpxnet]

Thanks, it's a software written by me.

[@AlexBradley123]

Mocap software?! Well... It's really cool.)

[@iscoffeetho]

agpxnet make a shortened link to a download

[@GamingDanDev]

This is amazing I can't wait for whats next. Are you going to make a detailed tutorial on how to setup the raspberry pi and the modification of the ps3 eye cameras? did you wrote the mocap sofware or is it something anybody can get?

[@agpxnet]

For the tutorials, I do experiments in my spare time and so I haven't much time, maybe in future. I wrote the mocap software by myself, using OpenCV as supporting library. Sorry, but for now I have no plan to release the source code.

[@JONIMASAKIS]

Then why the hell you make a tutorial on how to build a mocap capture system that can be used only for YOUR software, which you will never release? Cant get it...

[@JONIMASAKIS]

Hmmm

[@harsh4537]

hey i didnt understand what software you used for this, in the video you said its a software called MoCap but there is no software like that so can you tell me what you used please

[@agpxnet]

It's a software made by me.

[@vinsolkarhak2441]

Can i use other software?

[@furry-max]

@@agpxnet Can you share this program of yours? Or what are similar programs, if any?

[@mr.a1503]

excellent

[@ivancuetowicq]

oh ok, ipi recorder is only to record the cameras at the same time, i mean they are already synchornized and you export one video with special extension for ipi studio. It`s in ipi studio where you calibrate the cameras ad give you the xml file so when you record the action in ipi recorder again, and the open this action(performance)video in ipi studio, you import the xml file that has the cameras calibration

[@agpxnet]

I don't think that ipi recorder implements any kind of shutter synchronization, simply because it cannot be done via software. However, ipi studio doesn't need it at all (or, at least, is not so critical), because the technique used is completely different. Ipi try to reconstruct the 3D volume of the moving actor (and so is markerless) and then try to fit the skeleton inside it, analyzing the 3D voxel space. However to do it properly, it has to remove the background and this is not an easy task to accomplish with a camera like the PS3 eye, that has a lot of noise. It works by far better, if you use a device like the Microsoft Kinect that, at the end of story, is just a realtime 3D-scanner.

[@Mirash5151]

hey can you make this video one more time so we understand it a little bit more.

[@terencetseng]

Nice work! Can I contract you for some detail?

[@GoblinGamesLtd]

It's a very interesting video, but of course a bit redundant these days. Ikinema Orion is a very decent setup for not much money. Smartsuit Pro even better.

[@muhameedjalal258]

I can not find software online

[@DanSpartan177]

Is it possible if i use clay as the actor?

[@terencetseng]

Can I purchase your system? Can talk detail?

[@ivancuetowicq]

by the way you can calibrate up to 8 cameras, i think more im not sure, with ipi recorder which is a free aplication and it gives you an xml file. another thing, in your video you mention a lot that photos and info abput topics you talk about are on the internet, can you please post the links? thanks in adnvance. i suggest you to check ipi recorder

[@agpxnet]

About the synchronization stuff, you can found useful info here: nuigroup.com/forums/viewthread/9628/ nuigroup.com/?ACT=28&fid=34&aid=7360_7LpLFlSc5HWnVlvjHemf www.red.pe.org.pl/articles/2014/4/40.pdf codelaboratories.com/forums/viewthread/84/

[@GiotaNikoyhandmadestuffs]

Ivan Cueto ffdgiŕhhxģjb. nxxdhhtfoygghjjkjjjjjjjuuiùuuiiuìiijlkijkkkkkopyyffyuuuiiii990opoploooopocfhhjjijkkòo opioid until just jbjjjjjhhnh have just hhhhbjjnjğhhģğjhj just uucp

[@Bhavin_Lathia]

do we need only one camera for it

[@agpxnet]

One camera is not enough. To reconstruct the 3D position of a marker, we need at least two cameras (this is the reason why we have 2 eyes :). But due to occlusion issues, it's better to have at least 6 cameras.

[@Bhavin_Lathia]

agpxnet but what about the new perception neuron set which is for about 1600$ is that a good one links below please give a advice neuronmocap.com/

[@thejetshowlive]

Thats the one Im going to buy next month. Clearly agpxnet is has no plans of finishing, sharing or selling. why have all the hardware when the Neuron system is clean and is compatible with other software systems.

[@DNSStudioAnimation]

Use camera kinect for windows is better. Like kinect xbox 360 or Xbox one.

[@figo951159]

can u use cameras for ps4?

[@agpxnet]

The PS3 camera has been studied by the community and so is known how it works and how to synchronize them. In addition there are drivers available for PC (Windows cannot recognize it automatically). I don't know about the PS4 cameras, but I choose PS3 cameras also because they are cheap. Probably could be better build a camera using arduino, performing the detection of the markers in the device itself and send only the marker position through the network (this will enable a higher framerate).

[@furry-max]

Can you sell this software?

[@etharyousof4007]

How many PCs do you use?

[@agpxnet]

A single notebook (core I7), but unfortunately the USB controller is unable to record at 60 FPS, so I have reduced the FPS to 30.

[@etharyousof4007]

Great! because i'm working on two PS3 cameras connected to one PC, and i'm using openCV for calibration and reconstruction, but i found that openCV capture function can't detect more than one PS3 camera! i think i need to use something like libusb-win32 to manage that instead of using video Capture function of openCV .

[@Nick-su4jg]

Are you sure that usb controller is a bottleneck here? I have similar issues when max fps was not more than 32fps when using 6 cameras, but after i tweak some code and turn it into multithreaded madness now it can capture with stable 60fps and also save each image as jpg, this saves a lot of disk space. I reduced data rate by capturing only raw bayer data from camera, then I run fast opencv debayer to turn it into grayscale. If you interested in this please feel free to contact me at guitarsessionmusic@gmail dot com i’ll send you my build for you to test.

[@marcinjezioro9730]

i did simple motion capture program with two webcams , i have it on youtube

[@marcinjezioro9730]

@John Aidan Logarta kzbin.info/www/bejne/fX6qiJasf56Zbqc

[@iamapie13]

I was able to get 4 pseye cameras for 20 bucks, they were selling them for 5 dollars each second-handed

[@khurramsa5133]

NO WAY YOUR IDEA I CANT GET ..SORRY

[@timhedden2369]

And this is the result: kzbin.info/www/bejne/mZWzlaKrmsZ2Y5I

[@fennecinspace]

or just use a kinect

[@horusxfilms3413]

Kinect has bad quality

[@userseveneleven]

The kinect costs bout' 70 euros for a single piece of hardware, that works, but isn't the greatest. According to this video a ps3 camera is around 14 euros, so you buy 5 of them and get better results.

[@JeanRodo]

dislike for not releasing the software, source or at least sell it dude you're sitting on a goldmine.

[@agpxnet]

Personally, I think it is a bit exaggerated, I do not think there are so many people interested in the subject. However, let me clarify that the system is far from be a ready product, usable, robust and generally sellable. There are many aspects that need to be improved and completed. The topic is quite complex, the video does not enter into the details of certain issues. The cameras of the PS3 (those with removable IR cut filter) are hard to find (out of production) and make the changes described (especially for synchronization) is less simple than it seems. It is necessary to utilize a better hardware, with a frame rate much higher in order to make the tracking more robust. The algorithms need to be improved (or even replaced with others, more robust). Again, the project is far from be complete and currently has more an educational value.

[@JeanRodo]

agpxnet me personally I thought Star Wars Rogue One was an excellent movie.

[@JeanRodo]

agpxnet ignore the last comment it was for another video, anyway I can see you're reasoning and everything but if it's more for educational value then that's even more reason to get it out there so other people can tinker with it and find hardware available now that can work with it and further build upon what you've made.

[@agpxnet]

May I ask a question? There are quite affordable motion capture system around, like the Perception Neuron (just to name one) that have no space requirements (this is a big deal!) because it uses no cameras, but active markers able to compute their position and orientation in space. Why make one by yourself and don't buy one ready?

[@JeanRodo]

agpxnet Sorry I didn't make it clear but I was asking because its something nice to tinker with and im sure other people would too.

[@crazyAlexander23]

never ever use hot glue on circuit boards

[@RobertJacobson]

There is absolutely nothing wrong with using hot glue on circuit boards. "Hot glue" is just molten EVA plastic. The temperature of hot glue is an order of magnitude lower than typical thermals used to solder parts to the board. Hot glue has been a mainstay in the DIY electronics scene since forever. Manufacturers use similar industrial adhesives, colloquially called hot snot, in manufacturing electronic devices.

[@beat91sms]

low cost my ass, everything will cost you at least 300, for 40 you can get xbox kinnect, put it on a computer record video to blender and bam done, or even with your phone if you dont have kinnect, so BAM very low cost 0 cost if you have a phone

[@Shubham-fk5oq]

all we can do with one device: Kinect its cheap and doesn't required any paid shit!!!