LabNote: MPU6050 Based IMU Arm Motion Capture + Blender Realtime Sync

▹ Come across the idea someday that if every joint's world rotation angle can be captured, combined with the dimension of the length of each bone, we can in fact compute the relative gesture of the entire armature, as well as the relative positional information!
▹ If this expands well to whole body, then perhaps we just need a single tracker for getting the world location?
▹ Code: github.com/T-K...

[@IamSH1VA]

Very cool & seems fast 👍🙌👏🏻👏🏻 This is the best mocap project I have seen 🤯

[@gokz9079]

Very cool video, I'm doing this exact thing for my college project and I wanted to ask how I could get started. I feel very overwhelmed and dont know what combination of sensors and arduino to use, what software to output the data visually etc. Any help would save my life!

[@-T.K.-]

I would recommend start from simple, follow the Arduino tutorials. If Adafruit is accessible to you, they provide decent tutorial and docs for their products.

[@DudimusReviews]

Great project! Is it possible to translate those coordinates into angles that servos can use to capture the arm motion? I can't seem to make the MPU6050 understand the difference when my arm is stretched out to the side versus when it's stretched out front since both of these positions only have acceleration in the Z-direction.

[@abelgetachew838]

its a great project and can you give us more details on the project about the wiring , the code is it only c++ or other ...

[@prateeksarkar8573]

Compilation error: 'class MPU6050' has no member named 'dmpInitialize'; did you mean 'initialize'? this is the error i get. I am running on esp8266

[@1Chitus]

Awesome!!!

[@chriscarr2488]

Great project! Long story short, working on a very important project. Trying to implement a boxing device for positioning, in this case, the shoulders and wrist will be connected together. The whole purpose will be to have the guard up, my question is it is possible to connect 2 sensors in 1? 2 sensors acting like 1 for each hand. Really need help on this please help us on this!

[@-T.K.-]

I don't quite understand what you mean by "two sensors acting like 1". In my case here, I'm putting one on upper arm and one on lower arm. To get the two joint positions, I would do a coordinate transform in blender.

[@gambikules]

@@-T.K.- Hello you use rasberi pi or anduino ?

[@-T.K.-]

@@gambikules Arduino, in particular the nRF52840 Itsy Bitsy by Adafruit

[@-T.K.-]

@@gambikules Arduino, in particular the nRF52840 Itsy Bitsy by Adafruit

[@yj1035]

Hi, I am currently doing a similar project, I have export a rigged model to three.js and tried to set quaternion value from sensor to the mode, but the arm's rotation is totally different than expected...is there anything need to be noticed during calibration and rig. do we need extra calculation to convert sensor rotation data to blender's one (because I have calibrate the sensor data from local coordinate system to human body coordinate)

[@-T.K.-]

@@yj1035 hmmm, I would recommend checking all the reference frame etc. to see if they match up

[@yj1035]

@@-T.K.- emm, what do you mean by reference frame. and not sure when we rig the character, do we need to care about the local coordinate system's direction of bone

[@ДанилСтарновский-д7н]

Большое спасибо очень крутой проект

[@corneliuswelling3210]

Hello, your Project looks very nice and I am trying to do it myself. I get the right data from my sensor in the Arduino IDE Serial Monitor and it changes when moving the sensors. However, when using your code with blender I only get the same starting value over and over again and it does not update after moving the sensors. Although the data changes in the Arduino IDE. So my animation only changes her arm position once. Do you know how I could fix this?

[@-T.K.-]

Can u try to add some print() statements in Python loop to first verify that the loop is actually running?

[@endrisbuzhala4914]

I have kind of the same problem...

[@corneliuswelling3210]

@@-T.K.- I added "print("Test")" in line 40 and the output seems weird. Output: All started. Test : Incorrect packet received. Test Test : Incorrect packet received. Test [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] Test [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] [`0.79`, `-0.60`, `0.06`, `-0.10`, `0.80`, `-0.60`, `0.07`, `0.07`, ``] …

[@-T.K.-]

@@corneliuswelling3210 hmmmm... I'll try to polish the project some time...

[@corneliuswelling3210]

@@-T.K.- okay thank you very much. Pls let me know if you find a solution :D.

[@sanjeevpatelsanjeevpatel4295]

Hello sir your project is too good I want to make this type of project for my college i tried your code and your file . Code is run perfectly but there is a problem in body there is no movement so please can you solve out my this problem

[@-T.K.-]

It's hard to debug with comment section... I'm just going to share my procedure: 1. Did you get reading from serial port? 2. Is the data from serial port correctly framed? 3. Is the data converted to correct angles? 4. Is blender responding to those angle changes commands? Can have a separate piece to test e.g. set joint to a certain fixed value 5. If it still don't work, I will just rewrite 😆

[@sanjeevpatelsanjeevpatel4295]

It's working now thank you so much

[@sanjeevpatelsanjeevpatel4295]

Can I expand this to whole body?

[@ahmedbenhassen2007]

@@sanjeevpatelsanjeevpatel4295 im having the same problem .Can you please share with us what modifications you have made ?

[@zasranets-vonu4ih]

Очень круто!

[@TheGnipahellir]

... thats a beautiful hand you have)

[@Zumito]

you have the MPU 6050 with 6 or 9 axis?

[@-T.K.-]

6

[@Zumito]

@@-T.K.- wow that's incredible, the result is perfect and you only use 6 axis, i'm going to try this

[@qntkk_]

Awesome!!! can you make a python based one for raspberry pi?

[@-T.K.-]

It will be pretty similar ---- the complex part is on Blender's side for serial communication

[@qntkk_]

@@-T.K.- i dont really need it for blender, just need the xyz values printed in console on the rpi also does this code measure world position? i want to have only one mpu6050 that i can move around anywhere and see how many meters it is far from starting position

[@TinNguyen-wx4fq]

good job

[@ztdeng]

您能提供 Python 数据与 Blender 通信的视频吗？

[@sinawkey]

How is the realtime sinc so fast?

[@maksimmaksim7872]

У меня blender зависает когда начинают данные поступать

[@SaideepPowar]

hello sir please provide connection diagram?

[@Royerbin]

The previous code worked wonderfully, boss, can you resend it to me or upload it to git, please, please .blend pls

[@-T.K.-]

Perhaps try this one? github.com/T-K-233/Dual-MPU6050-Motion-Sync

[@Royerbin]

@@-T.K.- thank you very much boss you are a crack....like

[@madcatdev]

what Ardruino are you using here ?

[@-T.K.-]

nRF52840. I use it simply because I have plenty of them sitting in my bin. Any small Arduino board would work here in the USB tethered setting.

[@lapateja7325]

.blend is empty :S Can you update on github please?

[@-T.K.-]

Shouldn't be... I've tried to download zip from GitHub and open it, looks good on my end. I'm using Blender 3.0.1, if that helps

[@lapateja7325]

@@-T.K.- It was that!, a very clean code. Congratulations!! Thank you!!

[@lapateja7325]

I receive a UTF8 error when the script is executed :S

[@-T.K.-]

@@lapateja7325 Oops, that's on Python side? That's weird.... I would suspect that the bone names are resulting this error (vrm model is imported from Vroid, which is a Japanese software). Perhaps try setting system language setting to support japanese or chinese?

[@-T.K.-]

@@lapateja7325 Something like this www.digitalcitizen.life/changing-display-language-used-non-unicode-programs/

[@balamurugan.k2981]

Script run perfectly but no movement in body and no any data in console

[@-T.K.-]

It's hard to debug in the comment section... But - have you programmed the microcontroller correctly? - Try opening the serial monitor and first check the reading there?

[@balamurugan.k2981]

It show value to but blender not

[@-T.K.-]

@@balamurugan.k2981 is baudrate set to the correct value?

[@sehntai2477]

Now play Bad Apple on it