Rokoko Guide: Full performance capture (and face rigging) workflow for Daz3D characters

[@kineticmotive4466]

Thanks for giving our product a shoutout Rokoko Team! Btw, I’ll look into the hassle you had with C4D, there should be a way to automate that process. There’s a Python API so maybe I can make a script that generates Daz Morphs in C4D :)

[@RokokoMotion]

Yes! That would be amazing :) Such a cool and helpful product you have built!

[@radgecs]

hey buddy, Did you work on that script? or anyone else?

[@kineticmotive4466]

@@radgecs Yes, we did end up developing a plug-in within C4D. For more information, contact us at [e-mail]

[@KRGraphicsCG]

Great video as always, Sam. If you are savvy, you can take the models into ZBrush and sculpt the blendshapes... I wish Rokoko could come up with a custom solution to use with GoPro cameras...

[@RokokoMotion]

Thanks Kashif! ZBrush is a little out of my league, but stay tuned for updates about new recording methods for the face cap! ;)

[@KRGraphicsCG]

@@RokokoMotion Sweet...I've been using facial markers and the quality is OUTSTANDING... just need ways to track it

[@opafritzsche]

@@RokokoMotion Aaaaaaaaaaaaaaaaaaaand again. Why not with Facial Bones ? Why everytime f... Blendshapes? Dazi Rig have completly Cinematic Bone Face Rig. Why nt record this ? Blend shapes are really annoying

[@francescoauguad1567]

Wow Awesome Job

[@a.p.2018]

The snowman sold me.lol

[@JayIsABoss07501]

Sheesh I wish I can work for you guys I love your product

[@themightyflog]

Why don't you calibrate your system to automatically be linked to the 52 shapes in DAZ so it is ready to go? WE don't use MAYA and now you are making us buy a plugin and Maya? Why not your system with DAZ directly?

[@RokokoMotion]

I believe the Genesis 3 and 8 models from Daz have a bunch more than 52 blendshapes, and they don't correspond directly with the AR kit blendshapes, so this plugin is doing a lot of stuff behind the scenes to re-rig the models. This is also only one solution to using Daz models, there may very well be others that don't involve Maya - we're new to Daz so hopefully we'll be able to come up with some more streamlined workflows in the future!

[@themightyflog]

@@RokokoMotion I would love to see it. I mean imagine what can be accomplished if you guys worked with it. I mean but getting DAZ, ROKOKO and buying 2 programs that cost mrore than the suit per year...there has to be another way.

[@XanthanGumOfficial]

@@RokokoMotion Also this plugin has been retired and now is called Face Mojo, and they offer a Daz plugin offered here: www.laylo3d.com/product/face-mojo-for-daz-studio-maya-add-on/ Is this plugin compatible with the Rococo suit? I already have a Rococo suit and would like to couple this new plugin within Daz directly to capture animations in Daz for export to Unity. Any information would be helpful, love these tutorials!

[@andreadipa9969]

@@XanthanGumOfficial I've buy Face Mojo but I can't work, because in this tutorial they use the command "GUI" and in Face Mojo it doesn't work. Do you have some information about it?

[@techarchsefa]

Fantastic, Thank you!

[@melohugs1200]

i wish if you make a video on this but for unity instead of cinema4d

[@RokokoMotion]

Hi! Thanks a lot for the feedback. We shared it with our Content Team and we will take it into consideration for our next tutorials. If you are experiencing issues, you can always check out our Help&Community page, ask our users in our Facebook Group here facebook.com/groups/252832679299596 , or reach out to our Support Team at [e-mail].

[@lucasklubb]

Is there a way to go from Daz > Maya then BACK to Daz and render it there? Not too keen on rendering in anything else besides Daz.

[@jbrd7668]

I have done Daz3d to iClone and then back to Daz3d. But, it only worked for Genesis 2. Genesis 3 is very off and Genesis 8 is completely unusable.

[@CircusOfBedlam]

you said the workflow only works for gen8 and 3, but since this video came out we have g9. Does g9 work with this workflow?

[@ChrisBraibant]

Hi. It works on a Daz model in cinema 4D. But could you resend the model to Daz and use the full performance capture in Daz studio?

[@scorpion3d]

0:55 you will need : capture recording or FBX ? ( FBX is a proprietary file format developed by Kaydara and owned by Autodesk since 2006 )

[@JeffersonDonald]

Spitballing here, but if Daz3d had a script to implement the 52 blendshapes, then one could go from DazStudio directly to Rokoko Studio through an FBX export?

[@RokokoMotion]

That sounds like a great solution to shorten this workflow - we'll look into it / if you find one send it to us (and we'll post a scripting solution if we find one)!!

[@rafaelgarcia-ti4xb]

@@RokokoMotion I'd buy a suit soOo got dam fast

[@CGRTist]

Wow! While I appreciate this tutorial, how much money will this cost to implement? Maya license, C4D license, Rokoko suit, Rokoko software, plugin for correct facial shapes? Going to need a second job to fund this. It does look fantastic though.

[@RokokoMotion]

Yeah, it would definitely be pricey from the ground up - this was more for people who already might have access to some of these tools. Unfortunately most facial mocap solutions are still expensive (although that is not always the case). We've recently released another tutorial about the DAZ Bridge to Blender workflow where you can achieve the same end result that you see here but with less investment: kzbin.info/www/bejne/fnzSnXuuaJ2Xf6s

[@themightyflog]

Wow this won't work at all for the general DAZ user. Basically you are saying Bring into maya and then to C4d which is a big why? If you are in Maya why C4D and you are telling us that to get it to work with DAZ then we have to buy a 2,000 a year program and another 2K a year program. Got another route by chance?

[@RokokoMotion]

This workflow definitely isn't for everyone - this was mostly intended to show non Daz users what was available on the Daz Marketplace, and how you could convert what is there for use in other programs (we may change the name of the video to make that clearer). Unfortunately, Maya is required for this workflow because the plugin was built for Maya, but that is only requirement - I dipped into C4D because I don't like rendering in Maya :) However, once you have used the Maya plugin to convert the blendshapes, you can use the model in Maya, UE4, Unity, Blender etc... It looks like the best in-Daz workflow for face capture was going to be "Face Mojo," which was supposed to be released earlier this year, but is MIA. That might be the best workflow if you are looking for retargeting in Daz, but we'll have to wait until the release. Hope that helps!

[@MylonasFilms]

thankyou so much. I'm keen to buy ROKOKO suit soon. Looking for an investor to start a studio.

[@aryansoni1105]

Do you know which app are they using in their iPhone

[@YukiTheRookie]

@@aryansoni1105 Rokoko has its own app (free to download) but you do need extra Rokoko subscription to use it

[@naifox1]

Good Tutoiral nice !!!

[@TopDawg4]

Thank you!

[@arvyt5751]

Can you do a tutorial on how to rig any face for it to work with rokoko

[@RokokoMotion]

If you don't mind using Blender as a middle step, we can also recommend the FaceIt plugin, as it's a really nice way to generate the ARKit blendshapes. Just export to FBX from Daz3D and you can bring the model into Blender and go through the rigging from there.

[@themightyflog]

Fix the hands guys. It looks weird. I'm sure there is something you can offest. Other than that...awesome!

[@RokokoMotion]

Our Smartgloves will open for pre-orders within a month and start shipping this summer. Stay tuned here :) www.rokoko.com/en/smartgloves

[@andreadipa9969]

Hi guys! When I did CTR+C + CTR+V in the timeline it doesn't work. I mean, It copy in another order after that I ordinated in alphabetical order

[@manojparamahamsa6757]

I have this weird doubt ! Will the sensors and the hub will get damage while using for Action sequence where the talents falls in the ground may be in a little hard surface ?

[@RokokoMotion]

Our system is very robust and durable! We have multiple users doing fight and stunt sequences without any issues. We put a lot of effort into making the cables and cases extra robust and the textile also helps to put it all into place. So need to worry there (unless you are doing something completely out of the ordinary :)). We offer 1 year full warranty and a 30 day full money back guarantee. Best, Rokoko

[@manojparamahamsa6757]

Rokoko ok cool I will trust you

[@NiteshKumar-cs4wo]

But in pose morph, each pose morph have there own mesh which is hidden, i.e if the pose morph is 200 then there is 200 mesh which consume lots of RAM, if there is lots of character in one scene then its lots and lots of RAM. Is there any solution

[@RokokoMotion]

I'm not sure what you mean, exactly - we're still pretty new to Daz! Do you mean lots of characters in Daz Studio or in Maya / another animation software?

[@huyle234]

Hi, I only have Blender in my laptop. Do we have other way to import model and set up the key shapes without Maya?. Thank you!

[@williamlosik6316]

Hey ! I was wondering, how to do this but with adding cloth from daz in the process ?

[@mechasuprim2609]

What head-cam/phone holder is being used at the start of the video?

[@aryansoni1105]

Which app are using in your iPhone

[@jaykotak4874]

Waiting to buy the suit if it directly connects to Daz3d models....

[@RokokoMotion]

Hi Jay! Thanks for your comment. You can use mention techniques from the video, but unfortunately we do not have a Daz3d plugin yet. If you have more questions about the product, you may reach out to our product specialist: [e-mail] Have a wonderful day!

[@a.m.studios6126]

@@RokokoMotion "Yet". Does it mean a plugin might be developped ? Who knows how large is the DAZ users market...

[@runidjurhuus3937]

@@a.m.studios6126 hope hope 🙏🙏

[@nirvrock77]

I am pulling my hair out. I've been trying to get face capture to work for days now. I guess I'm an idiot. I downloaded Maya on a free trial and spent $200 on the plugin. I can't get the plugin to work in maya. I just get the following error when I try to load the plugin: // Error: file: C:/Program Files/Autodesk/Maya2022/scripts/others/pluginWin.mel line 316: (FaceMojo) To be honest, Rokoko, you guys make this stuff seem like it's plug and play but the hoops to be jumped through to use the most basic features is killing me. I don't think I'm that retarded, but this whole process is really making me question my IQ as well as my sanity. This shouldn't be as hard as it is! I don't even know who or where to reach out to somebody. This Laylo guy is super unresponsive. I find this process it's easy to spend a bunch of money in search of an easy workflow, and it just leads to headaches and wasted money. At least I didn't commit to a $1700 a year Maya subscription. Can anyone help or guide me someplace?

[@andreadipa9969]

I've solve today this. Do you need help?

[@LarsLeonhard]

How difficult is it to skip Maya and go straight to Blender and create the 52 blend shapes there in your own model?

[@RokokoMotion]

Hi Lars - Daz3D actually has a new workflow where you can create most of the blendshapes natively, for free, when you export out. Check out our new character creation tutorial, hopefully that will get you started on the right track! kzbin.info/www/bejne/oYSZnKGYgbami9E

[@LarsLeonhard]

​@@RokokoMotion Hi - I am not interested in third party models, but would like to make my own. I am still new to all this though and still learning Blender. Hopefully I can find a way to do that, if I ever buy the suit and facial recognition software. My interest is piqued though. Do you have a list of improvements over the old suit? I saw MOON's video on KZbin review about the previous edition (three months ago) and he mentioned a lot of issues with his.

[@wakeup2.369]

will the face reader software work on Android?

[@RokokoMotion]

We're working on a solution for Android that will come out late this year.

[@sveinolaskaaret8598]

Hi and thanks for the video. We have bought your 3D-suit and using the AppleX-phone for face capture, and working to get that working with UE4. We have followed you in this video, but we do not have C4D. We are wondering: Why do you need to go to C4D and Mixamo, once you have exported it from Maya (using the LayLo-plugin). Can we use the finished exported character directly in UE4? Or do we have to do something simial in UE4, as you have done in C4D and Mixamo? Thanks in advance for your answer.

[@RokokoMotion]

Hello and thanks for reaching out! 😄 You can do it directly in UE4, but mind that it doesn't retarget the faces. Also make sure that your characters are in the same rest pose before retargeting. Cheers!

[@sveinolaskaaret8598]

@@RokokoMotion ok, thanks for your answer! But isn't that the job of the LayLo-plugin? (Sorry, we are just getting familiar with this "3D-world" :-)

[@RokokoMotion]

@@sveinolaskaaret8598 Hey Svein - you can use an exported FBX from Maya directly in UE4, but you might run into some issues if you accidentally export the Laylo GUI with the character - it may work perfectly, we just haven't tried doing that yet. If you run into issues doing that, would recommend bringing the Maya export into Blender, making sure that you are just getting the character with the Blendshapes, and then taking it into UE4.

[@sveinolaskaaret8598]

@@RokokoMotion Ok, thank you for your answer!

[@akshayrathod4312]

Can you do a Same model setup for Unreal Engine please .

[@RokokoMotion]

Any model that you convert using this process can then be used in Unreal, Unity, C4D - any program that uses pose morphs - the FBX is bundled with them when you export out of Maya. Unfortunately, this process of converting Daz blendshapes into standard AR Kit blendshapes is only possible through the Laylo 3D plugin, which is only available for Maya. There is probably a simple script one could write to do the same sort of thing in Unreal, but for now we don't have a workflow that is Unreal only.

[@scarletworeyellow]

@@RokokoMotion Just to add to what he was asking you go through a lot of steps in Cinema 4D in order for everything to work properly, do we imitate these steps for Unreal? Some of the tools you accessed in C4D I do not see in UE4. Thanks!

[@HeLIEl]

Can this work without Maya? There’s another guy who made a facial motion software where you don’t need Maya

[@RokokoMotion]

Hi! With our facial motion capture solution, you can visualize your data in Rokoko Studio and either export your data in FBX, BVH or CSV, or livestream the data in realtime into your custom character through our native plugins, not only Maya. You can find more info about our Face Capture add-on here: https//www.rokoko.com/products/face-capture. Have a great day!

[@HeLIEl]

@@RokokoMotion Ok thanks a lot

[@mariagudjohnsen9266]

how do I keep the eyelashes on?

[@RokokoMotion]

To keep the eyelashes on, you want to select both the body mesh and the eyelashes mesh when you hit "Set-Up GUI" in the Laylo Face Cap X menu. Thanks for watching!

[@andreizhitkov5012]

Why not share as much data used in this demo as possible without stepping on anybody's copyrights, of course?

[@RokokoMotion]

Hey Andrei - these were all purchased models from Daz, so we can't share them unfortunately. If you want to play around with some sample files, we include some with our Rokoko Studio software that you can download for free - I believe we are adding face capture examples in the next version as well. Also, you can download over 150 free mocap assets from the Motion Library - which can be accessed from our Rokoko Studio software as well. Hope that answers your question!

[@andreizhitkov5012]

@@RokokoMotion Great, thank you so much!

[@themightyflog]

Can you show us how to do this for Blender? Blender is free. :)

[@RokokoMotion]

We have this guide on how to use our livestreaming plugin for Blender if that helps :) kzbin.info/www/bejne/gnrJhWyBlM-WiNE

[@themightyflog]

@@RokokoMotion thats cool but no...this is more for the face part. Daz to Blender and bring in the FACE capture capabilites. The mocap part is easy.

[@geroldwaefler9485]

But why Maya?????????? Maya and Autodesk are way to expensive and Companies like Autodesk and only Subscriptioncompanies we need to boykot!!!!!!!