Text To 3D Character Animation With AI • MOOTION

[@user-nh2ul8ii2w]

Looks so cool! Guess it can be the most genius motion generation product I ever used.

[@mmilamix]

thank you so much - just tested for unity works well! hope that will speed up my cartoons

[@LearningZoneLanguages]

Thank you for showing us. I can't wait to try it. I think we are facing a very interesting era.

[@ryebread1003]

I took a pass at a text->3D during the April AI rush. Ultimately the papers I was following just came to be too intensive for floating point operations (literally 100gigs of vram for a potted plant) Taking a whack at it again now that I have some compute time banked. Keep it up this is awesome work.

[@braveitor]

This is very good stuff. It's amazing how these tools have evolved in such a little time. Impressive. Good job and thanks!

[@casksandy]

Amazing! I look forward to us being able to generate our own characters. Maybe even uploading a JPEG of a character and the character is automatically animated.

[@aa-xn5hc]

Amazing, especially the second half

[@PhoenixAi479]

amazing video!.I was wondering do you have a workflow to download? and I'm wondering how did you add the openpose rig as video for comfyui?.I subscribed!. and add animation from the openpose video to comfyui.

[@796r]

Question, do you think this could apply to action figures and stop motion?

[@hairgalleryusa8corp401]

How to make character bigger in photo

[@mehdiarjmandi5280]

nice

[@aa-xn5hc]

Wow!!!! Amazing topic this

[@AIAnimationStudio]

Thanks. Yeah it's going to be interesting to see this space develop.

[@user-kc1ij1fl8v]

Looks amazing and fun

[@PieterLaroy]

Fantastic video! Thx!

[@AIAnimationStudio]

Cheers, glad you liked it.

[@lucilehujeux6934]

Hello, I’m new here, and I would ask you what is your computer specification? Thank you, your work help me a lot !

[@AIAnimationStudio]

Thanks. My local computer is an M1 Max, Mac Book Pro. 32GB of ram. But for running ComfyUI. which I was running locally. I'm now running via the cloud. Renting 2 x 4090 Nvidia GPU's. I'll post a video about this approach soon.

[@dennisnewsky7890]

Hello! Thank you for sharing such valuable information with us! You have the best quality animation content I've ever seen! Tell me, is there already a way to obtain 3D models from 2D images?

[@AIAnimationStudio]

Hi, thanks so much. Yeah you can do 2D to 3D models using something list CSM.ai . Plus there are a few others on the scene now too. Plus a slightly simpler but free (or paid if you need to skip the queue) approach is to use Dream Gaussian via Hugging Face.

[@manifixdesigns]

​@@AIAnimationStudiohow to do this animation in a1111

[@NobbySavage]

Hi, is there a a way to lip sync 3D models? Or can we use 2D lip syncing tools available and apply them as a texture to a 3D model?

[@AIAnimationStudio]

Hi, whilst you can animate the mouth of the openpose rig it does'nt translate across as overly great lip sync. There are some pretty complex 3D head rig, lip sync approaches which are probably not what you're after. An alternative approach would be to pass your rendered footage of a character as a video clip through Lalamu Studio (demo) or synclabs.so where it attempts to add animated mouths to your characters aligned to your audio file. The results can be pretty good particularly if you're character isn't too stylised and has typicaly human proportions. For more stylised options you may have to go a bit more old school, and create the mouth shapes and animate manually in your 3D package.

[@NobbySavage]

@@AIAnimationStudio thanks, will check that out

[@AlgorithimError]

can you please do the video on using auto rig pro to retarget the animation fbx

[@AIAnimationStudio]

Hey. I actually covered retargeting the FBX animation to another Character rig here last week. Hope it helps. kzbin.info/www/bejne/eJLFnJSKe5KerZIsi=Q46S0nSCFKbryxNk&t=247

[@DoooBeeDoooBeeDooo]

Does this program allow to upload your own 3d characters?, and generate animations of that character that only you will have access to?

[@AIAnimationStudio]

Not yet, but maybe they'll add that in the future. So the approach with the FBX file download and transferring the animation to your own character is the best approach at the moment. In other platforms WonderStudio have made setting up your own character to use on their platform a bit easier. (Their platform turns live-action footage into, motion capture and applies it to a 3D model, plus matches the scene lighting).

[@VaiseFuderFDPuta]

Genial!

[@aaagaming2023]

How capable would this workflow be of making a virtual music video of a rapper with as lifelike a look as possible?

[@AIAnimationStudio]

Describe the rapper and I might give it a go on the weekend. If it needs to be a specific rapper some new face swapper approaches are working really well. Good quality lip sync though is still tricky, but different approaches are improving all the time.

[@aaagaming2023]

@@AIAnimationStudio Yeah there are some decent quality lipsync colabs up. The other option is rapping it with UE5 face animator and deepfaking over the top of the metahuman with Deepface Live. The main thing Im looking for is if this Mootion can do lifelike rapping animation or if its still too generic. How does it handle fingers for example. Currently I have high quality Tupac and Eminem rap tracks made with AI voice clones. Just looking for the most realistic method to put them in music videos. So far however nothing has come close to licensed stock footage with lipsync and DFL dfm deepfaked over the top.

[@AIAnimationStudio]

@@aaagaming2023 . Yeah the whole Metahuman approach is definitely the bets way I've seen for really good lipsync. Otherwise you're a bit restricted with Sad Talker Colabs/Discord, or D-iD etc and trying to pair it up with the body aniamation. I defintely think Mootion could make some cool rap moves. Plus (I think) there is some finger animation... but you might want more refined control, though it's the only text to 3D animation tool I've seen with decent results so far. Otherwise you could explore the Vid 2 vid wokrflows with Comfy UI, where it cna create the OpenPose/Controlnets from the video source and use that to drive the animation. Hmmm.... potential to get it all working... just needs some time and a bit of R&D to find a way that does'nt take too long.

[@ghostsquadme]

As requested: Hi! 👋

[@paygame01]

Gimana cara nya

[@ze1st_]

9:42 I was here only for this thing and you didnt do it

[@AIAnimationStudio]

:-) ... here you go ... think this is what you're looking for.. kzbin.info/www/bejne/eJLFnJSKe5KerZIsi=LQoGydnCOlbq_QSL&t=335

[@ze1st_]

@@AIAnimationStudio thx

[@sounder7564]

low sound

[@AIAnimationStudio]

Yeah.. annoyingly.. rushed the export and didn't double check the Db levels. 🤦‍♂

[@zck2020]

In the future anyone will be able to make today's blockbuster movies.

[@j.x.7655]

AI will generate different movies for everyone.

[@ryansaper]

These examples look awful. I was expecting something like the prompt/ run and fall down after tripping on untied shoelaces and then sitting on the ground holding your toes in pain. Maybe in a few years the tech will be there

[@AIAnimationStudio]

It's already improved quite a bit since this video was first made at launch. You can now do multiple actions, one after another in a single prompt. Not sure if it would deliver on your described prompt... but maybe... It can now also do motion capture using your own recorded video to drive the 3D animation. Plus it also now lets you generate Stable Diffision (animateDiff style) animations directly from the Mootion generation as well as the FBX download. No doubt there's tonnes of areas that can and will be improved but as one of the first (if not the first) text to 3D animaton generations they're leading the way. So many new tools coming out and improving... it's a lot to keep up with 😅

[@turnerfrank-pw3bi]

Dude, the feet of the model are all over the place. What a disappointment.

[@reeanimationgaming1034]

Yes, How dare they not make it perfect on the first try

[@AIAnimationStudio]

As with all this AI stuff, It's only going to get better. Plus lots of the other 'Tea making' mootion generations had static feet.

[@oDannyMitchello]

Trivial fix to constraint feet bones.