AMUSE: Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion (CVPR 2024)

[@LucasMatuszewski]

Do you know if it works live? Is inference fast enough to be used for a live 3D AI Avatar? Could you recommend other models usable for this purpose? I'm working with Nvidia Audio2Face currently, the best I've found so far.

[@voxyloids8723]

Don't think so

[@youtou252]

this just looks like random hand motion tbh

[@LucasMatuszewski]

Yeah, same with Nvidia Omniverse audio2gesture, looks too random to be usable, but its beter every year, so we are close to usable 3d audio to animation. Nvidia audio2face is already quite good.