[SIGGRAPH 2023] Listen, denoise, action! Audio-driven motion synthesis with diffusion models

This video presents our SIGGRAPH 2023 paper on audio-driven motion synthesis using diffusion models. Given audio and (optionally) a desired style, our models generate dancing or full-body gesticulation with top-of-the-line motion quality. The style expression can be made more or less pronounced, and we also demonstrate a new way to blend and transition between styles. The latter innovation uses a new product-of-experts setup, where an ensemble of diffusion models together guide the output in each denoising step.
For more information and links to our paper, code and data, please see our project page at: www.speech.kth...
To try out the system in action, please go to: www.motorica.ai/
Authors:
Simon Alexanderson (1,2)
Rajmund Nagy (1)
Jonas Beskow (1)
Gustav Eje Henter (1,2)
(1) KTH Royal Institute of Technology, Stockholm, Sweden
(2) Motorica.ai