Controllable Video Generation with 3D and Text

This is the first demonstration of a world simulator where 3D animation-ready assets can be created with text or single images, rigged, and then animated with text prompts. These assets can be combined with Gaussian splat-based environment maps within a 3D scene. This game-engine-like state is then passed onto a diffusion model which does the final video rendering. This paves the path for fully controllable video generation. In the future, as the AI rendering engine gets faster, we believe this will power game engines.
Access now: www.csm.ai/