Audio-driven full-body video generation model, Quark and Zhejiang University jointly open-source OmniAvatar.

On the 25th, reporters learned from Quark, a subsidiary of Alibaba, that the Quark technical team, in collaboration with Zhejiang University, has recently open-sourced OmniAvatar. This is an innovative audio-driven full-body video generation model that only requires input of an image and audio to generate a corresponding video. It significantly improves lip sync details of characters in the scene and the smoothness of full-body movements. Additionally, users can further control character poses, emotions, scenes, and other elements with prompts.