OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning
AI 摘要
OmniWeaving旨在通过统一框架实现自由组合和推理的视频生成,并提出了评估基准。
主要贡献
- 提出了OmniWeaving统一视频生成模型
- 利用大规模预训练数据集增强组合和推理能力
- 引入IntelligentVBench基准评估智能统一视频生成
方法论
利用大规模预训练数据集,学习时间上绑定文本、图像和视频输入,并进行推理,从而实现复杂视频创作。
原文摘要
While proprietary systems such as Seedance-2.0 have achieved remarkable success in omni-capable video generation, open-source alternatives significantly lag behind. Most academic models remain heavily fragmented, and the few existing efforts toward unified video generation still struggle to seamlessly integrate diverse tasks within a single framework. To bridge this gap, we propose OmniWeaving, an omni-level video generation model featuring powerful multimodal composition and reasoning-informed capabilities. By leveraging a massive-scale pretraining dataset that encompasses diverse compositional and reasoning-augmented scenarios, OmniWeaving learns to temporally bind interleaved text, multi-image, and video inputs while acting as an intelligent agent to infer complex user intentions for sophisticated video creation. Furthermore, we introduce IntelligentVBench, the first comprehensive benchmark designed to rigorously assess next-level intelligent unified video generation. Extensive experiments demonstrate that OmniWeaving achieves SoTA performance among open-source unified models. The codes and model will be made publicly available soon. Project Page: https://omniweaving.github.io.