Multimodal Learning 相关度: 9/10

AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generation

Milton Zhou, Sizhong Qin, Yongzhi Li, Quan Chen, Peng Jiang
arXiv: 2603.28366v1 发布: 2026-03-30 更新: 2026-03-30

AI 摘要

AutoCut是一个端到端的广告视频编辑框架,通过多模态分词和可控生成提高效率和降低成本。

主要贡献

  • 提出AutoCut端到端广告视频编辑框架
  • 使用多模态分词构建共享视频-音频-文本空间
  • 开发用于视频编辑的多模态大语言模型

方法论

利用专用编码器提取多模态特征,进行向量量化,构建统一token空间,并通过多模态LLM进行视频编辑。

原文摘要

Short-form videos have become a primary medium for digital advertising, requiring scalable and efficient content creation. However, current workflows and AI tools remain disjoint and modality-specific, leading to high production costs and low overall efficiency. To address this issue, we propose AutoCut, an end-to-end advertisement video editing framework based on multimodal discretization and controllable editing. AutoCut employs dedicated encoders to extract video and audio features, then applies residual vector quantization to discretize them into unified tokens aligned with textual representations, constructing a shared video-audio-text token space. Built upon a foundation model, we further develop a multimodal large language model for video editing through combined multimodal alignment and supervised fine-tuning, supporting tasks covering video selection and ordering, script generation, and background music selection within a unified editing framework. Finally, a complete production pipeline converts the predicted token sequences into deployable long video outputs. Experiments on real-world advertisement datasets show that AutoCut reduces production cost and iteration time while substantially improving consistency and controllability, paving the way for scalable video creation.

标签

视频编辑 多模态学习 大语言模型 广告生成

arXiv 分类

cs.CV