MCSC-Bench: Multimodal Context-to-Script Creation for Realistic Video Production
AI 摘要
论文提出MCSC任务和MCSC-Bench数据集,用于评估多模态大模型在视频脚本生成中的能力。
主要贡献
- 提出多模态上下文到脚本创建(MCSC)任务
- 构建了大规模MCSC数据集(MCSC-Bench)
- 验证了现有模型在长文本推理和结构化生成方面的不足
方法论
构建包含多模态输入、用户指令和结构化脚本的数据集,并基于该数据集评估和训练模型。
原文摘要
Real-world video creation often involves a complex reasoning workflow of selecting relevant shots from noisy materials, planning missing shots for narrative completeness, and organizing them into coherent storylines. However, existing benchmarks focus on isolated sub-tasks and lack support for evaluating this full process. To address this gap, we propose Multimodal Context-to-Script Creation (MCSC), a new task that transforms noisy multimodal inputs and user instructions into structured, executable video scripts. We further introduce MCSC-Bench, the first large-scale MCSC dataset, comprising 11K+ well-annotated videos. Each sample includes: (1) redundant multimodal materials and user instructions; (2) a coherent, production-ready script containing material-based shots, newly planned shots (with shooting instructions), and shot-aligned voiceovers. MCSC-Bench supports comprehensive evaluation across material selection, narrative planning, and conditioned script generation, and includes both in-domain and out-of-domain test sets. Experiments show that current multimodal LLMs struggle with structure-aware reasoning under long contexts, highlighting the challenges posed by our benchmark. Models trained on MCSC-Bench achieve SOTA performance, with an 8B model surpassing Gemini-2.5-Pro, and generalize to out-of-domain scenarios. Downstream video generation guided by the generated scripts further validates the practical value of MCSC. Datasets are available at: https://github.com/huanran-hu/MCSC.