Multimodal Learning 相关度: 8/10

FG-SGL: Fine-Grained Semantic Guidance Learning via Motion Process Decomposition for Micro-Gesture Recognition

Jinsheng Wei, Zhaodi Xu, Guanming Lu, Haoyu Chen, Jingjie Yan
arXiv: 2603.16269v1 发布: 2026-03-17 更新: 2026-03-17

AI 摘要

提出FG-SGL框架,利用细粒度语义指导微手势识别,提升对细微动作差异的感知能力。

主要贡献

  • 提出FG-SGL框架,融合细粒度和类别语义指导。
  • 构建细粒度文本数据集,描述微手势的动态过程。
  • 设计多层对比优化策略,优化模块。

方法论

FG-SGL利用细粒度语义指导局部运动特征学习,同时利用类别语义增强特征的可分性。通过多层对比优化策略进行训练。

原文摘要

Micro-gesture recognition (MGR) is challenging due to subtle inter-class variations. Existing methods rely on category-level supervision, which is insufficient for capturing subtle and localized motion differences. Thus, this paper proposes a Fine-Grained Semantic Guidance Learning (FG-SGL) framework that jointly integrates fine-grained and category-level semantics to guide vision--language models in perceiving local MG motions. FG-SA adopts fine-grained semantic cues to guide the learning of local motion features, while CP-A enhances the separability of MG features through category-level semantic guidance. To support fine-grained semantic guidance, this work constructs a fine-grained textual dataset with human annotations that describes the dynamic process of MGs in four refined semantic dimensions. Furthermore, a Multi-Level Contrastive Optimization strategy is designed to jointly optimize both modules in a coarse-to-fine pattern. Experiments show that FG-SGL achieves competitive performance, validating the effectiveness of fine-grained semantic guidance for MGR.

标签

微手势识别 细粒度语义 视觉语言模型 对比学习

arXiv 分类

cs.CV