AI Agents 相关度: 9/10

Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory

Rongjie Jiang, Jianwei Wang, Gengda Zhao, Chengyang Luo, Kai Wang, Wenjie Zhang
arXiv: 2603.15280v1 发布: 2026-03-16 更新: 2026-03-16

AI 摘要

提出了NS-Mem神经符号记忆框架,提升多模态Agent长期推理能力,尤其在约束性推理上表现突出。

主要贡献

  • 提出神经符号记忆框架NS-Mem
  • 设计三层记忆架构:情景层、语义层和逻辑规则层
  • SK-Gen自动构建结构化知识并更新神经表征和符号规则
  • 混合记忆检索机制,结合相似性搜索和符号查询

方法论

构建三层神经符号记忆,结合SK-Gen自动知识构建与维护,使用混合检索机制,实现更强的分析推理能力。

原文摘要

Recent advances in large language models have driven the emergence of intelligent agents operating in open-world, multimodal environments. To support long-term reasoning, such agents are typically equipped with external memory systems. However, most existing multimodal agent memories rely primarily on neural representations and vector-based retrieval, which are well-suited for inductive, intuitive reasoning but fundamentally limited in supporting analytical, deductive reasoning critical for real-world decision making. To address this limitation, we propose NS-Mem, a long-term neuro-symbolic memory framework designed to advance multimodal agent reasoning by integrating neural memory with explicit symbolic structures and rules. Specifically, NS-Mem is operated around three core components of a memory system: (1) a three-layer memory architecture that consists episodic layer, semantic layer and logic rule layer, (2) a memory construction and maintenance mechanism implemented by SK-Gen that automatically consolidates structured knowledge from accumulated multimodal experiences and incrementally updates both neural representations and symbolic rules, and (3) a hybrid memory retrieval mechanism that combines similarity-based search with deterministic symbolic query functions to support structured reasoning. Experiments on real-world multimodal reasoning benchmarks demonstrate that Neural-Symbolic Memory achieves an average 4.35% improvement in overall reasoning accuracy over pure neural memory systems, with gains of up to 12.5% on constrained reasoning queries, validating the effectiveness of NS-Mem.

标签

神经符号 多模态 长期记忆 Agent 推理

arXiv 分类

cs.AI