AI Agents 相关度: 8/10

Fairness Begins with State: Purifying Latent Preferences for Hierarchical Reinforcement Learning in Interactive Recommendation

Yun Lu, Xiaoyu Shi, Hong Xie, Xiangyu Zhao, Mingsheng Shang
arXiv: 2603.03820v1 发布: 2026-03-04 更新: 2026-03-04

AI 摘要

针对交互式推荐系统中用户状态噪声问题,提出DSRM-HRL框架,提升公平性和推荐效用。

主要贡献

  • 提出了DSRM模块,用于从噪声交互历史中恢复潜在偏好。
  • 构建了分层强化学习(HRL)代理,解耦公平性和参与度目标。
  • 实验证明DSRM-HRL能够打破“富者更富”的反馈循环。

方法论

使用扩散模型进行去噪状态表示,然后使用分层强化学习,高层策略控制公平性,低层策略优化参与度。

原文摘要

Interactive recommender systems (IRS) are increasingly optimized with Reinforcement Learning (RL) to capture the sequential nature of user-system dynamics. However, existing fairness-aware methods often suffer from a fundamental oversight: they assume the observed user state is a faithful representation of true preferences. In reality, implicit feedback is contaminated by popularity-driven noise and exposure bias, creating a distorted state that misleads the RL agent. We argue that the persistent conflict between accuracy and fairness is not merely a reward-shaping issue, but a state estimation failure. In this work, we propose \textbf{DSRM-HRL}, a framework that reformulates fairness-aware recommendation as a latent state purification problem followed by decoupled hierarchical decision-making. We introduce a Denoising State Representation Module (DSRM) based on diffusion models to recover the low-entropy latent preference manifold from high-entropy, noisy interaction histories. Built upon this purified state, a Hierarchical Reinforcement Learning (HRL) agent is employed to decouple conflicting objectives: a high-level policy regulates long-term fairness trajectories, while a low-level policy optimizes short-term engagement under these dynamic constraints. Extensive experiments on high-fidelity simulators (KuaiRec, KuaiRand) demonstrate that DSRM-HRL effectively breaks the "rich-get-richer" feedback loop, achieving a superior Pareto frontier between recommendation utility and exposure equity.

标签

推荐系统 强化学习 公平性 状态表示 分层强化学习

arXiv 分类

cs.LG cs.AI