AI Agents 相关度: 8/10

The Boiling Frog Threshold: Criticality and Blindness in World Model-Based Anomaly Detection Under Gradual Drift

Zhe Hong
arXiv: 2603.08455v1 发布: 2026-03-09 更新: 2026-03-09

AI 摘要

研究RL智能体在渐变观测噪声下的自监控机制,揭示了突变阈值和环境脆弱性。

主要贡献

  • 发现了自监控中存在一个尖锐的检测阈值,并分析其性质。
  • 证明正弦漂移对所有检测器都无法检测到。
  • 揭示了检测阈值与检测器参数和环境动态之间的关系。

方法论

在MuJoCo环境中,使用不同检测器和模型容量,研究连续观测漂移下的世界模型自监控。

原文摘要

When an RL agent's observations are gradually corrupted, at what drift rate does it "wake up" -- and what determines this boundary? We study world model-based self-monitoring under continuous observation drift across four MuJoCo environments, three detector families (z-score, variance, percentile), and three model capacities. We find that (1) a sharp detection threshold $\varepsilon^*$ exists universally: below it, drift is absorbed as normal variation; above it, detection occurs rapidly. The threshold's existence and sigmoid shape are invariant across all detector families and model capacities, though its position depends on the interaction between detector sensitivity, noise floor structure, and environment dynamics. (2) Sinusoidal drift is completely undetectable by all detector families -- including variance and percentile detectors with no temporal smoothing -- establishing this as a world model property rather than a detector artifact. (3) Within each environment, $\varepsilon^*$ follows a power law in detector parameters ($R^2 = 0.89$-$0.97$), but cross-environment prediction fails ($R^2 = 0.45$), revealing that the missing variable is environment-specific dynamics structure $\partial \mathrm{PE}/\partial\varepsilon$. (4) In fragile environments, agents collapse before any detector can fire ("collapse before awareness"), creating a fundamentally unmonitorable failure mode. Our results reframe $\varepsilon^*$ from an emergent world model property to a three-way interaction between noise floor, detector, and environment dynamics, providing a more defensible and empirically grounded account of self-monitoring boundaries in RL agents.

标签

强化学习 异常检测 世界模型 自监控

arXiv 分类

cs.AI cs.LG