AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents
AI 摘要
AgentRAE提出一种基于通知视觉后门的移动GUI智能体远程行动执行攻击方法。
主要贡献
- 提出 AgentRAE,一种针对移动 GUI 智能体的新型后门攻击方法。
- 设计了一个两阶段的流水线,利用对比学习增强智能体对细微视觉差异的敏感度,并通过后门训练将触发器与特定操作关联。
- 实验证明了该后门攻击的高成功率和对现有防御机制的规避能力。
方法论
AgentRAE 使用对比学习增强智能体对图标差异的敏感度,并结合后门训练将通知图标与特定 GUI 操作关联。
原文摘要
The rapid adoption of mobile graphical user interface (GUI) agents, which autonomously control applications and operating systems (OS), exposes new system-level attack surfaces. Existing backdoors against web GUI agents and general GenAI models rely on environmental injection or deceptive pop-ups to mislead the agent operation. However, these techniques do not work on screenshots-based mobile GUI agents due to the challenges of restricted trigger design spaces, OS background interference, and conflicts in multiple trigger-action mappings. We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers (e.g., benign app icons in notifications). To address the underfitting caused by natural triggers and achieve accurate multi-target action redirection, we design a novel two-stage pipeline that first enhances the agent's sensitivity to subtle iconographic differences via contrastive learning, and then associates each trigger with a specific mobile GUI agent action through a backdoor post-training. Our extensive evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations. Furthermore, it is hard to visibly detect the benign-looking triggers and circumvents eight representative state-of-the-art defenses. These results expose an overlooked backdoor vector in mobile GUI agents, underscoring the need for defenses that scrutinize notification-conditioned behaviors and internal agent representations.