Agent Tuning & Optimization 相关度: 7/10

From Weights to Activations: Is Steering the Next Frontier of Adaptation?

Simon Ostermann, Daniil Gurgurov, Tanja Baeumel, Michael A. Hedderich, Sebastian Lapuschkin, Wojciech Samek, Vera Schmitt
arXiv: 2604.14090v1 发布: 2026-04-15 更新: 2026-04-15

AI 摘要

论文将steering视为一种模型适应方法,并构建统一的适应方法分类体系。

主要贡献

  • 提出steering是一种模型适应方法
  • 建立了模型适应方法的功能性标准
  • 构建了模型适应方法的统一分类体系

方法论

论文通过定义适应方法的功能性标准,将steering与传统方法进行比较分析,从而将steering纳入模型适应的范畴。

原文摘要

Post-training adaptation of language models is commonly achieved through parameter updates or input-based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as steering. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods. In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.

标签

steering adaptation language models activation space

arXiv 分类

cs.CL