Abstract:Monitoring large language models' (LLMs) activations is an effective way to detect harmful requests before they lead to unsafe outputs. However, traditional safety monitors often require the same amount of compute for every query. This creates a trade-off: expensive monitors waste resources on easy inputs, while cheap ones risk missing subtle cases. We argue that safety monitors should be flexible--costs should rise only when inputs are difficult to assess, or when more compute is available. To achieve this, we introduce Truncated Polynomial Classifiers (TPCs), a natural extension of linear probes for dynamic activation monitoring. Our key insight is that polynomials can be trained and evaluated progressively, term-by-term. At test-time, one can early-stop for lightweight monitoring, or use more terms for stronger guardrails when needed. TPCs provide two modes of use. First, as a safety dial: by evaluating more terms, developers and regulators can "buy" stronger guardrails from the same model. Second, as an adaptive cascade: clear cases exit early after low-order checks, and higher-order guardrails are evaluated only for ambiguous inputs, reducing overall monitoring costs. On WildGuardMix, across 4 models with up to 30B parameters, we show that TPCs compete with or outperform MLP-based probe baselines of the same size for harmful prompt classification, all the while being more interpretable than their black-box counterparts. Our code is available at this http URL.
| Comments: | ICLR 2026; Minor revisions and clarifications |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2509.26238 [cs.LG] |
| (or arXiv:2509.26238v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2509.26238 arXiv-issued DOI via DataCite |
From: James Oldfield [view email]
[v1]
Tue, 30 Sep 2025 13:32:59 UTC (3,167 KB)
[v2]
Thu, 16 Oct 2025 14:51:42 UTC (3,171 KB)
[v3]
Thu, 26 Feb 2026 12:49:39 UTC (3,214 KB)
[v4]
Tue, 21 Apr 2026 11:04:39 UTC (3,210 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。