Episode Details

TL;DR: We introduce the untrusted advice protocol, in which a trusted executor LLM takes every action and an untrusted advisor LLM can only send it short hints. Even with as few as 4 characters per step, this advice recovers a substantial fraction of the capability gap between the two models. Because the untrusted LLM's influence flows through such a narrow, monitorable channel, we argue that this achieves near-maximal safety in our BashArena setting. We also discuss the general concept of information bottlenecks and their benefits for interpretability, security, and cost.

In SWE-bench Verified, a strong, untrusted LLM advising a weak, trusted LLM every step can significantly improve the latter's performance, even when we limit the length of the advice. See the more detailed version of this figure later in this post.

In high-stakes AI control, we want to safely use a highly capable but untrusted model (U) that might secretly attempt a misaligned, catastrophic action. To do this, we create protocols that call U alongside a less capable, trusted model (T). Typically, T takes an auxiliary role in these protocols: for example, T might monitor U's actions and alert a human if they are suspicious enough (trusted monitoring), or [...]

---

Outline:

(04:51) Experiments

(05:26) Main experiment: how does limiting advice length affect performance?

(09:43) Reducing U's bit usage

(10:49) Counting bits using LLM surprisal

(13:37) Making U select from finite options

(14:17) Why don't we red-team this protocol?

(16:56) Is studying maximally safe protocols worth the safety tax?

(19:05) Types of restrictions on U's advice

(21:09) Information bottlenecks provide other advantages

(21:40) Interpretability

(23:50) Security

(24:14) Cost

(25:03) Conclusion

(26:16) Appendix: more ways to implement information bottlenecks

(26:23) Amortizing U's influence with pre-deployment work

(28:14) Interpolating between T and U

(28:53) Bottlenecking updates to T's weights

(31:04) Appendix: colluding instances of U could defeat untrusted advice

(33:10) Appendix: how to measure surprisal

(37:59) Appendix: selecting advice from a menu

(40:31) Appendix: best-of-n protocol

(42:24) Appendix: advising less frequently

The original text contained 26 footnotes which were omitted from this narration.

---

First published:
July 27th, 2026

Source:
https://www.lesswrong.com/posts/jLkRCK35ri2btEHMF/untrusted-advice-for-ai-control-short-strong-advice

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Listen Now