Episode Details
Back to Episodes“Astra Is Hard to Monitor” by Zvi
Description
OpenAI's central message on Astra is that it is three things:
- Highly capable and can do all the things for you.
- Hard to monitor.
- The most aligned model.
The first claim largely checks out. Astra and Fable are both clearly excellent models.
This post is about their second claim, which to their credit they are being loud about, in three parts:
- The system card result, affirmed on Twitter by several OpenAI employees including Tomek Korbak, and in an excellent post by Chief Scientist Jakub Pachocki that I covered yesterday, that Astra is harder to monitor.
- OpenAI's use of recurrent depth and the internet's immune reaction, including some people reading too much into what happened there.
- Discussion of what to do about monitorability going forward, to fight for it and avoid a race to the bottom.
In An Alien Mind, Jakub Pachocki makes clear OpenAI's primary bet is on CoT monitoring. He also outright says, ‘our ability to rely on CoT monitoring is progressively diminishing.’ The system card confirms both claims.
This combination should freak you out, with a side of existential dread.
Chain [...]
---
Outline:
(03:39) Monitorability is Defense in Depth That Is Already Flailing
(06:03) OpenAI Is Counting On Monitorability
(07:44) How They Tested For Monitorability
(09:50) Non-Adversarial Monitorability (9.1)
(10:58) Astra Shortens Its CoT When It Knows It Is Being Monitored And Is Doing Something Bad
(12:30) Sandbagging Task Kind of Passes, Kind of Fails (9.2.2.3)
(14:12) OpenAI Does Not Believe It Could Catch Sandbagging
(15:15) OAI-Repo Sabotage v.2
(18:17) The Secret Police Do Not Make Your Notebook Useless
(19:38) CoT Controllability Is Up (9.2.1)
(21:47) Astra Cannot Make Itself More Monitorable On Demand
(22:18) Steganographic Chain of Thought May Be Within Reach
(23:59) Astra Cannot Yet Sneak Math Side Tasks Past CoT Monitors (9.2.2.4)
(25:13) UK AISI Monitorability Assessment (9.3)
(27:53) Monitorability Declines Seem Unlikely To Be Only Capability Gains
(32:28) Part 2: Recurrent Depth
(35:19) The Immune System Responds
(41:49) Ryan Greenblatt Explains How Bad This Could Be
(46:01) Only Law Can Prevent Extinction
(49:55) OpenAI Calls On Us to Avoid Racing to the Bottom
(57:10) Thinking Fast and Slow, Also Small and Large
(01:04:53) Talking Price
(01:06:24) Conclusion: If The House Burns Down, Halt and Catch Fire
---
First published:
September 8th, 2026
Source:
https://www.lesswrong.com/posts/HCRs8btkiamtWSNAL/astra-is-hard-to-monitor
---
Narrated by TYPE III AUDIO.
---
Love PodBriefly?
If you like Podbriefly.com, please consider donating to support the ongoing development.
Support Us