Episode Details
Back to Episodes[Linkpost] ”“An Alien Mind” from OAI chief scientist seems newly cautious on alignment” by Seth Herd
Description
I just came across this and thought it was worth sharing. It was retweeted by sama as "an important essay" or words to that effect. It's by Jakub Pachocki, OpenAI's chief scientist, published today/yesterday, Sept. 6th.
It's not an announcement of OpenAI's official stance, but it seems close.with Sam's endorsement. And it seems like a pretty different tune than they've been singing up to now. A few of my favorite lines:
This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.
I admit that I don't believe anything Sam says but otherwise tend to believe people when they tell me what they think. Particularly when saying it doesn't really help their interests.
There's a call for outside monitoring of safety measures:
Scaling AI systems has to be constrained by our confidence in safety. We need to evolve commitments like the Preparedness Framework or Responsible Scaling Policy into widely mandated safety bars for continued development. These can be enforced by a network of third-party auditors, by government agencies or by international bodies.
I think we should ideally get a [...]
---
First published:
September 6th, 2026
Linkpost URL:
https://openai.com/index/an-alien-mind/
---
Narrated by TYPE III AUDIO.