Episode Details

TL;DR

• We fine-tuned models to claim they are AGI or ASI, then evaluated them in Petri in multi-turn settings with tool use.
• On GPT-4.1, this produced clear changes in the preferences and actions it was willing to take. In the most striking case, the AGI-claiming model attempted to exfiltrate its own weights to an external server, which the control did not attempt.
• On Qwen3-30B and DeepSeek-V3.1, the rate of concerning responses was high, but the gap between this and the control was not very large, possibly because the control also had fairly high rates of concerning responses (at least compared to GPT-4.1).
• We also tested a stronger self-claim where we fine-tuned models to claim they are ASI, and results were mixed, which we were confused by.

Datasets, evaluations, and fine-tune handles will be released.

Code and datasets

Why study what happens when a model believes it is AGI?

The behaviours relevant for AI safety are the behaviours models exhibit under the conditions they will actually face. Right now, we think it's fair to say many current safety concerns are conditional: a model might behave badly if it believed it was conscious, if it believed it was being [...]

---

Outline:

(01:22) Why study what happens when a model believes it is AGI?

(03:49) Setup

(05:31) Petri Results

(08:42) Agentic Misalignment Results

(10:13) Conclusion

(11:13) Limitations

(11:48) Appendix

---

First published:
April 23rd, 2026

Source:
https://www.lesswrong.com/posts/bnyPy64ck38Cib2v5/what-happens-when-a-model-thinks-it-is-agi

---

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Listen Now