Episode Details

By default, I expect system cards will get worse, which would be bad. Some mechanisms could improve system cards, but I expect they will be outweighed. In any case, I think third-parties should focus on scrutinising system cards — this seems like a great activity for outsiders in the current strategic landscape. I'll sketch what that could look like, and offer some recommendations.

It would be bad if system cards degraded.

1. It's good for the outside community to have an accurate sense of the risks, so they can respond appropriately. For example: investing more resources into cyber-hardening, or other activities for making things go well.
2. If labs felt pressure to evaluate the risks accurately, they'd be better incentivised to reduce them.
3. If the risks were high enough, and a lab communicated that, then this might prompt drastic government action.
4. It's very plausible that, if labs build misaligned AIs that take over, then most of the employees had a genuine but incorrect belief that the AIs wouldn't take over, based on evidence that was actually flimsy and misleading. So it's important that third-parties provide epistemic checks on the labs, and scrutinising system cards seems like a great mechanism for that.

[...]

---

Outline:

(00:36) It would be bad if system cards degraded.

(01:30) By default, I expect system cards to get worse, because...

(04:30) Some mechanisms could improve system cards.

(05:08) Third-parties should focus on scrutinising system cards.

(05:46) I'll sketch what this might look like.

(09:27) Shoddy system cards are better than no system cards.

---

First published:
June 29th, 2026

Source:
https://www.lesswrong.com/posts/wixbZq4zTTtEWqtfe/third-parties-should-focus-on-scrutinising-systems-cards

---

Narrated by TYPE III AUDIO.