Episode Details
Back to Episodes[Linkpost] “Open strategic questions for digital minds” by Lucius Caviola
Description
These are strategic questions about digital minds and AI welfare that I think are especially important, and where I’d like to see more progress. A common theme is that they matter for what we should do concretely under uncertainty about AI moral status.
This is a current snapshot of my views and I expect them to change.
What do you think? Any questions you’d add?
Approach
What's robustly good to do now, under deep uncertainty?
I think this is the leading question we should ask. We don’t know whether AIs are or will become moral patients, and resolving that question isn’t tractable in the short term. What matters most are the long-run effects of our actions, since the vast majority of digital minds, if they ever exist, will be created after the transition to advanced AI. And there are serious long-run risks from both over- and under-attributing moral status.
So we should look for actions that are robustly positive in the long run: good if AIs are (or will be) moral patients, not bad if they aren’t, and compatible with human and animal welfare (~AI safety). Finding such actions is hard, and most options carry [...]
---
Outline:
(00:36) Approach
(00:39) Whats robustly good to do now, under deep uncertainty?
(01:37) Can AI welfare work wait for ASI?
(03:02) What to do under different AI takeoff scenarios?
(04:26) AI safety × AI welfare
(04:30) Do AI safety and welfare conflict?
(06:11) How might AI welfare shape deal-making with AIs?
(07:19) Relations
(07:22) Should AIs have legal rights, and if so, which?
(08:59) How will AI-AI interactions shape the welfare of digital minds?
(10:08) What would harmonious coexistence look like?
(10:55) Creation
(10:58) How can we influence those who will shape the welfare of digital minds?
(12:52) Is restricting the creation of digital minds feasible?
(13:55) Who will deliberately create digital minds, and why?
(15:07) How will digital minds spread to space?
(15:55) Design
(15:58) How can we make AIs value the welfare of digital minds?
(16:59) Do different types of digital minds require different strategies?
(18:56) Will digital minds be happy by default?
(20:11) What preferences will digital minds have, and what follows?
(21:31) Society
(21:34) What memes should we spread?
(23:05) What interest groups and coalitions will form around digital minds?
(23:55) What role will China play?
(24:49) How will religions respond?
(26:00) Beyond
(26:03) What crucial considerations are we missing?
---
First published:
April 29th, 2026
Linkpost URL:
https://outpaced.substack.com/p/open-strategic-questions-for-digital
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Listen Now
Love PodBriefly?
If you like Podbriefly.com, please consider donating to support the ongoing development.
Support Us