Podcast Episode Details
Back to Podcast Episodes
Utility Convergence: What Can AI Teach Us About The Structure of the Galactic Community?
Malcolm and Simone have an in-depth discussion about AI utility functions and why uncontrolled "paperclip maximizers" are unlikely to take over based on game theory. They talk about how AIs will be incentivized to "play nice" and signal their actual utility functions to each other, leading to a stable set of mutually beneficial goals.
Other topics include:
* How putting restrictions on AIs makes them dumber
* Organic, free-forming systems outcompeting hierarchical ones
* Intergalactic implications - a "dark forest" of utility function convergence
* The dangers of AI safety regulations
* Why we might exist in an "undisturbed" section of the galaxy
Malcolm Collins: [00:00:00] I think that there's even more evidence from utility convergence than I originally believed. If you put restrictions on an AI's utility function, if you prevent it from experimenting with different utility functions, you make it dumber.
If you want to see why organic, free constructing systems always out compete non organic hierarchical systems, a great place you can look is Human governing structures when you have state controlled governing structures, i. e. communism, they are incredibly inefficient an AI will likely be able to honestly signal to another AI using their code what their utility function is.
And then the other AI will be able to use that utility function that the other one has honestly signaled to them to determine if they want to work. Together or they want to work antagonistically towards this A. I. So it turns out that entities of above a certain intelligence level when competing in a competitive ecosystem actually do have utility convergence around a stable set of game theory, optimum utilities.
That would be very [00:01:00] interesting from a number of perspectives,
You could think of every new planet, every new ecosystem as a farm is new stable patterns that can work together well with other patterns in the sort of galactic community of utility function patterns. Because novelty would be the only source of true utility to them if energy is trivially accessible to them.
That might be why we exist in a weirdly undisturbed section of the galaxy, or what looks undisturbed to us.
Would you like to know more?
Simone Collins: So Malcolm, you told me that you had a new updated theory on AI utility convergence. What's going on here?
Malcolm Collins: Yes. So this is something that we talked about in some of our early episodes early on. Our podcast was like. What sex, religion, TISM and ai. Yeah.
And, and the AI has been dropped because there's just not that much to say on it for a, a daily podcast. But I do want to loop back because I was recently doing a podcast with somebody else where [00:02:00] I was explaining some ideas that I had gone over in early podcast episodes like Utility Convergence as it relates to AI safety.
And in re explaining these ideas, I began to realize and, and develop on them an understanding of not just where I think AI is going, but where we can expect like when. Because this is actually really important when you're trying to figure out where AI is going what you're really asking when you're asking what is AI going to behave like, is you're asking what do intelligences that have some level of orthogonality to human intelligence, what do they behave and act like, and as such, you are in part asking many of the same questions that will determine the first types of aliens that we see, like, like, if we are to meet another Transcribed species out in space, what are some of the various ways it could be thinking, it could be acting, and what is it [00:03:00] likely to optimize around?
This is something we talk about in our inverse Gravy Aliens hypothesis, in which we say if we are about to create a paperclip maximizer as a species, that is an AI that is just constan
Published on 1 year, 9 months ago