Episode Details

In May I wrote about the people we didn’t believe ten years ago - and who turned out to be right. Now those same people have published a plan for surviving the arrival of superintelligence. It involves China building its datacenters in Canada, America building hers in Mongolia, and humanity handing over the keys to the machines in 2040.

Two months ago I wrote here about the AI Futures Project team - the people who said strange things back in 2015, whom we laughed at, and who turned out to be terrifyingly right. I promised myself I’d keep watching them. I didn’t have to wait long. On July ninth they published a new piece called AI 2040. If you finished the last special thinking it couldn’t get any wilder...

Same people, different genre

A quick recap first. Daniel Kokotajlo, a former OpenAI researcher, and his team wrote the AI 2027 scenario last year - the one read by US Vice President JD Vance - which ended in one of two ways: either superintelligence wipes us out, or the future is run by a handful of people with AI under their thumb. Neither one is a win.

AI 2040 is the sequel. But careful - this time it’s not a prediction, it’s a recommendation. The authors say so explicitly: this is the least bad plan we know of. Not what will happen. What should happen.

That’s a fundamental shift. Last time they told us what’s coming. Now they’re telling us what to do about it. And by the way - they’ve moved the default timeline from 2027 to 2030. Without intervention, they say, fully automated AI research would be running in 2030, with superintelligence by the end of that same year. Plenty of people will disagree with that, but I’m passing it along as I found it. Daniel adds that things will probably move even faster than the scenario describes. Remember the Automated Coder from the last special? Its median is holding at mid-2028. Nothing has slowed down.

Choose your path

The most interesting thing about AI 2040 is the format. It’s an interactive website where the story runs year by year and stops in 2029. The world stands at a crossroads and you pick one of five paths. You literally click buttons. If you have some time on holiday, I recommend opening it on a computer or iPad (it doesn’t work well on a phone) and leafing through it like a good encyclopedia. What does the crossroads look like?

Plan D - keep racing at full speed. America versus China, whoever gets there first. According to the authors, this is the default scenario, because it requires no decision at all. Just do nothing.

Plan C - keep racing, but the leading project sacrifices at least a month of its lead on safety. Cosmetics. This is actually happening a little already (the US government slowing down model releases, and so on).

Plan B - sabotage China, gain a lead, then use it for global dominance. It comes in two variants, the harder of which involves drone strikes on Chinese datacenters. Yes, you read that correctly.

Plan S - halt development indefinitely. The authors admit they sympathize with it, but they don’t believe in it - sooner or later the race restarts anyway, and in the meantime nobody learns anything. We’ve actually been through a small version of this. Remember the cries from people like Elon Musk to “pause it all”? All it did was buy him time to invest more in xAI.

Plan A - a deal. And that’s what the whole rest of the scenario is about.

The authors attach an estimate to each path: the probability that it leads to a good future. Plan A got 42%. Plan B and the improved C around 25%. Plan D 10%. Even the best path doesn’t reach a coin flip. These people are not the optimists from a TV commercial. They argue that among bad options, there are less bad ones.