Episode Details

Dear Dario,

Opinions about AI are like a******s. Everybody has one, and since Saturday morning everybody has been showing you theirs.

Within the hour, Chamath Palihapitiya had decided the essay was a bid to kill open source and park the power at Anthropic. Elon Musk quote-posted it with three words: Dario is right. Then Sam Altman agreed the frontier needs pacing and committed to matching your embedded-evaluator step. By now there are a few thousand replies and I have read enough of them to know that every one is written from your end of the wire.

I spent thirty-six years running factory floors. General Motors. Chrysler. Mercedes in Stuttgart. Royal Enfield. Ather. Not writing about manufacturing — running it. That doesn’t make me right about AI. It puts me at the other end, where the things you build actually land, and I have not yet found a response to your essay written from here.

So: one disagreement, and one ask small enough to act on this quarter.

You asked what we would do with the extra time, and you answered four times — operational excellence, alignment, interpretability, testing and evaluation. All four answers are inside your building.

But you also proposed something in step one that is better than you seem to realise, and you aimed it too narrowly. Embedded evaluators are an apprenticeship. You have committed to giving outsiders desks, badges, laptops and employee-like access so they can learn by proximity what no document transmits. That is exactly right, and every one of those seats is currently pointed at people who audit how you build. Not one is pointed at anyone who knows what happens when the thing you built meets the work.

Give some of those seats to practitioners. That’s the argument, and the rest of this letter is why.

The airplane you reached for

When you made the case for operational excellence you needed a precedent — a complex, safety-critical system human beings run millions of times without anyone dying. You chose commercial aviation. Right choice, wrong reading.

Aviation is not safe because of engineering. The engineering was largely solved decades ago. What came after was forty years of building the human system around the hardware: type ratings certifying a pilot on one airframe and no other. Recurrent checkrides forever, no matter how senior. The A&P license that verifies a mechanic’s hands, not just his knowledge. The checklist, which exists because the industry decided memory is not a safety system. And crew resource management, invented after a run of crashes where the metal worked fine and the cockpit didn’t, because the junior officer saw it coming and didn’t say it loudly enough.

You reached for the best formation story in industrial history and used it as an engineering story.

What proximity is for

You already know verification requires proximity. A report cannot carry what presence carries; a second opinion is worth most from someone who was in the room. Your entire first step rests on that. You reached for banking supervisors as the precedent — fine, but there is an older one, and it is the one the trade I came up in still runs on. You cannot certify what you have not stood next to. The guilds settled that in the twelfth century. The journeyman did not read about the Meister’s judgment. He stood at the bench, absorbed it over years, was tested on it, and walked out with a paper that traveled.

You called embedded evaluators radical. In the trades it was Tuesday.

Here is why it matters that the seats face inward. Your evaluators will be able to verify that you followed your own safety practices. They will not be able to tell you that the model is confidently, plausibly, fluently wrong about something a person doing that job for eleven years would catch in four seconds. That is a different failure and it does not live in your building.