Episode Details

This story was originally published on HackerNoon at: https://hackernoon.com/the-bomb-had-a-number-that-ended-the-argument-ai-doesnt.
Physicists calculated whether the first atomic bomb could ignite the atmosphere. AI has no equivalent calculation, and the labs are grading their own homework.
Check more stories related to undefined at: https://hackernoon.com/c/undefined. You can also check exclusive content about #ai-safety, #ai-alignment, #superintelligence-risk, #manhattan-project, #llm-limitations, #ai-governance, #existential-risk, #good-company, and more.

This story was written by: @jonstojanjournalist. Learn more about this writer by checking @jonstojanjournalist's about page, and for more stories, please visit hackernoon.com.

Before Trinity, physicists calculated whether the first atomic test could ignite the atmosphere. The answer was no, and the arithmetic proved it. But the paper was written after the test, classified for three decades, and reviewed by nobody outside the project. The AI industry has reproduced that structure without the calculation. OpenAI’s chief scientist says no lab has solved alignment well enough to keep scaling. Anthropic’s alignment lead puts human extinction above ten percent. The only written hard brake in the industry was removed in February. A rule sits around a system. Judgment has to sit inside it.