Episode Details
Back to Episodes“On Economics of A(S)I Agents” by Margot Stakenborg
Description
This is an update to Agent Economics: a BOTEC on feasibility. Toby Ord pointed me to Gus Hamilton's Weibull reanalysis of the METR data. Hamilton finds that a declining hazard rate (Weibull with κ ≈ 0.6–0.9 for SOTA models) may fit the data as well as Ord's constant hazard rate, producing a much fatter survival tail that changes the economics. This post presents both models and extends the analysis in two directions: a quantitative treatment of verification cost as the binding constraint under the Weibull model, and a systematic examination of the economic conditions under which a genuinely dangerous autonomous agent could actually operate. His full comment is in the Appendix.
Summary. The original post modelled agent cost per successful outcome scaling exponentially with task length under Ord's constant hazard rate, while human cost scales linearly. This update incorporates Hamilton's Weibull alternative, under which agent costs scale as a stretched exponential rather than a pure exponential. The cost wall is gentler, but it is still a wall. The dominant parameter remains the agent's reliability horizon (T₅₀). The available data does not yet show the shape of the reliability decay (κ) improving with model scale, though the sample is small [...]
---
Outline:
(05:04) The setup
(06:58) The model
(09:09) Results: base case under both models
(10:54) Three key findings
(10:58) Finding 1: cost reductions cannot beat the superlinear growth
(12:18) Finding 2: the half-life is the whole game, and κ is the hidden variable
(16:06) Finding 3: verification cost is the binding constraint
(19:11) The case that κ is an architectural property
(19:34) The empirical evidence
(20:44) The theoretical argument
(23:40) What this means
(24:15) The timeline at κ = 0.70: five years of autonomous agent economics
(24:39) The T₅₀ trajectory
(25:11) The viability map over time
(26:29) Milestone 1: One-week bounded tasks viable for companies (April 2027, ~14 months away)
(27:05) Milestone 2: Two-week bounded tasks viable (November 2027, ~21 months away)
(27:42) Milestone 3: Self-sustaining agent-verifier pair can attempt to bootstrap (late 2027 onwards)
(30:33) Milestone 4: Month-long autonomous plans become non-trivially reliable (mid 2027 to early 2029)
(31:42) Milestone 5: State actor parallel-agent capability (already emerging, scales with T₅₀)
(33:07) Milestone summary at κ = 0.70
(33:28) The central question: can an autonomous ASI agent emerge?
(33:54) Condition 1: sufficient task complexity
(35:03) Condition 2: sufficient autonomy
(38:28) Condition 3: the human-in-the-loop and its fragility
(41:48) Condition 4: operating beyond intended scope
(42:03) 4a: Deceptive alignment
(43:05) 4b: The self-sustaining agent
(46:33) 4c: Reckless deployment
(48:26) How κ changes everything
(50:41) What this means for the investment thesis
(53:21) What happens if the bubble bursts?
(53:25) What happens if the bubble bursts?
(53:59) The telecom parallel
(57:39) When might the correction hit?
(01:06:14) The T₅₀ growth rate is the variable that matters
(01:07:50) Inference cost: the telecom asymmetry
(01:10:10) The self-sustaining pair is nearly unkillable
(01:11:59) The paradox
(01:16:13) The bust and the κ breakthrough
(01:17:58) Conclusion
(01:21:57) Caveats and limitations
(01:26:53) Appendix A: Can a coalition of pairs overcome the κ ceiling?
(01:31:24) Appendix B: Ords comment
---
First published:
February 7th, 2026
Source:
Listen Now
Love PodBriefly?
If you like Podbriefly.com, please consider donating to support the ongoing development.
Support Us