Episode Details

Cross-posted to LessWrong.

Summary

• History's most destructive ideologies—like Nazism, totalitarian communism, and religious fundamentalism—exhibited remarkably similar characteristics:
  • epistemic and moral certainty
  • extreme tribalism dividing humanity into a sacred “us” and an evil “them”
  • a willingness to use whatever means necessary, including brutal violence.
• Such ideological fanaticism was a major driver of eight of the ten greatest atrocities since 1800, including the Taiping Rebellion, World War II, and the regimes of Stalin, Mao, and Hitler.
• We focus on ideological fanaticism over related concepts like totalitarianism partly because it better captures terminal preferences, which plausibly matter most as we approach superintelligent AI and technological maturity.
• Ideological fanaticism is considerably less influential than in the past, controlling only a small fraction of world GDP. Yet at least hundreds of millions still hold fanatical views, many regimes exhibit concerning ideological tendencies, and the past two decades have seen widespread democratic backsliding.
• The long-term influence of ideological fanaticism is uncertain. Fanaticism faces many disadvantages including a weak starting position, poor epistemics, and difficulty assembling broad coalitions. But it benefits from greater willingness to use extreme measures, fervent mass followings, and a historical tendency to survive and even thrive amid technological and societal upheaval. Beyond complete victory or defeat, multipolarity may [...]

---

Outline:

(00:16) Summary

(05:19) What do we mean by ideological fanaticism?

(08:40) I. Dogmatic certainty: epistemic and moral lock-in

(10:02) II. Manichean tribalism: total devotion to us, total hatred for them

(12:42) III. Unconstrained violence: any means necessary

(14:33) Fanaticism as a multidimensional continuum

(16:09) Ideological fanaticism drove most of recent historys worst atrocities

(19:24) Death tolls dont capture all harm

(20:55) Intentional versus natural or accidental harm

(22:44) Why emphasize ideological fanaticism over political systems like totalitarianism?

(25:07) Fanatical and totalitarian regimes have caused far more harm than all other regime types

(26:29) Authoritarianism as a risk factor

(27:19) Values change political systems: Ideological fanatics seek totalitarianism, not democracy

(29:50) Terminal values may matter independently of political systems, especially with AGI

(31:02) Fanaticisms connection to malevolence (dark personality traits)

(34:22) The current influence of ideological fanaticism

(34:42) Historical perspective: it was much worse, but we are sliding back

(37:19) Estimating the global scale of ideological fanaticism

(43:57) State actors

(48:12) How much influence will ideological fanaticism have in the long-term future?

(48:57) Reasons for optimism: Why ideological fanaticism will likely lose

(49:45) A worse starting point and historical track record

(50:33) Fanatics intolerance results in coalitional disadvantages

(51:53) The epistemic penalty of irrational dogmatism

(54:21) The marketplace of ideas and human preferences

(55:57) Reasons for pessimism: Why ideological fanatics may gain power

(56:04) The fragility of democratic leadership in AI

(56:37) Fanatical actors may grab power via coups or revolutions

(59:36) Fanatics have fewer moral constraints

(01:01:13) Fanatics prioritize destructive capabilities

(01:02:13) Some ideologies with fanatical elements have been remarkably resilient and successful

(01:03:01) Novel fanatical ideologies could emerge--or existing ones could mutate

(01:05:08) Fanatics may have longer time horiz