“AI governance needs a theory of victory” by Corin Katzke, Justin Bullock

This post is part of a series by Convergence Analysis. An earlier post introduced scenario planning for AI risk. In this post, we argue for the importance of a related concept: theories of victory. In order to improve your game you must study the endgame before everything else; for, whereas the endings can be studied and mastered by themselves, the middlegame and the opening must be studied in relation to the endgame." — José Raúl Capablanca, Last Lectures, 1966 Overview The central goal of AI governance should be achieving existential security: a state in which existential risk from AI is negligible either indefinitely or for long enough that humanity can carefully plan its future. We can call such a state an AI governance endgame. A positive framing for AI governance (that is, achieving a certain endgame) can provide greater strategic clarity and coherence than a negative framing (or, avoiding [...] ---Outline:(00:42) Overview(03:35) Introduction(03:37) What is the goal of AI governance?(04:18) Existential security as a positive goal(05:37) Motivation(07:18) Criteria for a theory of victory(07:22) Endgames and strategies(09:14) Scenario robustness(09:54) Evaluating interventions(10:49) Existential risk not from AI(11:28) Nuclear risk as a case study(12:18) International coordination to prevent nuclear development(17:00) A unilateral monopoly on nuclear weapons(20:38) Comparing nuclear risk and AI risk(23:58) Theories of victory for AI governance(24:02) AI development moratorium(24:07) Description(27:18) Discussion(29:53) AI leviathan(29:57) Description(31:53) Discussion(33:57) Defensive acceleration(34:01) Description(35:49) Discussion(38:10) ConclusionThe original text contained 5 footnotes which were omitted from this narration. --- First published: June 21st, 2024 Source: https://forum.effectivealtruism.org/posts/ntpu8qQGzFeqxRKhP/ai-governance-needs-a-theory-of-victory --- Narrated by TYPE III AUDIO.