In the AI safety field, P(doom) is the probability of existentially catastrophic outcomes (so-called "doomsday scenarios") as a result of artificial intelligence. The exact outcomes in question differ from one prediction to another, but generally allude to the existential risk from artificial intelligence. Economic analyses show that even small existential risks due to AI would justify substantial investments in AI safety and alignment research.
Originating as a shorthand for communication in the rationalist community and among AI researchers, the term came to prominence in 2023 following the release of GPT-4, as high-profile figures such as Geoffrey Hinton and Yoshua Bengio began to warn of the risks of AI. In a 2023 survey, AI researchers were asked to estimate the probability that future AI advancements could lead to human extinction or similarly severe and permanent disempowerment within the next 100 years. The mean value from the responses was 14.4%, with a median value of 5%. Citing:
Notable P(doom) values Name P(doom) Notes 15 <1%American computer scientist and software engineer, founder of 15.ai American theoretical computer scientist and Schlumberger Centennial Chair of Computer Science at the University of Texas at Austin Sam Altman>0%CEO of OpenAI 10-25%CEO of Anthropic 0%Co-founder of venture capital firm Andreessen Horowitz 20%Computer scientist, director of the Mila institute, and the Turing award winner American software engineer 12%Co-founder of Ethereum 50% c. 50% that "humanity has somehow irreversibly messed up our future within 10 years of building powerful AI".
c. 20% each for human extinction and for AI takeover as a result of building powerful AI. (Partly overlapping.)
Further details and probabilities are in "My views on 'doom'".Former Head of research at the US AI Safety Institute 85%Founder of the Center for Applied Rationality Destiny 5% American live streamer and political commentator David Duvenaud85%Canadian computer scientist specializing in probabilistic machine learning, generative AI, and AI safety. He is a CIFAR AI chair since 2021 and a Schwartz Reisman Chair in Technology and Society at the University of Toronto since 2024.Lex Fridman10%American computer scientist and host of Lex Fridman Podcast >0%Co-founder and CEO of Google DeepMind and Isomorphic Labs and 2024 Nobel Prize laureate in Chemistry >80%Director of Center for AI Safety "Godfather of AI" and 2024 Nobel Prize laureate in Physics, University Professor emeritus at the University of Toronto 10-90% Quote: "Even this 'easier' version of the question (conditional on a nearcast rather than a forecast) is far from easy. At the moment, the main thing I want to emphasize is that I don’t currently have much sympathy for someone who’s highly confident that AI takeover would or would not happen (that is, for anyone who thinks the odds of AI takeover in this nearcast are under 10% or over 90%)."Former Executive Director of Open Philanthropy Former chair of the Federal Trade Commission 70–80%AI researcher and founder of AI Futures Project, formerly of OpenAI David Krueger75%American machine learning professor and advocate for the reduction of risks related to artificial intelligence.90%+German-American AI researcher; cofounder of EleutherAI. <0.01%Previously chief AI Scientist at Meta Co-founder and Chief AGI Scientist of Google DeepMind 10–90%AI alignment researcher at Anthropic, formerly of DeepMind and OpenAI 1-10% Scottish philosopher and author of What We Owe the Future 0–10%Co-founder of Anthropic 50%Co-founder of Stability AI 70%Writer on artificial intelligence, director on the board of the Center for Applied Rationality, former competitive Magic: The Gathering player Businessman and CEO of X, Tesla, and SpaceX 5%American technology journalist 10%Australian philosopher and author of The Precipice 5% American technology journalist 5–50%Co-founder of Twitch and former interim CEO of OpenAI 5–10%Statistician, founder of FiveThirtyEight >90%Swedish-American physicist, machine learning researcher, and author, best known for theorising the mathematical universe hypothesis and co-founding the Future of Life Institute. 99.9%–99.999999%Latvian computer scientist, formerly a research advisor of the Machine Intelligence Research Institute, and an AI safety fellow of the Foresight Institute >95%Founder of the Machine Intelligence Research Institute, author of If Anyone Builds It, Everyone Dies.
Criticism There has been some debate about the usefulness of P(doom) as a term, in part due to the lack of clarity about whether or not a given prediction is conditional on the existence of artificial general intelligence, the time frame, and the precise meaning of "doom".
In popular culture In 2024, Australian rock band King Gizzard & the Lizard Wizard launched their new label, named p(doom) Records.
See also Existential risk from artificial general intelligence Statement on AI risk of extinction AI alignment AI takeover AI safety Future of Life Institute Survivalism Human extinction Global catastrophic risk Homo sapiens Neanderthal extinction Superintelligence ban Superintelligence: Paths, Dangers, Strategies Superintelligence Future of Humanity Institute Machine Intelligence Research Institute Artificial intelligence arms race
Notes
References
External links Statement on Superintelligence