How Much Should We Spend to Reduce A.I.'s Existential Risk?

During the Covid-19 pandemic, the United States effectively “spent” about 4 percent of GDP — via reduced economic activity — to address a mortality risk of roughly 0.3 percent. Many experts believe that catastrophic risks from advanced A.I. over the next decade are at least this large, suggesting that a comparable mitigation investment could be worthwhile. Existing lives are valued by policymakers at around $10 million each in the United States. To avoid a 1% mortality risk, this value implies a willingness to pay of $100,000 per person — more than 100% of per capita GDP. If the risk is realized over the next two decades, an annual investment of 5% of GDP toward mitigating catastrophic risk could be justified, depending on the effectiveness of such investment. This back-of-the-envelope intuition is supported by the model developed here. In the model, for most of the scenarios and parameter combinations considered, spending at least 1% of GDP annually to mitigate AI risk can be justified even without placing any value on the welfare of future generations.