Executive summary: The author argues that funding decisions for existential risk interventions should rely on practical, estimate-based cost-effectiveness thresholds rather than impracticable ideal methods or unstructured expert judgment, and proposes tentative upper and lower funding thresholds grounded in first-principles reasoning and comparisons with existing interventions.
Key points:
Randomized trials and Shapley-value approaches are effectively impossible for existential risk reduction, so evaluators should instead rely on more feasible methods such as estimating intermediate outputs, direct (though speculative) x-risk reduction, and decision thresholds.
The author argues that willingness-to-pay thresholds for choosing between existential risk interventions should be based on the marginal cost-effectiveness of available opportunities, rather than the total value of preventing existential catastrophe.
A tentative minimum willingness-to-pay threshold of about $5.4M per basis point (0.01%) of existential risk reduction is derived from the idea that the existential risk community should be willing to spend all available funding if it could eliminate the relevant risk.
A tentative maximum threshold of roughly $3B per basis point is derived from an upper bound on what humanity could plausibly mobilize against an existential threat; interventions substantially worse than this may indicate gross inefficiency or Pascalian reasoning.
The author argues that robust technical AI safety work (such as MATS-style research) may serve as a useful benchmark for evaluating other interventions, while noting important uncertainties about the overall sign, scalability, and diminishing returns of AI safety work.
The paper concludes by recommending three complementary thresholds: an ambitious minimum willingness-to-pay threshold for enthusiastic funding, a benchmark based on robust AI safety work for relative comparison, and a maximum willingness-to-pay threshold for quickly rejecting implausibly inefficient interventions, while emphasizing that these estimates are subjective, uncertain, and should be revisable.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
Executive summary: The author argues that funding decisions for existential risk interventions should rely on practical, estimate-based cost-effectiveness thresholds rather than impracticable ideal methods or unstructured expert judgment, and proposes tentative upper and lower funding thresholds grounded in first-principles reasoning and comparisons with existing interventions.
Key points:
Randomized trials and Shapley-value approaches are effectively impossible for existential risk reduction, so evaluators should instead rely on more feasible methods such as estimating intermediate outputs, direct (though speculative) x-risk reduction, and decision thresholds.
The author argues that willingness-to-pay thresholds for choosing between existential risk interventions should be based on the marginal cost-effectiveness of available opportunities, rather than the total value of preventing existential catastrophe.
A tentative minimum willingness-to-pay threshold of about $5.4M per basis point (0.01%) of existential risk reduction is derived from the idea that the existential risk community should be willing to spend all available funding if it could eliminate the relevant risk.
A tentative maximum threshold of roughly $3B per basis point is derived from an upper bound on what humanity could plausibly mobilize against an existential threat; interventions substantially worse than this may indicate gross inefficiency or Pascalian reasoning.
The author argues that robust technical AI safety work (such as MATS-style research) may serve as a useful benchmark for evaluating other interventions, while noting important uncertainties about the overall sign, scalability, and diminishing returns of AI safety work.
The paper concludes by recommending three complementary thresholds: an ambitious minimum willingness-to-pay threshold for enthusiastic funding, a benchmark based on robust AI safety work for relative comparison, and a maximum willingness-to-pay threshold for quickly rejecting implausibly inefficient interventions, while emphasizing that these estimates are subjective, uncertain, and should be revisable.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.