Terminology proposal: a class-n (or tier-n) megaproject reduces x-risk by between 10^-n and 10^-(n+1). This is intended as a short way to talk about the scale of longtermist megaprojects, inspired by 80k’s scale-scale but a bit cleaner because people can actually remember how to use it.
Class-0 project: reduces x-risk by >10%, e.g. creating 1,000 new AI safety researchers as good as Paul Christiano
Class-1 project: reduces x-risk by 1-10%, e.g. reducing pandemic risk to zero
Class-2 project: reduces x-risk by 0.1-1%, e.g. the Anthropic interpretability team
Class-3 project: reduces x-risk by 0.01-0.1%, e.g. most of these, though some make it into class 2
The classes could also be non-integer for extra precision, so if I thought creating 1,000 Paul Christianos reduced x-risk by 20%, I could call it a -log10(20%) = class-0.70 megaproject.
I’m still not sure about some details, so leave a comment if you have opinions:
“class” vs “tier”
I originally thought of having the percentages be absolute, but perhaps one could also make the case for relative percentages.
should class-n be between 10^-n and 10^-(n+1), or between 10^-(n-1) and 10^-n?
Are we evaluating outcomes or projects? What should the tier of a project be with a 10% chance to produce some outcome that reduces x-risk by 2%? I think it’s a class-1 (2%) project and the likelihood to succeed falls under tractability or neglectedness.
Terminology proposal: a class-n (or tier-n) megaproject reduces x-risk by between 10^-n and 10^-(n+1). This is intended as a short way to talk about the scale of longtermist megaprojects, inspired by 80k’s scale-scale but a bit cleaner because people can actually remember how to use it.
Class-0 project: reduces x-risk by >10%, e.g. creating 1,000 new AI safety researchers as good as Paul Christiano
Class-1 project: reduces x-risk by 1-10%, e.g. reducing pandemic risk to zero
Class-2 project: reduces x-risk by 0.1-1%, e.g. the Anthropic interpretability team
Class-3 project: reduces x-risk by 0.01-0.1%, e.g. most of these, though some make it into class 2
The classes could also be non-integer for extra precision, so if I thought creating 1,000 Paul Christianos reduced x-risk by 20%, I could call it a -log10(20%) = class-0.70 megaproject.
I’m still not sure about some details, so leave a comment if you have opinions:
“class” vs “tier”
I originally thought of having the percentages be absolute, but perhaps one could also make the case for relative percentages.
should class-n be between 10^-n and 10^-(n+1), or between 10^-(n-1) and 10^-n?
Are we evaluating outcomes or projects? What should the tier of a project be with a 10% chance to produce some outcome that reduces x-risk by 2%? I think it’s a class-1 (2%) project and the likelihood to succeed falls under tractability or neglectedness.
edit: changed my opinion on last bullet point