I also mentally noted that âdrastically lessâ was ambiguous, though for the sake of my quick forecasts I decided that whether you meant (or whether others would interpret you as meaning) â5% lessâ or â90% lessâ didnât really matter to my forecasts, so I didnât bother commenting.
Yeah, a big part of why I left the term vague is because I didnât want people to get hung up on those details when many AGI catastrophe scenarios are extreme enough to swamp those details. E.g., focusing on whether the astronomical loss threshold is 80% vs. 50% is besides the point if you think AGI failure almost always means losing 98+% of the futureâs value.
I might still do it differently if I could re-run the survey, however. It would be nice to have a number, so we could more easily do EV calculations.
Iâd be interested in seeing operationalizations at some subset of {1%, 10%, 50%, 90, 99%}.* I can imagine that most safety researchers will give nearly identical answers to all of them, but I can also imagine that large divergences, so decent value of information here.
*Probably canât do all 5, at least not at once, because of priming effects.
I also mentally noted that âdrastically lessâ was ambiguous, though for the sake of my quick forecasts I decided that whether you meant (or whether others would interpret you as meaning) â5% lessâ or â90% lessâ didnât really matter to my forecasts, so I didnât bother commenting.
Yeah, a big part of why I left the term vague is because I didnât want people to get hung up on those details when many AGI catastrophe scenarios are extreme enough to swamp those details. E.g., focusing on whether the astronomical loss threshold is 80% vs. 50% is besides the point if you think AGI failure almost always means losing 98+% of the futureâs value.
I might still do it differently if I could re-run the survey, however. It would be nice to have a number, so we could more easily do EV calculations.
Iâd be interested in seeing operationalizations at some subset of {1%, 10%, 50%, 90, 99%}.* I can imagine that most safety researchers will give nearly identical answers to all of them, but I can also imagine that large divergences, so decent value of information here.
*Probably canât do all 5, at least not at once, because of priming effects.