(I canât actually find those calculations in Thorstadâs paper, could you point them out to me? afaik he mostly looks at the value of fractional reduction in x-risk, while microdooms are an absolute reduction if I understand correctly? happy to be corrected or shown in the right direction!)
My concerns here are twofold:
1 - epistemological: Letâs say those numbers are correct from the Thorstad paper, that a microdoom has to cost â $32bn to be GiveWell cost-effective. The question is, how would we know this. In his recent post Paul Cristiano thinks that RSPs could lead to a â10x reductionâ in AI risk. How does he know this? Is this just a risk reduction this century? This decade? Is it a permanent reduction?
Itâs one thing to argue that under set of conditions X work on x-risk reduction is cost-effective as youâve done here. But Iâm more interested in the question of whether conditions X hold, because thatâs where the rubber hits the road. If those conditions donât hold, then thatâs why longtermism might not ground x-risk work.[1]
Thereâs also the question of persistence. I think the Thorstad model either assumes the persistence of x-risk reduction, or the persistence of a low-risk post-peril rate. But how could we possibly know that such considerations hold? It seems to me that cluelessness just obliterates grounding strong longtermist interventions here.
2 - practical: Letâs say that longtermist organisations/âfunds have actually answered criticisms in 1, and under their own lights have made these calculations. I want to see those calculations! How have they calculated the post-risk peril rate? Where is the support for their views on population growth? Do they assume that we colonise the galaxy by default or not? How do they get feedback that overcomes cluelessness objections?
And critically, I want to see why theyâve argued or assumed these things. afaict I canât find these calculations.
Iâm not being snarky here, I really want to be wrong on this and be shown robust calculations on longtermist work, but I havenât found any. I think the risk is that longtermist EA looks like any other charity/âmovement that funds âwhat they kinda they think is goodâ rather than what we actually know does good, which was meant to be the whole point of EA in the first place.
It could still be grounded for other reasons though, for example if x-risk reduction is highly correlated with GCR reduction, and the latter is of high value
(I canât actually find those calculations in Thorstadâs paper, could you point them out to me? afaik he mostly looks at the value of fractional reduction in x-risk, while microdooms are an absolute reduction if I understand correctly? happy to be corrected or shown in the right direction!)
He assumes 20% risk and a 10% relative risk reduction, which I translate into 2% absolute risk of doom, and then see the table on p12.
(I canât actually find those calculations in Thorstadâs paper, could you point them out to me? afaik he mostly looks at the value of fractional reduction in x-risk, while microdooms are an absolute reduction if I understand correctly? happy to be corrected or shown in the right direction!)
My concerns here are twofold:
1 - epistemological: Letâs say those numbers are correct from the Thorstad paper, that a microdoom has to cost â $32bn to be GiveWell cost-effective. The question is, how would we know this. In his recent post Paul Cristiano thinks that RSPs could lead to a â10x reductionâ in AI risk. How does he know this? Is this just a risk reduction this century? This decade? Is it a permanent reduction?
Itâs one thing to argue that under set of conditions X work on x-risk reduction is cost-effective as youâve done here. But Iâm more interested in the question of whether conditions X hold, because thatâs where the rubber hits the road. If those conditions donât hold, then thatâs why longtermism might not ground x-risk work.[1]
Thereâs also the question of persistence. I think the Thorstad model either assumes the persistence of x-risk reduction, or the persistence of a low-risk post-peril rate. But how could we possibly know that such considerations hold? It seems to me that cluelessness just obliterates grounding strong longtermist interventions here.
2 - practical: Letâs say that longtermist organisations/âfunds have actually answered criticisms in 1, and under their own lights have made these calculations. I want to see those calculations! How have they calculated the post-risk peril rate? Where is the support for their views on population growth? Do they assume that we colonise the galaxy by default or not? How do they get feedback that overcomes cluelessness objections?
And critically, I want to see why theyâve argued or assumed these things. afaict I canât find these calculations.
Iâm not being snarky here, I really want to be wrong on this and be shown robust calculations on longtermist work, but I havenât found any. I think the risk is that longtermist EA looks like any other charity/âmovement that funds âwhat they kinda they think is goodâ rather than what we actually know does good, which was meant to be the whole point of EA in the first place.
It could still be grounded for other reasons though, for example if x-risk reduction is highly correlated with GCR reduction, and the latter is of high value
He assumes 20% risk and a 10% relative risk reduction, which I translate into 2% absolute risk of doom, and then see the table on p12.