which might make longtermist interventions much less cost-effective than the most effective “neartermist” ones.
Why do you think this?
For some very rough maths (appologies in advance for any errors), even Thorstad’s paper (with a 2 century long time of perils, a 0.1% post-peril risk rate, no economic/population growth, no moral progress, people live for 100 years) suggests that reducing p(doom) by 2% is worth as much as saving 16x8billion lives—i.e. each microdoom is worth 6.4million lives. I think we can buy microdooms more cheaply than $5,000*6.4million = $32billion each.
(I can’t actually find those calculations in Thorstad’s paper, could you point them out to me? afaik he mostly looks at the value of fractional reduction in x-risk, while microdooms are an absolute reduction if I understand correctly? happy to be corrected or shown in the right direction!)
My concerns here are twofold:
1 - epistemological: Let’s say those numbers are correct from the Thorstad paper, that a microdoom has to cost ⇐ $32bn to be GiveWell cost-effective. The question is, how would we know this. In his recent post Paul Cristiano thinks that RSPs could lead to a ’10x reduction’ in AI risk. How does he know this? Is this just a risk reduction this century? This decade? Is it a permanent reduction?
It’s one thing to argue that under set of conditions X work on x-risk reduction is cost-effective as you’ve done here. But I’m more interested in the question of whether conditions X hold, because that’s where the rubber hits the road. If those conditions don’t hold, then that’s why longtermism might not ground x-risk work.[1]
There’s also the question of persistence. I think the Thorstad model either assumes the persistence of x-risk reduction, or the persistence of a low-risk post-peril rate. But how could we possibly know that such considerations hold? It seems to me that cluelessness just obliterates grounding strong longtermist interventions here.
2 - practical: Let’s say that longtermist organisations/funds have actually answered criticisms in 1, and under their own lights have made these calculations. I want to see those calculations! How have they calculated the post-risk peril rate? Where is the support for their views on population growth? Do they assume that we colonise the galaxy by default or not? How do they get feedback that overcomes cluelessness objections?
And critically, I want to see why they’ve argued or assumed these things. afaict I can’t find these calculations.
I’m not being snarky here, I really want to be wrong on this and be shown robust calculations on longtermist work, but I haven’t found any. I think the risk is that longtermist EA looks like any other charity/movement that funds ‘what they kinda they think is good’ rather than what we actually know does good, which was meant to be the whole point of EA in the first place.
It could still be grounded for other reasons though, for example if x-risk reduction is highly correlated with GCR reduction, and the latter is of high value
(I can’t actually find those calculations in Thorstad’s paper, could you point them out to me? afaik he mostly looks at the value of fractional reduction in x-risk, while microdooms are an absolute reduction if I understand correctly? happy to be corrected or shown in the right direction!)
He assumes 20% risk and a 10% relative risk reduction, which I translate into 2% absolute risk of doom, and then see the table on p12.
Why do you think this?
For some very rough maths (appologies in advance for any errors), even Thorstad’s paper (with a 2 century long time of perils, a 0.1% post-peril risk rate, no economic/population growth, no moral progress, people live for 100 years) suggests that reducing p(doom) by 2% is worth as much as saving 16x8billion lives—i.e. each microdoom is worth 6.4million lives. I think we can buy microdooms more cheaply than $5,000*6.4million = $32billion each.
(I can’t actually find those calculations in Thorstad’s paper, could you point them out to me? afaik he mostly looks at the value of fractional reduction in x-risk, while microdooms are an absolute reduction if I understand correctly? happy to be corrected or shown in the right direction!)
My concerns here are twofold:
1 - epistemological: Let’s say those numbers are correct from the Thorstad paper, that a microdoom has to cost ⇐ $32bn to be GiveWell cost-effective. The question is, how would we know this. In his recent post Paul Cristiano thinks that RSPs could lead to a ’10x reduction’ in AI risk. How does he know this? Is this just a risk reduction this century? This decade? Is it a permanent reduction?
It’s one thing to argue that under set of conditions X work on x-risk reduction is cost-effective as you’ve done here. But I’m more interested in the question of whether conditions X hold, because that’s where the rubber hits the road. If those conditions don’t hold, then that’s why longtermism might not ground x-risk work.[1]
There’s also the question of persistence. I think the Thorstad model either assumes the persistence of x-risk reduction, or the persistence of a low-risk post-peril rate. But how could we possibly know that such considerations hold? It seems to me that cluelessness just obliterates grounding strong longtermist interventions here.
2 - practical: Let’s say that longtermist organisations/funds have actually answered criticisms in 1, and under their own lights have made these calculations. I want to see those calculations! How have they calculated the post-risk peril rate? Where is the support for their views on population growth? Do they assume that we colonise the galaxy by default or not? How do they get feedback that overcomes cluelessness objections?
And critically, I want to see why they’ve argued or assumed these things. afaict I can’t find these calculations.
I’m not being snarky here, I really want to be wrong on this and be shown robust calculations on longtermist work, but I haven’t found any. I think the risk is that longtermist EA looks like any other charity/movement that funds ‘what they kinda they think is good’ rather than what we actually know does good, which was meant to be the whole point of EA in the first place.
It could still be grounded for other reasons though, for example if x-risk reduction is highly correlated with GCR reduction, and the latter is of high value
He assumes 20% risk and a 10% relative risk reduction, which I translate into 2% absolute risk of doom, and then see the table on p12.