Skeptic says “longtermism is false because premises X don’t hold in case Y.”
Defender says “maybe X doesn’t hold for Y, but it holds for case Z, so longtermism is true. And also Y is better than Z so we prioritize Y.”
What is being proven here? The prevailing practice of longtermism (AI risk reduction) is being defended by a case whose premises are meaningfully different from the prevailing practice. It feels like a motte and bailey.
I’m not defending AI risk reduction, nor even longtermism. I’m arguing only that David Thorstad’s claim in “The Scope of Longtermism” was rebutted before it was written.
I don’t see how Thorstad’s claim that the Space Guard Survey is a “special case” of a strong longtermist priority being reasonable (and that other longtermist proposals did not have the same justification) is “rebutted” by the fact that Greaves and McAskill use the Space Guard Survey as its example. The actual scope of longtermism is clearly not restricted to observing exogenous risks with predictable regularity and identifiable and sustainable solutions, and thus is subject at least to some extent to the critiques Thorstad identified.
Even the case for the Space Guard Survey looks a lot weaker than Thorstad granted if one considers that the x-risk from AI in the near term is fairly significant, which most longtermists seem to agree with. Suddenly instead of it having favourable odds of enabling a vast future, it simply observes asteroids[1] for three decades before AI becomes so powerful that human ability to observe asteroids is irrelevant, and any positive value it supplies is plausibly swamped by alternatives like researching AI that doesn’t need big telescopes to predict asteroid trajectories and can prevent unfriendly AI and other x-risks. The problem is of course, that we don’t know what that best case solution looks like[2] and most longtermists think many areas of spending on AI look harmful rather than near best case, but don’t high certainty (or any consensus) about which areas those are. Which is Thorstad’s ‘washing out’ argument
As far as I can see, Thorstad’s core argument is that even if it’s [trivially] true that the theoretical best possible course of action has most of its consequences in the future, we don’t know what that course of action is or even near best solutions are. Given that most longtermists don’t think the canonical asteroid example is the best possible course of action and there’s widespread disagreement over whether actions like accelerating “safe” AI research are increasing or reducing risk, I don’t see his concession the Space Guard Survey might have merit under some assumptions as undermining that.
in theory it could even involve saving a child who grows up to be an AI researcher from malaria. This is improbable, but when you’re dealing with unpredictable phenomena with astronomical payoffs...
First, to clarify, Greaves and MacAskill don’t use the Spaceguard Survey as their example. They use giving to the Planetary Society or B612 Foundation as their example, which do similar work.
Could you spell out what you mean by “the actual scope of longtermism”? In everyday language this might sound like it means “the range of things it’s justifiable to work on for the sake of improving the long term”, or something like that, but that’s not what either Thorstad or Greaves and MacAskill mean by it. They mean [roughly; see G&M for the exact definition] the set of decision situations in which the overall best act does most of its good in the long term.
Long before either of these papers, people in EA (and of course elsewhere) had been making fuzzy arguments for and against propositions like “the best thing to do is to lower x-risk from AI because this will realize a vast and flourishing future”. The project G&M, DT, and other philosophers in this space were engaged in at the time was to go back and carefully, baby step by baby step, formalize the arguments that go into the various building blocks of these “the best thing to do is...” conclusions, so that it’s easier to identify which elements of the overall conclusion follow from which assumptions, how someone might agree with some elements but disagree with others, and so on. The “[scope of] longtermism” framing was deliberately defined broadly enough that it doesn’t make claims about what the best actions are: it includes the possibility that giving to the top GiveWell charity is the best act because of its long-term benefits (e.g. saving the life of a future AI safety researcher).
The Case offers a proof that if you accept the premises (i) giving to the top GiveWell charity is the way to do the most good in the short term and (ii) giving to PS/B612F does >2x more good [~all in the long term] than the GiveWell charity does in the short term, then you accept (iii) that the scope of longtermism includes every decision situation in which you’re giving money away. It also argues for premise (ii), semi-formally but not with anything like a proof.
Again, the whole point of doing these sorts of formalizations is that it helps to sharpen the debate: it shows that a response claiming that the scope of longtermism is actually narrow has to challenge one of those premises. All I’m pointing out is that Thorstad’s “Scope of Longtermism” doesn’t do that. You’ve done that here, which is great: maybe (ii) is false because giving to PS/B612F doesn’t actually do much good at all.
By “scope of longtermism” I took Thorstad’s reference to “class of decision situations” in terms of permutations to be evaluated (maximising welfare, maximising human proliferation, minimising suffering etc) rather than categories of basic actions (spending, voting, selecting clothing).[1] I’m not actually sure it makes a difference to my interpretation of the thrust of his argument (diminution, washing out and unawareness means solutions whose far future impact swamps short term benefits are vanishingly rare and generally unknowable) either way.
Sure, Thorstad absolutely starts off by conceding that under certain assumptions about the long term future,[2] a low probability but robustly positive action like preparing to stop asteroids from hitting earth which indirectly enables benefits to accrue over the very long term absolutely can be a valid priority.[3] But it doesn’t follow that one should prioritise the long term future in every decision making situation in which money is given away. The funding needs of asteroid monitoring sufficient to alert us to impending catastrophe are plausibly already met[4], and his core argument is we’re otherwise almost always clueless about what the [near] best solution for the long term future is. It’s not a particularly good heuristic to focus spending on outcomes you are most likely to be clueless about, and a standard approach to accumulation of uncertainty is to discount for it, which of course privileges the short term.
I mean, I agree that Thorstad makes no dent in arguments to the effect that if thereis an action which leads to positive utility sustained over a very long period of time for a very large number of people it will result in very high utility relative to actions which don’t have that impact: I’m not sure that argument is even falsifiable within a total utilitarian framework.[5] But I don’t think his intention is to argue with [near] tautologies, so much as to insist that the set of decisions which credibly result in robustly positive long term impact is small enough to usually be irrelevant.
Some of the implicit assumptions behind the salience of asteroid x-risk aren’t robust: if AI doomers are right then that massive positive future we’re trying to protect looks a lot smaller. On the other hand compared with almost any other x-risk scenario, asteroids are straightforward: we don’t have to factor in the possibility of asteroids becoming sneaky in response to us monitoring them, or attach much weight to the idea that informing people about asteroids will motivate them to try harder to make it hit the earth.
you correctly point out his choice of asteroid monitoring service is different from Greaves and MacAskill’s. I assume he does so partly to steelman the original, as the counterfactual impact of a government agency incubating the first large-scale asteroid monitoring programme is more robust than that of the marginal donation to NGOs providing additional analysis. And he doesn’t make this point, but I doubt the arguments that decided its funding actually depended on the very long term anyway....
Likewise, pretty much anyone familiar with total utilitarianism can conceive a credible scenario in which the highest total utility outcome would be to murder a particular individual (baby Hitler etc), and I don’t think it would be credible to insist such a situation could never occur or never be known. This would not, however, fatally weaken arguments against the principle of “murderism” that focused on doubting there were many decision situations where murder should be considered as a priority
Thanks for saying a bit more about how you’re interpreting “scope of longtermism”. To be as concrete as possible, what I’m assuming is that we both read Thorstad as saying “a philanthropist giving money away so as to maximize the good from a classical utilitarian perspective” is typically outside the scope of decision-situations that are longtermist, but let me know if you read him differently on that. (I think it’s helpful to focus on this case because it’s simple, and the one G&M most clearly argue is longtermist on the basis of those two premises.)
It’s a tautology that the G&M conclusion that the above decision-situation is longtermist follows from the premises, and no, I wouldn’t expect a paper disputing the conclusion to argue against this tautology. I would expect it to argue, directly or indirectly, against the premises. And you’ve done just that: you’ve offered two perfectly reasonable arguments for why the G&M premise (ii) might be false, i.e. giving to PS/B612F might not actually do 2x as much good in the long term as the GiveWell charity in the short term. (1) In footnote 2, you point out that the chance of near-term x-risk from AI may be very high. (2) You say that the funding needs of asteroid monitoring sufficient to alert us to impending catastrophe are plausibly already met. You also suggest in footnote 3 that maybe NGOs will do a worse job of it than the government.
I won’t argue against any of these possibilities, since the topic of this particular comment thread is not how strong the case for longtermism is all things considered, but whether Thorstad’s “Scope of LTism” successfully responds to G&M’s argument. I really don’t think there’s much more to say. If there’s a place in “Scope of LTism” where Thorstad offers an argument against (i) or (ii), as you’ve done, I’m still not seeing it.
Skeptic says “longtermism is false because premises X don’t hold in case Y.” Defender says “maybe X doesn’t hold for Y, but it holds for case Z, so longtermism is true. And also Y is better than Z so we prioritize Y.”
What is being proven here? The prevailing practice of longtermism (AI risk reduction) is being defended by a case whose premises are meaningfully different from the prevailing practice. It feels like a motte and bailey.
I’m not defending AI risk reduction, nor even longtermism. I’m arguing only that David Thorstad’s claim in “The Scope of Longtermism” was rebutted before it was written.
I don’t see how Thorstad’s claim that the Space Guard Survey is a “special case” of a strong longtermist priority being reasonable (and that other longtermist proposals did not have the same justification) is “rebutted” by the fact that Greaves and McAskill use the Space Guard Survey as its example. The actual scope of longtermism is clearly not restricted to observing exogenous risks with predictable regularity and identifiable and sustainable solutions, and thus is subject at least to some extent to the critiques Thorstad identified.
Even the case for the Space Guard Survey looks a lot weaker than Thorstad granted if one considers that the x-risk from AI in the near term is fairly significant, which most longtermists seem to agree with. Suddenly instead of it having favourable odds of enabling a vast future, it simply observes asteroids[1] for three decades before AI becomes so powerful that human ability to observe asteroids is irrelevant, and any positive value it supplies is plausibly swamped by alternatives like researching AI that doesn’t need big telescopes to predict asteroid trajectories and can prevent unfriendly AI and other x-risks. The problem is of course, that we don’t know what that best case solution looks like[2] and most longtermists think many areas of spending on AI look harmful rather than near best case, but don’t high certainty (or any consensus) about which areas those are. Which is Thorstad’s ‘washing out’ argument
As far as I can see, Thorstad’s core argument is that even if it’s [trivially] true that the theoretical best possible course of action has most of its consequences in the future, we don’t know what that course of action is or even near best solutions are. Given that most longtermists don’t think the canonical asteroid example is the best possible course of action and there’s widespread disagreement over whether actions like accelerating “safe” AI research are increasing or reducing risk, I don’t see his concession the Space Guard Survey might have merit under some assumptions as undermining that.
ex post, we know that so far it’s observed asteroids that haven’t hit us and won’t in the foreseen future.
in theory it could even involve saving a child who grows up to be an AI researcher from malaria. This is improbable, but when you’re dealing with unpredictable phenomena with astronomical payoffs...
First, to clarify, Greaves and MacAskill don’t use the Spaceguard Survey as their example. They use giving to the Planetary Society or B612 Foundation as their example, which do similar work.
Could you spell out what you mean by “the actual scope of longtermism”? In everyday language this might sound like it means “the range of things it’s justifiable to work on for the sake of improving the long term”, or something like that, but that’s not what either Thorstad or Greaves and MacAskill mean by it. They mean [roughly; see G&M for the exact definition] the set of decision situations in which the overall best act does most of its good in the long term.
Long before either of these papers, people in EA (and of course elsewhere) had been making fuzzy arguments for and against propositions like “the best thing to do is to lower x-risk from AI because this will realize a vast and flourishing future”. The project G&M, DT, and other philosophers in this space were engaged in at the time was to go back and carefully, baby step by baby step, formalize the arguments that go into the various building blocks of these “the best thing to do is...” conclusions, so that it’s easier to identify which elements of the overall conclusion follow from which assumptions, how someone might agree with some elements but disagree with others, and so on. The “[scope of] longtermism” framing was deliberately defined broadly enough that it doesn’t make claims about what the best actions are: it includes the possibility that giving to the top GiveWell charity is the best act because of its long-term benefits (e.g. saving the life of a future AI safety researcher).
The Case offers a proof that if you accept the premises (i) giving to the top GiveWell charity is the way to do the most good in the short term and (ii) giving to PS/B612F does >2x more good [~all in the long term] than the GiveWell charity does in the short term, then you accept (iii) that the scope of longtermism includes every decision situation in which you’re giving money away. It also argues for premise (ii), semi-formally but not with anything like a proof.
Again, the whole point of doing these sorts of formalizations is that it helps to sharpen the debate: it shows that a response claiming that the scope of longtermism is actually narrow has to challenge one of those premises. All I’m pointing out is that Thorstad’s “Scope of Longtermism” doesn’t do that. You’ve done that here, which is great: maybe (ii) is false because giving to PS/B612F doesn’t actually do much good at all.
By “scope of longtermism” I took Thorstad’s reference to “class of decision situations” in terms of permutations to be evaluated (maximising welfare, maximising human proliferation, minimising suffering etc) rather than categories of basic actions (spending, voting, selecting clothing).[1] I’m not actually sure it makes a difference to my interpretation of the thrust of his argument (diminution, washing out and unawareness means solutions whose far future impact swamps short term benefits are vanishingly rare and generally unknowable) either way.
Sure, Thorstad absolutely starts off by conceding that under certain assumptions about the long term future,[2] a low probability but robustly positive action like preparing to stop asteroids from hitting earth which indirectly enables benefits to accrue over the very long term absolutely can be a valid priority.[3] But it doesn’t follow that one should prioritise the long term future in every decision making situation in which money is given away. The funding needs of asteroid monitoring sufficient to alert us to impending catastrophe are plausibly already met[4], and his core argument is we’re otherwise almost always clueless about what the [near] best solution for the long term future is. It’s not a particularly good heuristic to focus spending on outcomes you are most likely to be clueless about, and a standard approach to accumulation of uncertainty is to discount for it, which of course privileges the short term.
I mean, I agree that Thorstad makes no dent in arguments to the effect that if there is an action which leads to positive utility sustained over a very long period of time for a very large number of people it will result in very high utility relative to actions which don’t have that impact: I’m not sure that argument is even falsifiable within a total utilitarian framework.[5] But I don’t think his intention is to argue with [near] tautologies, so much as to insist that the set of decisions which credibly result in robustly positive long term impact is small enough to usually be irrelevant.
all of which can be reframed in terms of making money to spend available to spend on priorities” in classic “hardcore EA” style anyway...
Some of the implicit assumptions behind the salience of asteroid x-risk aren’t robust: if AI doomers are right then that massive positive future we’re trying to protect looks a lot smaller. On the other hand compared with almost any other x-risk scenario, asteroids are straightforward: we don’t have to factor in the possibility of asteroids becoming sneaky in response to us monitoring them, or attach much weight to the idea that informing people about asteroids will motivate them to try harder to make it hit the earth.
you correctly point out his choice of asteroid monitoring service is different from Greaves and MacAskill’s. I assume he does so partly to steelman the original, as the counterfactual impact of a government agency incubating the first large-scale asteroid monitoring programme is more robust than that of the marginal donation to NGOs providing additional analysis. And he doesn’t make this point, but I doubt the arguments that decided its funding actually depended on the very long term anyway....
this is possibly another reason for his choice of asteroid monitoring service...
Likewise, pretty much anyone familiar with total utilitarianism can conceive a credible scenario in which the highest total utility outcome would be to murder a particular individual (baby Hitler etc), and I don’t think it would be credible to insist such a situation could never occur or never be known. This would not, however, fatally weaken arguments against the principle of “murderism” that focused on doubting there were many decision situations where murder should be considered as a priority
Thanks for saying a bit more about how you’re interpreting “scope of longtermism”. To be as concrete as possible, what I’m assuming is that we both read Thorstad as saying “a philanthropist giving money away so as to maximize the good from a classical utilitarian perspective” is typically outside the scope of decision-situations that are longtermist, but let me know if you read him differently on that. (I think it’s helpful to focus on this case because it’s simple, and the one G&M most clearly argue is longtermist on the basis of those two premises.)
It’s a tautology that the G&M conclusion that the above decision-situation is longtermist follows from the premises, and no, I wouldn’t expect a paper disputing the conclusion to argue against this tautology. I would expect it to argue, directly or indirectly, against the premises. And you’ve done just that: you’ve offered two perfectly reasonable arguments for why the G&M premise (ii) might be false, i.e. giving to PS/B612F might not actually do 2x as much good in the long term as the GiveWell charity in the short term. (1) In footnote 2, you point out that the chance of near-term x-risk from AI may be very high. (2) You say that the funding needs of asteroid monitoring sufficient to alert us to impending catastrophe are plausibly already met. You also suggest in footnote 3 that maybe NGOs will do a worse job of it than the government.
I won’t argue against any of these possibilities, since the topic of this particular comment thread is not how strong the case for longtermism is all things considered, but whether Thorstad’s “Scope of LTism” successfully responds to G&M’s argument. I really don’t think there’s much more to say. If there’s a place in “Scope of LTism” where Thorstad offers an argument against (i) or (ii), as you’ve done, I’m still not seeing it.