Postdoc at the Digital Economy Lab, Stanford, and research affiliate at the Global Priorities Institute, Oxford. I’m slightly less ignorant about economic theory than about everything else.
trammell
Karma: 2,248
Thanks for saying a bit more about how you’re interpreting “scope of longtermism”. To be as concrete as possible, what I’m assuming is that we both read Thorstad as saying “a philanthropist giving money away so as to maximize the good from a classical utilitarian perspective” is typically outside the scope of decision-situations that are longtermist, but let me know if you read him differently on that. (I think it’s helpful to focus on this case because it’s simple, and the one G&M most clearly argue is longtermist on the basis of those two premises.)
It’s a tautology that the G&M conclusion that the above decision-situation is longtermist follows from the premises, and no, I wouldn’t expect a paper disputing the conclusion to argue against this tautology. I would expect it to argue, directly or indirectly, against the premises. And you’ve done just that: you’ve offered two perfectly reasonable arguments for why the G&M premise (ii) might be false, i.e. giving to PS/B612F might not actually do 2x as much good in the long term as the GiveWell charity in the short term. (1) In footnote 2, you point out that the chance of near-term x-risk from AI may be very high. (2) You say that the funding needs of asteroid monitoring sufficient to alert us to impending catastrophe are plausibly already met. You also suggest in footnote 3 that maybe NGOs will do a worse job of it than the government.
I won’t argue against any of these possibilities, since the topic of this particular comment thread is not how strong the case for longtermism is all things considered, but whether Thorstad’s “Scope of LTism” successfully responds to G&M’s argument. I really don’t think there’s much more to say. If there’s a place in “Scope of LTism” where Thorstad offers an argument against (i) or (ii), as you’ve done, I’m still not seeing it.
First, to clarify, Greaves and MacAskill don’t use the Spaceguard Survey as their example. They use giving to the Planetary Society or B612 Foundation as their example, which do similar work.
Could you spell out what you mean by “the actual scope of longtermism”? In everyday language this might sound like it means “the range of things it’s justifiable to work on for the sake of improving the long term”, or something like that, but that’s not what either Thorstad or Greaves and MacAskill mean by it. They mean [roughly; see G&M for the exact definition] the set of decision situations in which the overall best act does most of its good in the long term.
Long before either of these papers, people in EA (and of course elsewhere) had been making fuzzy arguments for and against propositions like “the best thing to do is to lower x-risk from AI because this will realize a vast and flourishing future”. The project G&M, DT, and other philosophers in this space were engaged in at the time was to go back and carefully, baby step by baby step, formalize the arguments that go into the various building blocks of these “the best thing to do is...” conclusions, so that it’s easier to identify which elements of the overall conclusion follow from which assumptions, how someone might agree with some elements but disagree with others, and so on. The “[scope of] longtermism” framing was deliberately defined broadly enough that it doesn’t make claims about what the best actions are: it includes the possibility that giving to the top GiveWell charity is the best act because of its long-term benefits (e.g. saving the life of a future AI safety researcher).
The Case offers a proof that if you accept the premises (i) giving to the top GiveWell charity is the way to do the most good in the short term and (ii) giving to PS/B612F does >2x more good [~all in the long term] than the GiveWell charity does in the short term, then you accept (iii) that the scope of longtermism includes every decision situation in which you’re giving money away. It also argues for premise (ii), semi-formally but not with anything like a proof.
Again, the whole point of doing these sorts of formalizations is that it helps to sharpen the debate: it shows that a response claiming that the scope of longtermism is actually narrow has to challenge one of those premises. All I’m pointing out is that Thorstad’s “Scope of Longtermism” doesn’t do that. You’ve done that here, which is great: maybe (ii) is false because giving to PS/B612F doesn’t actually do much good at all.
I’m not defending AI risk reduction, nor even longtermism. I’m arguing only that David Thorstad’s claim in “The Scope of Longtermism” was rebutted before it was written.
Almost all longtermists think that some interventions are better than asteroid monitoring. To be conservative and argue that longtermism is true even if one disagrees with the harder-to-quantify interventions most longtermists happen to favor, the Case uses an intervention with low but relatively precise impact, namely asteroid monitoring, and argues that it does more than 2x as much good in the long term as the top GiveWell charity does in the short term.
This is a non-sequitur. The “scope” which he claims is narrow is a scope of decision situations, and in every decision situation involving where to give a dollar, we can give it to asteroid monitoring efforts.
I know David well, and David, if you’re reading this, apologies if it comes across as a bit uncharitable. But as far as I’ve ever been able to see, every important argument he makes in any of his papers against longtermism or the astronomical value of x-risk reduction was refuted pretty unambiguously before it was written. An unfortunate feature of an objection that comes after its own rebuttal is that sometimes people familiar with the arguments will skim it and say “weird, nothing new here” and move on, and people encountering it for the first time will think no response has been made.
For example,[1] I think the standard response to his arguments in “The Scope of Longtermism” would just be the Greaves and MacAskill “Case for Strong Longtermism”.[2] The Case, in a nutshell, is that by giving to the Planetary Society or B612 Foundation to improve our asteroid/comet monitoring, we do more than 2x as much good in the long term, even on relatively low estimates of the value of the future, than giving to the top GiveWell charity does in the short term. So if you think GiveWell tells us the most cost-effective way to improve the short term, you have to think that, whenever your decision problem is “where to give a dollar”, the overall best action does more good in the long term than in the short term.
You can certainly disagree with this argument on various grounds—e.g. you can think that non-GiveWell charities do much more good in the short term, or that the value of preventing extinction by asteroid is negative, or for that matter that the Planetary Society or B612 Foundation will just steal the money—but not with the arguments David offers in “The Scope of Longtermism”.
His argument [again, in a nutshell] is that there are three common “scope-limiting phenomena”, i.e. phenomena that make it the case that the overall best action does more good in the long term than in the short term in relatively few decision situations. These are
rapid diminution (the positive impact of the action per unit time quickly falls to 0),
washing out (the long-term impact of the action has positive and negative features which are hard to predict and cancel out in expectation), and
option unawareness (there’s an action that would empirically have large long-term impact but we don’t know what it is).
He grants that when Congress was deciding what to do with the money that originally went into an asteroid monitoring program called the Spaceguard Survey, longtermism seems to have held. So he’s explicitly not relying on an argument that there isn’t much value to trying to prevent x-risk from asteroids. Nevertheless, he never addresses the natural follow-up regarding contributing to improved asteroid monitoring today.
Re (1), he cites Kelly (2019) and Sevilla (2021) as reasons to be skeptical of claims from the “persistence” literature about various distant cultural, technological, or military developments having had long-term effects on the arc of history. Granting this doesn’t affect the Case that whenever your decision problem is “where to give a dollar”, the overall best action does more good in the long term than in the short term.[3]
Re (2), he says that we often have only weak evidence about a given action’s impact on the long-term future. He defends this by pointing out (a) that attempts to forecast actions’ impacts on a >20 year timescale have a mixed track record, (b) that professional forecasters are often skeptical of the ability to make such forecasts, and (c) that the overall impact of an action on the value of world is typically composed of its impacts on various other variables (e.g. the number of people and how well-off they are), and since it’s hard to forecast any of these components, it’s typically even harder to forecast the action’s impact on value itself. None of this applies to the Case. We can grant that most actions have hard-to-predict long-term consequences, and that forecasters would recognize this, without denying that in most decision-situations (including all those where the question is where to give a dollar), there is one action that has long-term benefits more than 2x as great as the short-term benefits of giving to the top GiveWell charity: namely the action of giving to the Planetary Society or B612 Foundation. There is not a mixed track record of forecasting the >20 year impact of asteroid/comet monitoring, and no evidence that professional forecasters are skeptical of making such forecasts, and he implicitly grants that the complexity of forecasting its long-term impact on value isn’t an issue in this case when it comes to the Space Guard Survey.
Re (3), again, the claim the Case makes is that we have identified one such action.
- ^
I also emailed him about an objection to his “Existential Risk Pessimism and the Time of Perils” in November and followed up in February, but he’s responded only to say that he’s been too busy to consider it.
- ^
Which he cites! Note that Greaves and MacAskill defend a stronger view than the one I’m presenting here, in particular that all near-best actions do much more good in the long term than in the short term. But what David argues against is the weaker view I lay out here.
- ^
Incidentally, he cites the fact that “Hiroshima and Nagasaki returned to their pre-war population levels by the mid-1950s” as an especially striking illustration of lack of persistence. But as I mentioned to him at the time, it’s compatible with the possibility that those regions have some population path, and we “jumped back in time” on it, such that from now on the cities always have about as many people at t as they would have had at t+10. If so, bombing them could still have most of its effects in the future.
In Young’s case the exponent on ideas is one, and progress looks like log(log(researchers)). (You need to pay a fixed cost to make the good at all in a given period, so only if you go above that do you make positive progress.) See Section 2.2.
Peretto (2018) and Massari and Peretto (2025) have SWE models that I think do successfully avoid the knife-edge issue (or “linearity critique”), but at the cost of, in some sense, digging the hole deeper when it comes to the excess variety issue.
Thanks!
And yeah, that’s fair. One possible SWE-style story I sort of hint at there is that we have preferences like the ones I use in the horses paper; process efficiency for any given product grows exponentially with a fixed population; and there are fixed labor costs to producing any given product. In this case, it’s clear that measured GDP/capita growth will be exponential (but all “vertical”) with a fixed population. But if you set things up in just the right way, so that measured GDP always increases by the same proportion when the range of products increases by some marginal proportion, it will also be exponential with a growing population (“vertical”+”horizontal”).
But I think it’s hard to not have this all be a bit ad-hoc / knife-edge. E.g. you’ll typically have to start out ever less productive at making the new products, or else the contribution to real GDP of successive % increases in the product range will blow up: as you satiate in existing products, you’re willing to trade ever more of them for a proportional increase in variety. Alternatively, you can say that the range of products grows subexponentially when the population grows exponentially, because the fixed costs of the later products are higher.
Second-wave endogenous growth models and automation
A bit tangential, but I can’t help sharing a data point I came across recently on how prepared the US government currently is for advanced AI: our secretary of education apparently thinks it stands for “A1”, like the steak sauce (h/t). (On the bright side, of course, this is a department the administration is looking to shut down.)
(FYI though I think we’ve chatted about several new varieties issues that I think could come up in the event of a big change in “growth mode”, and this post is just about one of them.)
No worries, good to hear!
Thanks! People have certainly argued at least since Marx that if the people owning the capital get all the income, that will affect the state. I think more recent/quantitative work on this, e.g. by Stiglitz, has generally focused on the effects of inequality in wealth or income, rather than the effects of inequality via a high capital share per se. But this isn’t my area at all—ask your favorite LLM : )
The reference point argument is also about consumption inequality rather than what gives rise to it. My guess would be that if we all really get radical life extension and a huge quantity of amazing goods and services, that will probably for most people outweigh whatever jealousy comes with the knowledge that others got more, but who knows.In any event, my guess would be that even if the marginal product of labor stays high or rises following full automation, most people’s incomes will eventually come not from wages, but from interest on whatever investments they have (even if they started small) or from redistribution. And full automation could well trigger so much redistribution that income inequality shrinks, since it will remove one motivation for letting income inequality remain high today, namely that unlike with robots, taxing productive people too much can discourage them from working as much.
Great to hear, thanks!
As for the prediction—fair enough. Just to clarify though, I’m worried that the example makes it look like we need growth in the new good(s) to get this weird slow GDP growth result, but that’s not true. In case that’s the impression you got, this example illustrates how we can have superexponential growth in every good but (arbitrarily slow) exponential growth in GDP.
Here’s an example in which utility is additively separable, is identical for all goods, the productivity and quantity of all goods grow hyperbolically, and yet GDP grows exponentially.
Ok, fair enough—thanks for getting me to make it clearer :). So I guess the disagreement (if any remains, post-retitling/etc) is just about how plausible we think it is that the technological advances that accompany full automation will be accompanied by further technological advances that counterintuitively slow GDP growth through the “new-products-Baumol” mechanism illustrated here. I don’t think that’s so implausible, and hopefully the note I’ll write later will make it clearer where I’m coming from on that.
But this post isn’t aiming to argue for the plausibility, just the possibility. It seems to me that a lot of discussion of this issue hasn’t noticed that it’s even a theoretical possibility.
not being on track to produce Good 2 only happens in your model specifically because you define automation to be a thing that takes Good-2 productivity from 0 to something positive… Automation is usually understood to be something that increases the productivity of something that we could already produce at least a little of in principle
Okay, I’m happy to change the title to (a more concise version of) “the ambiguous effect of a technological advancement that achieves full automation, and also allows new goods to be introduced on GDP growth” if that would resolve the disagreement. [Update: have just changed the title and a few words of the body text; let me know.]
On the second point: in practice I don’t think we have additively separable utility, and I don’t know what you mean by “extracting this from our utility function”. But anyway, if I’m understanding you, that is wrong: if your utility function is additively separable with an upper bound in each good, say , a technological shift can yield superexponential growth in the quantity of each n but exponential GDP growth. I’ll write up a note on how that works this evening if that would be helpful, but I was hoping this post could just be a maximally simple illustration of the more limited point that Baumol-like effects can slow growth even past the point of full automation.
Right, I’m just not taking a stand here.
I might be more pessimistic than you about wages on balance, since I would argue for the importance of the “reallocation of capital from labor-augmenting to non-labor-augmenting uses” point, which if strong enough could lower wages through a channel other than the DRS+PS one you focus on.
Hey Jess, thanks for the thoughtful comments.
On whether “this is still basically Baumol”
If we make that one tiny tweak and say that good 2 was out there to be made all along, just too expensive to be demanded, then yes, it’s the same! That was the goal of the example: to introduce a Baumol-like effect in a way so similar to how everyone agrees Baumol effects have played out historically that it’s really easy to see what’s going on.
I’m happy to say it’s basically Baumol. The distinction I think is worth emphasizing here is that, when people say “Baumol effects could slow down the growth effects of AI”, they are usually—I think always, in my experience—pointing to the fact that if
AI dramatically speeds productivity growth on most goods but leaves some that only humans can produce (say, artisanal handicrafts), and
consumers see those goods as not very substitutable for the goods we’re getting way more productive at,
then GDP growth won’t speed up much. This then invites the response that, when we look around, there doesn’t seem to be any human-only good strongly satisfying (1) and (2). My point is that the existence of a human-only good satisfying (1) and (2) is unnecessary: the very same effect can arise even given true full automation, not due to a limitation of our ability to fully automate, but due to the fact that a technological advance can encompass full automation and go beyond yielding a world where we can produce way more of everything we would ever produce without the advance, by letting us produce some goods we otherwise wouldn’t have been on track to produce at all. This has not been widely appreciated.
On whether there is any reason to expect the productivity acceleration to coincide with the “kink in the utility function”
Here I think I disagree with you more substantively, though maybe the disagreement stems from the small framing point above.
If indeed “good 2” were always out there, just waiting for its price to fall, and if a technology were coming that would just replace all our existing workers, factory parts, and innovators with versions that operate more quickly in equal proportion—so that we move along the same paths of technology and the quantity of each good produced, but more quickly—then I agree that the past would be a good guide to the future, and GDP growth would reliably rise a lot. The only way it wouldn’t would be if the goods we were just about to start producing anyway were like good 2, featuring less steeply diminishing marginal utility but way slower productivity growth, so that the rise in productivity growth across the board coincidentally turned up at the same time as the “utility function kink”.
But if the technological advances that are allowing us to automate the production of everything people would ever be able to produce without the advances are also what allow for the invention of goods like good 2, it wouldn’t be a coincidence. I.e. presumably full automation will coincide with not only a big increase in productivity growth (which raises GDP growth, in the absence of a random “utility function kink”) but also a big change in the direction of productivity growth, including via making new products available (which introduces the kind of “utility function kink” that has an arbitrary effect on GDP growth). The idea that we’re soon producing very different products than we otherwise ever would have, whose productivity is growing at very different rates, seems all the more likely to me when we remember that even at 30% growth we’re soon in an economy several orders of magnitude bigger: the kink just needs to show up somewhere, not anywhere near the current margin.
To reiterate what I noted at the beginning though, I’d be surprised if the ambiguous second effect single-handedly outweighed the unambiguously positive first effect. And it could just as well amplify it, if “good 2” exhibits faster than average productivity growth.
Fair enough, I think the lack of a direct response has been due to an interaction between the two things. At first, people familiar with the existing arguments didn’t see much to respond to in David’s arguments, and figured most people would see through them. Later, when David’s arguments had gotten around more and it became clear that a response would be worthwhile (and for that matter when new arguments had been made which were genuinely novel), the small handful of people who had been exploring the case for longtermism had mostly moved on to other projects.
I would disagree a bit about why they moved on, though: my impression is that the bad association with FTX the word “longtermism” got was only slightly responsible for their shift in focus, and the main driver was just that faster-than-expected AI progress mostly convinced them that the most valuable philosophy work to be done was more directly AI-related.