I think the argument in the section āA meta-option: Funding research into longtermist intervention prospectsā is important and is sometimes overlooked by non-longtermists.
Hereās a somewhat streamlined version of the sectionās key claims:
let us suppose instead, for the sake of argument, that some reasonable credences do not assign higher expected cost-effectiveness to any particular one of the proposed longtermist interventions than they do to the best short-termist interventions, because of the thinness of the case in support of each such intervention. [...]
It does not follow that the credences in question would recommend funding short-termist interventions. That is because Shivani also has what we might call a āsecond-orderā longtermist option: funding research into the cost-effectiveness of various possible attempts to influence the very long run, such as those discussed above. Provided that subsequent philanthropists would take due note of the results of such research, this second-order option could easily have higher expected value (relative to Shivaniās current probabilities) than the best short-termist option, since it could dramatically increase the expected effectiveness of future philanthropy (again, relative to Shivaniās current probabilities).
Finally, here is another option that is somewhat similar in spirit: rather than spending now, Shivani could save her money for a later time. [...] This fund would pay out whenever there comes a time when there is some action one could take that will, in expectation, sufficiently affect the value of the very long-run future.
These two considerations show that the bar for empirical objections to our argument to meet is very high. Not only would it need to be the case that, out of all the (millions) of actions available to an actor like Shivani, for none of them should one have non-negligible credence that one can positively affect the expected value of the long-run future by any non-negligible amount. It would also need to be the case that one should be virtually certain that there will be no such actions in the future, and that there is almost no hope of discovering any such actions through further research. This constellation of conditions seems highly unlikely.
Roughly the same argument has often come to my mind as well as one of the strongest arguments for at least doing longtermist research, even if one felt that all object-level longtermist interventions that have been proposed so far are too speculative. (Iād guess that I didnāt independently come up with the argument, but rather heard a version of it somewhere else.)
One thing Iād add is that one could also do cross-cutting work, such as work on the epistemic challenge to longtermism, rather than just work to better evaluate the cost-effectiveness of specific interventions or classes of interventions.
It might be too difficult to ever identify ahead of time a long-termist intervention as robustly good, due to the absence of good feedback and skepticism, cluelessness or moral uncertainty.
Cross-cutting work, if public especially, can also benefit others with goals/āvalues unaligned with your own and do more harm than good. More generally, resources and capital, including knowledge, you try to build can also end up in the wrong hands eventually, which undermines patient philanthropy, too.
Given that you said ārobustlyā in your first point, it might be that youāre adopting something like risk-neutrality or another alternative to expected value theory. If so, Iād say that:
That in itself is a questionable assumption, and people could do more work on which decision theory we should use.
I personally lean more towards just expected value theory (but with this incorporating skeptical priors, adjusting for the optimiserās curse, etc.), at least in situations that donāt involve āfanaticismā. But I acknowledge uncertainty on that front too.
If you just meant āIt might be too difficult to ever identify ahead of time a long-termist intervention as better in expectation than short-termist interventionsā, then yeah, I think this might be true (at least if fanaticism in the philosophical sense is bad, which seems to be an open question). But I think we actually have extremely little evidence for this claim.
We know from Tetlockās work that some people can do better than chance at forecasts over the range of months and years.
We seem to have basically no evidence about how well people who are actually trying (and especially ones aware of Tetlockās work) do on forecasts over much longer timescales (so we donāt have specific evidence that theyāll do well or that theyāll do badly).
We have a scrap of evidence suggesting that forecasting accuracy declines as the range increases, but relatively slowly (though this was comparing a few months to about a year; ).
So currently it seems to me that our best guess should be that forecasting accuracy continues to decline, but doesnāt hit zero, although maybe it asymptotes to it eventually.
That decline might be sharp enough to offset the increased āscaleā of the future, or might not, depending both on various empirical assumptions and on whether we accept or reject āfanaticismā (see Tarsneyās epistemic challenge paper).
I agree that basically all interventions have downside risks, and that one notable category of downside risks is the risk that resources/ācapital/āknowledge/āwhatever end up being used for bad things by other people. (This could be because they have bad goals or because they have good goals but bad plans; see also.) I think this will definitely mean we should deprioritise some otherwise plausible longtermist interventions. I also agree that it might undermine strong longtermism as a whole, but that seems very unlikely to me.
One reason is that similar points also apply to short-termist interventions.
Another is that it seems very likely that, if we try, we can make it more likely that the resources end up in the hands of people who will (in expectation) use them well, rather than in the hands of people who will (in expectation) use them poorly.
We can also model these downside risks.
We havenāt done this in detail yet as far as Iām aware
But we have come up with a bunch of useful concepts and frameworks for that (e.g., information hazards, unilateralistās curse, this post of mine [hopefully thatās useful!])
(All that said, you did just say āTwo possible objectionsā, and I do think pointing out possible objections is a useful part of the cause prioritisation project.)
I basically agree with those two points, but also think they donāt really defeat the case for strong longtermism, or at least for e.g. some tens or hundreds or thousands of people doing āsecond- or third-orderā research on these things.
This research could, for example, attempt to:
flesh out the two points you raised
quantify how much those points reduce the value of second- or third-order research into longtermism
consider whether there are any approaches to first- or second- or third-order longtermism-related work that donāt suffer those objections, or suffer them less
Itās hard to know how to count these things, but, off the top of my head, Iād estimate that:
something like 50-1000 people have done serious, focused work to identify high-priority longtermist interventions
fewer have done serious, focused work to evaluate the cost-effectiveness of those interventions, or to assess arguments for and against longtermism (e.g., work like this paper or Tarsneyās epistemic challenge paper)
So I think we should see āstrong longtermism actually isnāt right, e.g. due to the epistemic challengeā as a live hypothesis, but that it does seem too early to say weāve concluded that or that weāve concluded itās not worth looking into. It seems that weāre sufficiently uncertain, the potential stakes are sufficiently high, and the questions have been looked into sufficiently little that, whether weāre leaning towards thinking strong longtermism is true or that itās false, itās worth having at least some people doing serious, focused work to ādouble-checkā.
I think the argument in the section āA meta-option: Funding research into longtermist intervention prospectsā is important and is sometimes overlooked by non-longtermists.
Hereās a somewhat streamlined version of the sectionās key claims:
Roughly the same argument has often come to my mind as well as one of the strongest arguments for at least doing longtermist research, even if one felt that all object-level longtermist interventions that have been proposed so far are too speculative. (Iād guess that I didnāt independently come up with the argument, but rather heard a version of it somewhere else.)
One thing Iād add is that one could also do cross-cutting work, such as work on the epistemic challenge to longtermism, rather than just work to better evaluate the cost-effectiveness of specific interventions or classes of interventions.
Two possible objections:
It might be too difficult to ever identify ahead of time a long-termist intervention as robustly good, due to the absence of good feedback and skepticism, cluelessness or moral uncertainty.
Cross-cutting work, if public especially, can also benefit others with goals/āvalues unaligned with your own and do more harm than good. More generally, resources and capital, including knowledge, you try to build can also end up in the wrong hands eventually, which undermines patient philanthropy, too.
On your specific points:
Given that you said ārobustlyā in your first point, it might be that youāre adopting something like risk-neutrality or another alternative to expected value theory. If so, Iād say that:
That in itself is a questionable assumption, and people could do more work on which decision theory we should use.
I personally lean more towards just expected value theory (but with this incorporating skeptical priors, adjusting for the optimiserās curse, etc.), at least in situations that donāt involve āfanaticismā. But I acknowledge uncertainty on that front too.
If you just meant āIt might be too difficult to ever identify ahead of time a long-termist intervention as better in expectation than short-termist interventionsā, then yeah, I think this might be true (at least if fanaticism in the philosophical sense is bad, which seems to be an open question). But I think we actually have extremely little evidence for this claim.
We know from Tetlockās work that some people can do better than chance at forecasts over the range of months and years.
We seem to have basically no evidence about how well people who are actually trying (and especially ones aware of Tetlockās work) do on forecasts over much longer timescales (so we donāt have specific evidence that theyāll do well or that theyāll do badly).
We have a scrap of evidence suggesting that forecasting accuracy declines as the range increases, but relatively slowly (though this was comparing a few months to about a year; ).
So currently it seems to me that our best guess should be that forecasting accuracy continues to decline, but doesnāt hit zero, although maybe it asymptotes to it eventually.
That decline might be sharp enough to offset the increased āscaleā of the future, or might not, depending both on various empirical assumptions and on whether we accept or reject āfanaticismā (see Tarsneyās epistemic challenge paper).
I agree that basically all interventions have downside risks, and that one notable category of downside risks is the risk that resources/ācapital/āknowledge/āwhatever end up being used for bad things by other people. (This could be because they have bad goals or because they have good goals but bad plans; see also.) I think this will definitely mean we should deprioritise some otherwise plausible longtermist interventions. I also agree that it might undermine strong longtermism as a whole, but that seems very unlikely to me.
One reason is that similar points also apply to short-termist interventions.
Another is that it seems very likely that, if we try, we can make it more likely that the resources end up in the hands of people who will (in expectation) use them well, rather than in the hands of people who will (in expectation) use them poorly.
We can also model these downside risks.
We havenāt done this in detail yet as far as Iām aware
But we have come up with a bunch of useful concepts and frameworks for that (e.g., information hazards, unilateralistās curse, this post of mine [hopefully thatās useful!])
And thereās been some basic analysis and estimation for some relevant things. e.g. in relation to āpunting to the futureā
(All that said, you did just say āTwo possible objectionsā, and I do think pointing out possible objections is a useful part of the cause prioritisation project.)
I basically agree with those two points, but also think they donāt really defeat the case for strong longtermism, or at least for e.g. some tens or hundreds or thousands of people doing āsecond- or third-orderā research on these things.
This research could, for example, attempt to:
flesh out the two points you raised
quantify how much those points reduce the value of second- or third-order research into longtermism
consider whether there are any approaches to first- or second- or third-order longtermism-related work that donāt suffer those objections, or suffer them less
Itās hard to know how to count these things, but, off the top of my head, Iād estimate that:
something like 50-1000 people have done serious, focused work to identify high-priority longtermist interventions
fewer have done serious, focused work to evaluate the cost-effectiveness of those interventions, or to assess arguments for and against longtermism (e.g., work like this paper or Tarsneyās epistemic challenge paper)
So I think we should see āstrong longtermism actually isnāt right, e.g. due to the epistemic challengeā as a live hypothesis, but that it does seem too early to say weāve concluded that or that weāve concluded itās not worth looking into. It seems that weāre sufficiently uncertain, the potential stakes are sufficiently high, and the questions have been looked into sufficiently little that, whether weāre leaning towards thinking strong longtermism is true or that itās false, itās worth having at least some people doing serious, focused work to ādouble-checkā.