I think the argument in the section “A meta-option: Funding research into longtermist intervention prospects” is important and is sometimes overlooked by non-longtermists.
Here’s a somewhat streamlined version of the section’s key claims:
let us suppose instead, for the sake of argument, that some reasonable credences do not assign higher expected cost-effectiveness to any particular one of the proposed longtermist interventions than they do to the best short-termist interventions, because of the thinness of the case in support of each such intervention. [...]
It does not follow that the credences in question would recommend funding short-termist interventions. That is because Shivani also has what we might call a “second-order” longtermist option: funding research into the cost-effectiveness of various possible attempts to influence the very long run, such as those discussed above. Provided that subsequent philanthropists would take due note of the results of such research, this second-order option could easily have higher expected value (relative to Shivani’s current probabilities) than the best short-termist option, since it could dramatically increase the expected effectiveness of future philanthropy (again, relative to Shivani’s current probabilities).
Finally, here is another option that is somewhat similar in spirit: rather than spending now, Shivani could save her money for a later time. [...] This fund would pay out whenever there comes a time when there is some action one could take that will, in expectation, sufficiently affect the value of the very long-run future.
These two considerations show that the bar for empirical objections to our argument to meet is very high. Not only would it need to be the case that, out of all the (millions) of actions available to an actor like Shivani, for none of them should one have non-negligible credence that one can positively affect the expected value of the long-run future by any non-negligible amount. It would also need to be the case that one should be virtually certain that there will be no such actions in the future, and that there is almost no hope of discovering any such actions through further research. This constellation of conditions seems highly unlikely.
Roughly the same argument has often come to my mind as well as one of the strongest arguments for at least doing longtermist research, even if one felt that all object-level longtermist interventions that have been proposed so far are too speculative. (I’d guess that I didn’t independently come up with the argument, but rather heard a version of it somewhere else.)
One thing I’d add is that one could also do cross-cutting work, such as work on the epistemic challenge to longtermism, rather than just work to better evaluate the cost-effectiveness of specific interventions or classes of interventions.
It might be too difficult to ever identify ahead of time a long-termist intervention as robustly good, due to the absence of good feedback and skepticism, cluelessness or moral uncertainty.
Cross-cutting work, if public especially, can also benefit others with goals/values unaligned with your own and do more harm than good. More generally, resources and capital, including knowledge, you try to build can also end up in the wrong hands eventually, which undermines patient philanthropy, too.
Given that you said “robustly” in your first point, it might be that you’re adopting something like risk-neutrality or another alternative to expected value theory. If so, I’d say that:
That in itself is a questionable assumption, and people could do more work on which decision theory we should use.
I personally lean more towards just expected value theory (but with this incorporating skeptical priors, adjusting for the optimiser’s curse, etc.), at least in situations that don’t involve “fanaticism”. But I acknowledge uncertainty on that front too.
If you just meant “It might be too difficult to ever identify ahead of time a long-termist intervention as better in expectation than short-termist interventions”, then yeah, I think this might be true (at least if fanaticism in the philosophical sense is bad, which seems to be an open question). But I think we actually have extremely little evidence for this claim.
We know from Tetlock’s work that some people can do better than chance at forecasts over the range of months and years.
We seem to have basically no evidence about how well people who are actually trying (and especially ones aware of Tetlock’s work) do on forecasts over much longer timescales (so we don’t have specific evidence that they’ll do well or that they’ll do badly).
We have a scrap of evidence suggesting that forecasting accuracy declines as the range increases, but relatively slowly (though this was comparing a few months to about a year; ).
So currently it seems to me that our best guess should be that forecasting accuracy continues to decline, but doesn’t hit zero, although maybe it asymptotes to it eventually.
That decline might be sharp enough to offset the increased “scale” of the future, or might not, depending both on various empirical assumptions and on whether we accept or reject “fanaticism” (see Tarsney’s epistemic challenge paper).
I agree that basically all interventions have downside risks, and that one notable category of downside risks is the risk that resources/capital/knowledge/whatever end up being used for bad things by other people. (This could be because they have bad goals or because they have good goals but bad plans; see also.) I think this will definitely mean we should deprioritise some otherwise plausible longtermist interventions. I also agree that it might undermine strong longtermism as a whole, but that seems very unlikely to me.
One reason is that similar points also apply to short-termist interventions.
Another is that it seems very likely that, if we try, we can make it more likely that the resources end up in the hands of people who will (in expectation) use them well, rather than in the hands of people who will (in expectation) use them poorly.
We can also model these downside risks.
We haven’t done this in detail yet as far as I’m aware
But we have come up with a bunch of useful concepts and frameworks for that (e.g., information hazards, unilateralist’s curse, this post of mine [hopefully that’s useful!])
(All that said, you did just say “Two possible objections”, and I do think pointing out possible objections is a useful part of the cause prioritisation project.)
I basically agree with those two points, but also think they don’t really defeat the case for strong longtermism, or at least for e.g. some tens or hundreds or thousands of people doing “second- or third-order” research on these things.
This research could, for example, attempt to:
flesh out the two points you raised
quantify how much those points reduce the value of second- or third-order research into longtermism
consider whether there are any approaches to first- or second- or third-order longtermism-related work that don’t suffer those objections, or suffer them less
It’s hard to know how to count these things, but, off the top of my head, I’d estimate that:
something like 50-1000 people have done serious, focused work to identify high-priority longtermist interventions
fewer have done serious, focused work to evaluate the cost-effectiveness of those interventions, or to assess arguments for and against longtermism (e.g., work like this paper or Tarsney’s epistemic challenge paper)
So I think we should see “strong longtermism actually isn’t right, e.g. due to the epistemic challenge” as a live hypothesis, but that it does seem too early to say we’ve concluded that or that we’ve concluded it’s not worth looking into. It seems that we’re sufficiently uncertain, the potential stakes are sufficiently high, and the questions have been looked into sufficiently little that, whether we’re leaning towards thinking strong longtermism is true or that it’s false, it’s worth having at least some people doing serious, focused work to “double-check”.
I think the argument in the section “A meta-option: Funding research into longtermist intervention prospects” is important and is sometimes overlooked by non-longtermists.
Here’s a somewhat streamlined version of the section’s key claims:
Roughly the same argument has often come to my mind as well as one of the strongest arguments for at least doing longtermist research, even if one felt that all object-level longtermist interventions that have been proposed so far are too speculative. (I’d guess that I didn’t independently come up with the argument, but rather heard a version of it somewhere else.)
One thing I’d add is that one could also do cross-cutting work, such as work on the epistemic challenge to longtermism, rather than just work to better evaluate the cost-effectiveness of specific interventions or classes of interventions.
Two possible objections:
It might be too difficult to ever identify ahead of time a long-termist intervention as robustly good, due to the absence of good feedback and skepticism, cluelessness or moral uncertainty.
Cross-cutting work, if public especially, can also benefit others with goals/values unaligned with your own and do more harm than good. More generally, resources and capital, including knowledge, you try to build can also end up in the wrong hands eventually, which undermines patient philanthropy, too.
On your specific points:
Given that you said “robustly” in your first point, it might be that you’re adopting something like risk-neutrality or another alternative to expected value theory. If so, I’d say that:
That in itself is a questionable assumption, and people could do more work on which decision theory we should use.
I personally lean more towards just expected value theory (but with this incorporating skeptical priors, adjusting for the optimiser’s curse, etc.), at least in situations that don’t involve “fanaticism”. But I acknowledge uncertainty on that front too.
If you just meant “It might be too difficult to ever identify ahead of time a long-termist intervention as better in expectation than short-termist interventions”, then yeah, I think this might be true (at least if fanaticism in the philosophical sense is bad, which seems to be an open question). But I think we actually have extremely little evidence for this claim.
We know from Tetlock’s work that some people can do better than chance at forecasts over the range of months and years.
We seem to have basically no evidence about how well people who are actually trying (and especially ones aware of Tetlock’s work) do on forecasts over much longer timescales (so we don’t have specific evidence that they’ll do well or that they’ll do badly).
We have a scrap of evidence suggesting that forecasting accuracy declines as the range increases, but relatively slowly (though this was comparing a few months to about a year; ).
So currently it seems to me that our best guess should be that forecasting accuracy continues to decline, but doesn’t hit zero, although maybe it asymptotes to it eventually.
That decline might be sharp enough to offset the increased “scale” of the future, or might not, depending both on various empirical assumptions and on whether we accept or reject “fanaticism” (see Tarsney’s epistemic challenge paper).
I agree that basically all interventions have downside risks, and that one notable category of downside risks is the risk that resources/capital/knowledge/whatever end up being used for bad things by other people. (This could be because they have bad goals or because they have good goals but bad plans; see also.) I think this will definitely mean we should deprioritise some otherwise plausible longtermist interventions. I also agree that it might undermine strong longtermism as a whole, but that seems very unlikely to me.
One reason is that similar points also apply to short-termist interventions.
Another is that it seems very likely that, if we try, we can make it more likely that the resources end up in the hands of people who will (in expectation) use them well, rather than in the hands of people who will (in expectation) use them poorly.
We can also model these downside risks.
We haven’t done this in detail yet as far as I’m aware
But we have come up with a bunch of useful concepts and frameworks for that (e.g., information hazards, unilateralist’s curse, this post of mine [hopefully that’s useful!])
And there’s been some basic analysis and estimation for some relevant things. e.g. in relation to “punting to the future”
(All that said, you did just say “Two possible objections”, and I do think pointing out possible objections is a useful part of the cause prioritisation project.)
I basically agree with those two points, but also think they don’t really defeat the case for strong longtermism, or at least for e.g. some tens or hundreds or thousands of people doing “second- or third-order” research on these things.
This research could, for example, attempt to:
flesh out the two points you raised
quantify how much those points reduce the value of second- or third-order research into longtermism
consider whether there are any approaches to first- or second- or third-order longtermism-related work that don’t suffer those objections, or suffer them less
It’s hard to know how to count these things, but, off the top of my head, I’d estimate that:
something like 50-1000 people have done serious, focused work to identify high-priority longtermist interventions
fewer have done serious, focused work to evaluate the cost-effectiveness of those interventions, or to assess arguments for and against longtermism (e.g., work like this paper or Tarsney’s epistemic challenge paper)
So I think we should see “strong longtermism actually isn’t right, e.g. due to the epistemic challenge” as a live hypothesis, but that it does seem too early to say we’ve concluded that or that we’ve concluded it’s not worth looking into. It seems that we’re sufficiently uncertain, the potential stakes are sufficiently high, and the questions have been looked into sufficiently little that, whether we’re leaning towards thinking strong longtermism is true or that it’s false, it’s worth having at least some people doing serious, focused work to “double-check”.