Thanks so much for writing this thought-provoking piece and for cross-posting it here to the forum for discussion. It is a long piece with lots of subtly different arguments that piece together in complex ways. Indeed, by the end of it, I’m not sure if I agree with the overall thrust or not. But there are many strands on which I do strongly agree or disagree and so I’ll try to break them out into different comments here to make the discussion easier.
Overall:
I’m enthusiastic about your work towards creating CBA for existential risks.
I’m sympathetic to the argument that concern about presently existing people is sufficient to warrant substantial government spending on existential risk (though I don’t think it quite gets there).
I’m also sympathetic to the idea that one should lead with this in policy circles (though I favour the view that one should sometimes lead with it and sometimes with future generations).
I strongly agree that we shouldn’t try to implement policies that are democratically infeasible (though it should be uncontroversial that it is sometimes correct to propose policies that go somewhat beyond current democratic feasibility, since feasibility is not fixed).
I strongly disagree with the idea that it is in some way undemocratic to argue that the effects on future generations give an especially strong reason to be concerned about existential risks (which I think your paper sometimes suggests).
I worry that your stated estimates, aren’t really enough to secure adequate funding on traditional CBA grounds.
For example, it doesn’t work in many countries. There are only a fifth as many people living in the UK, so only a fifth as many beneficiaries. Thus the benefits internalised by the UK are a fifth as high, so the cost-benefit ratio is only a fifth as good. Moreover, the UK is poorer, so people are less willing to pay to avoid risk and the VSLs used by the government are substantially lower (I think about a fifth your stated figure — £2M), such that the package you list would dramatically fail the CBA by a factor of 25 or so in the UK. Many other countries are poorer or smaller than the US in some combination and will have trouble meeting this cost-benefit bar on the estimates you gave.
Even in the US, your estimates for the cost-benefit calculation aren’t really enough to make it go through. It would be if all these interventions were inextricably bound up as a package, or if all funding on them had equal effect. But CBA cares about marginal cost effectiveness and presumably the package can be broken into chunks of differing ex-ante cost-effectiveness (e.g. by intervention type, or by tranches of funding in each intervention). Indeed you suggest this later in the piece. Since the average only just meets the bar, if there is much variation, the marginal work won’t meet the bar, so government funding would cap out at something less than this, perhaps substantially so.
Moreover, your analysis suggests the package only marginally meets the cost-benefit threshold and requires some educated guesswork to do so (the amounts it would lower existential risk). Such speculative estimates that drive a large funding choice are usually frowned on in CBA. So even if it is marginally worth funding if government attention and capacity were free, it wouldn’t be a high priority.
My guess is therefore that a national package of interventions by the US would need to be notably more modest than this one to get funded purely on traditional CBA grounds, and much more modest to get funded in the UK or elsewhere, widening the gap between what traditional CBA gets us and what the addition of longtermist considerations could achieve.
But CBA cares about marginal cost effectiveness and presumably the package can be broken into chunks of differing ex-ante cost-effectiveness (e.g. by intervention type, or by tranches of funding in each intervention). Indeed you suggest this later in the piece. Since the average only just meets the bar, if there is much variation, the marginal work won’t meet the bar, so government funding would cap out at something less than this, perhaps substantially so.
Yes, this is an important point. If we were to do a more detailed cost-benefit analysis of catastrophe-preventing interventions, we’d want to address it more comprehensively (especially since we also mention how different interventions can undermine each other elsewhere in the paper).
On the point about the average only just meeting the bar, though, I think it’s worth noting that our mainline calculation uses very conservative assumptions. In particular, we assume:
A total cost of $400b rounded up from $319.6b
That all the costs of the interventions are paid upfront and that the risk-reduction occurs only at the end of the decade
The lowest VSL figure used by the DoT
The highest discount rate recommended by OIRA
A 1-in-1,000 GCR-reduction from the whole suite
And we count only these interventions benefits in terms of GCR-reduction. We don’t count any of the benefits arising from these interventions reducing the risk of smaller catastrophes.
I think that, once you replace these conservative assumptions with more reasonable ones, it’s plausible that each intervention we propose would pass a CBA test.
I think that these points also help our conclusions apply to other countries, even though all other countries are either smaller than the US or employ a lower VSL. And even though reasonable assumptions will still imply that some GCR-reducing interventions are too expensive for some countries, it could be worthwhile for these countries to participate in a coalition that agrees to share the costs of the interventions.
I agree that speculative estimates are a major problem. Making these estimates less speculative – insofar as that can be done – seems to me like a high priority. In the meantime, I wonder if it would help to emphasise that a speculative-but-unbiased estimate of the risk is just as likely to be too low as to be too high.
I guess this shows that the case won’t get through with the conservative rounding off that you applied here, so future developments of this CBA would want to go straight for the more precise approximations in order to secure a higher evaluation.
Re the possibility of international agreements, I agree that they can make it easier to meet various CBA thresholds, but I also note that they are notoriously hard to achieve, even when in the interests of both parties. That doesn’t mean that we shouldn’t try, but if the CBA case relies on them then the claim that one doesn’t need to go beyond it (or beyond CBA-plus-AWTP) becomes weaker.
That said, I think some of our residual disagreement may be to do with me still not quite understanding what your paper is claiming. One of my concerns is that I’m worried that CBA-plus-AWTP is a weak style of argument — especially with elected politicians. That is, arguing for new policies (or treaties) on grounds of CBA-plus-AWTP has some sway for fairly routine choices made by civil servants who need to apply government cost-effectiveness tests, but little sway with voters or politicians. Indeed, many people who would be benefited by such cost-effectiveness tests are either bored by — or actively repelled by — such a methodology. But if you are arguing that we should only campaign for policies that would pass such a test, then I’m more sympathetic. In that case, we could still make the case for them in terms that will resonate more broadly.
What we’re arguing for is a criterion: governments should fund all those catastrophe-preventing interventions that clear the bar set by cost-benefit analysis and altruistic willingness to pay. One justification for funding these interventions is the justification provided by CBA itself, but it need not be the only one. If longtermist justifications help us get to the place where all the catastrophe-preventing interventions that clear the CBA-plus-AWTP bar are funded, then there’s a case for employing those justifications too.
which answers my final paragraph in the parent comment, and suggests that we are not too far apart.
I guess this shows that the case won’t get through with the conservative rounding off that you applied here, so future developments of this CBA would want to go straight for the more precise approximations in order to secure a higher evaluation.
And thanks again for making this point (and to weeatquince as well). I’ve written a new paragraph emphasising a more reasonable, less conservative estimate of benefit-cost ratios. I expect it’ll probably go in the final draft, and I’ll edit the post here to include it as well (just waiting on Carl’s approval).
Re the possibility of international agreements, I agree that they can make it easier to meet various CBA thresholds, but I also note that they are notoriously hard to achieve, even when in the interests of both parties. That doesn’t mean that we shouldn’t try, but if the CBA case relies on them then the claim that one doesn’t need to go beyond it (or beyond CBA-plus-AWTP) becomes weaker.
I think this is right (and I must admit that I don’t know that much about the mechanics and success-rates of international agreements) but one cause for optimism here is Cass Sunstein’s view about why the Montreal Protocol was such a success (see Chapter 2): cost-benefit analysis suggested that it would be in the US’s interest to implement unilaterally and that the benefit-cost ratio would be even more favourable if other countries signed on as well. In that respect, the Montreal Protocol seems akin to prospective international agreements to share the cost of GCR-reducing interventions.
Like Richard, I was quite confused while reading the piece about the use of this term. I agree with him that if you are trying to get normative mileage out of this idea, then it doesn’t work. In representative democracies like the US, we elect people to represent us in the highest levels of national decision-making. They are allowed to act in our narrow self-interest, but also in furtherance of our ideals. Indeed, they are aren’t constrained to leading from the back — to leading people to where they already want to go. They are also allowed to to lead from the front — to use good judgment to see things the public hadn’t yet seen, to show why it is an attractive course of action and to take it. Indeed, when we think of great Presidents or Prime Ministers it is often this quality that we most admire. So it seems entirely fine to me to appeal to people in government on moral grounds — including on longtermist grounds such as the effects on future generations. To say otherwise would involve some surprising judgments about other acts of moral leadership that are widely celebrated (such as in the ending of slavery).
I think the best version of this argument about democratic acceptability is not a moral or political philosophy argument (that it is wrong or unjust or undemocratic for governments to implement policies on these grounds) because there is little evidence it would be any of those things. Before a funding allocation reached those levels it would likely become politically unacceptable — i.e. it couldn’t practically be implemented and you would get less in the long run than if you asked for a more modest amount. I think there is broad agreement among longtermists that political feasibility is a key constraint, so the question is then about where it kicks in and what we can do to loosen the constraint by raising moral awareness.
At times it sounds like you agree with this, such as when you explicitly say “democratically unacceptable, by which we mean it could not be adopted and maintained by a democratic government”, but at other times you seem to trade on the idea that there is something democratically tainted about political advocacy on behalf of the people of the future — this is something I strongly reject. I think the paper would be better if it were clearer on this. I think using the term ‘democratically infeasible’ would have been more apposite and would dispel the confusion.
(I should add that part of this is the question of whether you mean infeasible vs normatively problematic, and part of it is the question of whether you are critiquing use of longtermist considerations (over and above CBA considerations) vs whether you are critiquing attempts to impose the policies one would come up with on strong longtermism + ignoring feasibility. I’ll address that more in another comment, but the short version is that I agree that imposing wildly unpopular policies on voters might be normatively problematic as well as infeasible, but think that comparison is a straw man.)
at other times you seem to trade on the idea that there is something democratically tainted about political advocacy on behalf of the people of the future — this is something I strongly reject.
I reject that too. We don’t mean to suggest that there is anything democratically tainted about that kind of advocacy. Indeed, we say that longtermists should advocate on behalf of future generations, in order to increase the present generation’s altruistic willingness to pay for benefits to future generations.
What we think would be democratically unacceptable is governments implementing policies that go significantly beyond the present generation’s altruistic willingness to pay. Getting governments to adopt such policies is infeasible, but we chose ‘unacceptable’ because we also think there would be something normatively problematic about it.
So I think we are addressing your strawman here. Certainly, we don’t take ourselves to be arguing against anyone in particular in saying that there would be something wrong with governments placing heavy burdens on the present generation for the sake of small reductions in existential risk. But it seems worth saying in any case, because I think it helps ward off misunderstandings from people not so familiar with longtermism (and we’re hoping to reach such people—especially policymakers—with this paper). For suppose that someone knows only of the following argument for longtermism: the expected future population is enormous, the lives of future people are good in expectation, and it is better if the future contains more good lives. Then that person might mistakenly think that the goal of longtermists in the political sphere must be something like a strong longtermist policy.
we chose ‘unacceptable’ because we also think there would be something normatively problematic about it.
I’m not so sure about that. I agree with you that it would be normatively problematic in the paradigm case of a policy that imposed extreme costs on current society for very slight reduction in total existential risk — let’s say, reducing incomes by 50% in order to lower risk by 1 part in 1 million.
But I don’t know that it is true in general.
First, consider a policy that was inefficient but small — e.g. one that cost $10 million to the US govt, but reduced the number of statistical lives lost in the US by only 0.1, I don’t think I’d say that this was democratically unacceptable. Policies like this are enacted all the time in safety contexts and are often inefficient and ill-thought-out, and I’m not generally in favour of them, but I don’t find them to be undemocratic. I suppose one could argue that all US policy that doesn’t pass a CBA is undemocratic (or democratically unacceptable), but that seems a stretch to me. So I wonder whether it is correct to count our intuitions on the extreme example as counting against all policies that are inefficient in traditional CBA terms or just against those that impose severe costs.
I wouldn’t call a small policy like that ‘democratically unacceptable’ either. I guess the key thing is whether a policy goes significantly beyond citizens’ willingness to pay not only by a large factor but also by a large absolute value. It seems likely to be the latter kinds of policies that couldn’t be adopted and maintained by a democratic government, in which case it’s those policies that qualify as democratically unacceptable on our definition.
Second, the argument overshoots. Given other plausible claims, building policy on this premise would not only lead governments to increase their efforts to prevent catastrophes. It would also lead them to impose extreme costs on the present generation for the sake of miniscule reductions in the risk of existential catastrophe.
I disagree with this.
First, I think that many moral views are compelled to find the possibility that their generation permanently eradicates all humans from the world to be especially bad and worthy of much extra effort to avoid. As I detailed in Chapter 2 of the Precipice, this can be based on considerations about the past or about the future. While longtermism is often associated with the future-directed reasons, I favour a broader definition. If someone is deeply moved by the Burkean partnership of the generations over an unbroken chain stretching back 10,000 generations and thinks this gives additional reason not to be the generation who breaks it, then I’m inclined to say they are a longtermist too. But whether it counts or doesn’t, my arguments in Chapter 2 still imply that many moral views are already committed to a special badness of extinction (and often other existential risks). This means there is a wide set of views that go beyond traditional CBA and I can’t see a good argument why they should all overshoot.
And what about for a longtermist view that is more typical of our community? Suppose we are committed to the idea that each person matters equally no matter when they would live. It doesn’t follow from this that the best policy is one that demands vast sacrifices from the current generation, anymore than this follows from the widely held view that all people matter equally regardless of race or place of birth. One could still have places in ethics or political philosophy where there are limits placed on the sacrifices that can be demanded of you, and especially limits placed on the sacrifices you can force others to endure in order to produce a larger amount of benefits for others. Theories with such limits could still be impartial in time and could definitely qualify as longtermist.
One could also have things tempered by moral uncertainty or political beliefs about pluralism or non-coercion.
And that is before we get to the fact that longtermist policy drafters don’t have to ignore the feasibility of their proposals — another clear way to stop before you overshoot.
I really don’t think it is clear that there are any serious policy suggestions from longtermists that do overshoot here. e.g. in The Precipice (p. 186) my advice on budget is:
We currently spend less than a thousandth of a percent of gross world product on them. Earlier, I suggested bringing this up by at least a factor of 100, to reach a point where the world is spending more on securing its potential than on ice cream, and perhaps a good longer-term target may be a full 1 percent.
And this doesn’t seem too different from your own advice ($400B spending by the US is 2% of a year’s GDP).
A different take might be that I and others could be commended for not going too far, but that in doing so we are being inconsistent with our stated principles. That is an interesting angle, and one raised by Jim Holt in his very good NYT book review. But I ultimately don’t think it works either: I can’t see any strong arguments that longtermism lacks the theoretical resources to consistently avoid overshooting.
The argument we mean to refer to here is the one that we call the ‘best-known argument’ elsewhere: the one that says that the non-existence of future generations would be an overwhelming moral loss because the expected future population is enormous, the lives of future people are good in expectation, and it is better if the future contains more good lives. We think that this argument is liable to overshoot.
I agree that there are other compelling longtermist arguments that don’t overshoot. But my concern is that governments can’t use these arguments to guide their catastrophe policy. That’s because these arguments don’t give governments much guidance in deciding where to set the bar for funding catastrophe-preventing interventions. They don’t answer the question, ‘By how much does an intervention need to reduce risks per $1 billion of cost in order to be worth funding?’.
We currently spend less than a thousandth of a percent of gross world product on them. Earlier, I suggested bringing this up by at least a factor of 100, to reach a point where the world is spending more on securing its potential than on ice cream, and perhaps a good longer-term target may be a full 1 percent.
And this doesn’t seem too different from your own advice ($400B spending by the US is 2% of a year’s GDP).
This seems like a good target to me, although note that $400b is our estimate for how much it would cost to fund our suite of interventions for a decade, rather than for a year.
Thanks, these comments are great! I’m planning to work through them later this week.
I agree with pretty much all of your bulletpoints. With regards to the last one, we didn’t mean to suggest that arguing for greater concern about existential risks is undemocratic. Instead, we meant to suggest that (in the world as it is today) it would be undemocratic for governments to implement polices that place heavy burdens on the present generation for the sake of small reductions in existential risk.
At times it seems like you are comparing traditional CBA justifications for government work on existential risk vs longtermist justifications. At other times it seems like you are comparing the policy prescriptions and funding that would pass a traditional CBA vs the policy prescriptions and funding that would come out of strong longtermist reasoning (deliberately setting aside political feasibility). I have different views on each of these comparisons.
On the first, I think we should use both traditional CBA justifications as well as longtermist considerations and don’t think the piece really offers much argument against including the latter at all (for one thing, it doesn’t seem to recognise that they can actually be popular and motivating with the public and with policy makers in a way that CBA isn’t, widening the feasibility window).
On the second, I agree that we should generally not use strong longtermist justifications (though the piece usually frames itself as arguing against regular longtermist justifications). For one thing, I don’t actually endorse strong longtermism anyway. I also agree that we shouldn’t set aside political feasibility in our policy recommendations. But it seems something of a straw man to suggest that the choice under discussion is to ignore effects on future generations or to consider all such effects on a total utilitarian basis and ignore the political feasibility. Are there any serious advocates of that position? The inadequacy of the second of these options doesn’t really help you conclude that we should deliberately restrict ourselves to the CBA option, as there are so many other options that were not considered.
On the first, I think we should use both traditional CBA justifications as well as longtermist considerations
I agree with this. What we’re arguing for is a criterion: governments should fund all those catastrophe-preventing interventions that clear the bar set by cost-benefit analysis and altruistic willingness to pay. One justification for funding these interventions is the justification provided by CBA itself, but it need not be the only one. If longtermist justifications help us get to the place where all the catastrophe-preventing interventions that clear the CBA-plus-AWTP bar are funded, then there’s a case for employing those justifications too.
But it seems something of a straw man to suggest that the choice under discussion is to ignore effects on future generations or to consider all such effects on a total utilitarian basis and ignore the political feasibility. Are there any serious advocates of that position?
We think that longtermists in the political sphere should (as far as they can) commit themselves to only pushing for policies that can be justified on CBA-plus-AWTP grounds (which need not entirely ignore effects on future generations). We think that, in the absence of such a commitment, the present generation may worry that longtermists would go too far. From the paper:
Longtermists can try to increase government funding for catastrophe-prevention by making longtermist arguments and thereby increasing citizens’ AWTP, but they should not urge governments to depart from a CBA-plus-AWTP catastrophe policy. On the contrary, longtermists should as far as possible commit themselves to acting in accordance with a CBA-plus-AWTP policy in the political sphere. One reason why is simple: longtermists have moral reasons to respect the preferences of their fellow citizens.
To see another reason why, note first that longtermists working to improve government catastrophe policy could be a win-win. The present generation benefits because longtermists solve the collective action problem: they work to implement interventions that cost-effectively reduce everyone’s risk of dying in a catastrophe. Future generations benefit because these interventions also reduce existential risk. But as it stands the present generation may worry that longtermists would go too far. If granted imperfectly accountable power, longtermists might try to use the machinery of government to place burdens on the present generation for the sake of further benefits to future generations. These worries may lead to the marginalisation of longtermism, and thus an outcome that is worse for both present and future generations.
The best solution is compromise and commitment.[39] A CBA-plus-AWTP policy – founded as it is on citizens’ preferences – is acceptable to a broad coalition of people. As a result, longtermists committing to act in accordance with a CBA-plus-AWTP policy makes possible an arrangement that is significantly better than the status quo, both by longtermist lights and by the lights of the present generation. It also gives rise to other benefits of cooperation. For example, it helps to avoid needless conflicts in which groups lobby for opposing policies, with some substantial portion of the resources that they spend cancelling each other out (see Ord 2015: 120–21, 135). With a CBA-plus-AWTP policy in place, those resources can instead be spent on interventions that are appealing to all sides.
Thanks so much for writing this thought-provoking piece and for cross-posting it here to the forum for discussion. It is a long piece with lots of subtly different arguments that piece together in complex ways. Indeed, by the end of it, I’m not sure if I agree with the overall thrust or not. But there are many strands on which I do strongly agree or disagree and so I’ll try to break them out into different comments here to make the discussion easier.
Overall:
I’m enthusiastic about your work towards creating CBA for existential risks.
I’m sympathetic to the argument that concern about presently existing people is sufficient to warrant substantial government spending on existential risk (though I don’t think it quite gets there).
I’m also sympathetic to the idea that one should lead with this in policy circles (though I favour the view that one should sometimes lead with it and sometimes with future generations).
I strongly agree that we shouldn’t try to implement policies that are democratically infeasible (though it should be uncontroversial that it is sometimes correct to propose policies that go somewhat beyond current democratic feasibility, since feasibility is not fixed).
I strongly disagree with the idea that it is in some way undemocratic to argue that the effects on future generations give an especially strong reason to be concerned about existential risks (which I think your paper sometimes suggests).
Regarding your CBA:
I worry that your stated estimates, aren’t really enough to secure adequate funding on traditional CBA grounds.
For example, it doesn’t work in many countries. There are only a fifth as many people living in the UK, so only a fifth as many beneficiaries. Thus the benefits internalised by the UK are a fifth as high, so the cost-benefit ratio is only a fifth as good. Moreover, the UK is poorer, so people are less willing to pay to avoid risk and the VSLs used by the government are substantially lower (I think about a fifth your stated figure — £2M), such that the package you list would dramatically fail the CBA by a factor of 25 or so in the UK. Many other countries are poorer or smaller than the US in some combination and will have trouble meeting this cost-benefit bar on the estimates you gave.
Even in the US, your estimates for the cost-benefit calculation aren’t really enough to make it go through. It would be if all these interventions were inextricably bound up as a package, or if all funding on them had equal effect. But CBA cares about marginal cost effectiveness and presumably the package can be broken into chunks of differing ex-ante cost-effectiveness (e.g. by intervention type, or by tranches of funding in each intervention). Indeed you suggest this later in the piece. Since the average only just meets the bar, if there is much variation, the marginal work won’t meet the bar, so government funding would cap out at something less than this, perhaps substantially so.
Moreover, your analysis suggests the package only marginally meets the cost-benefit threshold and requires some educated guesswork to do so (the amounts it would lower existential risk). Such speculative estimates that drive a large funding choice are usually frowned on in CBA. So even if it is marginally worth funding if government attention and capacity were free, it wouldn’t be a high priority.
My guess is therefore that a national package of interventions by the US would need to be notably more modest than this one to get funded purely on traditional CBA grounds, and much more modest to get funded in the UK or elsewhere, widening the gap between what traditional CBA gets us and what the addition of longtermist considerations could achieve.
Yes, this is an important point. If we were to do a more detailed cost-benefit analysis of catastrophe-preventing interventions, we’d want to address it more comprehensively (especially since we also mention how different interventions can undermine each other elsewhere in the paper).
On the point about the average only just meeting the bar, though, I think it’s worth noting that our mainline calculation uses very conservative assumptions. In particular, we assume:
A total cost of $400b rounded up from $319.6b
That all the costs of the interventions are paid upfront and that the risk-reduction occurs only at the end of the decade
The lowest VSL figure used by the DoT
The highest discount rate recommended by OIRA
A 1-in-1,000 GCR-reduction from the whole suite
And we count only these interventions benefits in terms of GCR-reduction. We don’t count any of the benefits arising from these interventions reducing the risk of smaller catastrophes.
I think that, once you replace these conservative assumptions with more reasonable ones, it’s plausible that each intervention we propose would pass a CBA test.
I think that these points also help our conclusions apply to other countries, even though all other countries are either smaller than the US or employ a lower VSL. And even though reasonable assumptions will still imply that some GCR-reducing interventions are too expensive for some countries, it could be worthwhile for these countries to participate in a coalition that agrees to share the costs of the interventions.
I agree that speculative estimates are a major problem. Making these estimates less speculative – insofar as that can be done – seems to me like a high priority. In the meantime, I wonder if it would help to emphasise that a speculative-but-unbiased estimate of the risk is just as likely to be too low as to be too high.
Thanks Elliott,
I guess this shows that the case won’t get through with the conservative rounding off that you applied here, so future developments of this CBA would want to go straight for the more precise approximations in order to secure a higher evaluation.
Re the possibility of international agreements, I agree that they can make it easier to meet various CBA thresholds, but I also note that they are notoriously hard to achieve, even when in the interests of both parties. That doesn’t mean that we shouldn’t try, but if the CBA case relies on them then the claim that one doesn’t need to go beyond it (or beyond CBA-plus-AWTP) becomes weaker.
That said, I think some of our residual disagreement may be to do with me still not quite understanding what your paper is claiming. One of my concerns is that I’m worried that CBA-plus-AWTP is a weak style of argument — especially with elected politicians. That is, arguing for new policies (or treaties) on grounds of CBA-plus-AWTP has some sway for fairly routine choices made by civil servants who need to apply government cost-effectiveness tests, but little sway with voters or politicians. Indeed, many people who would be benefited by such cost-effectiveness tests are either bored by — or actively repelled by — such a methodology. But if you are arguing that we should only campaign for policies that would pass such a test, then I’m more sympathetic. In that case, we could still make the case for them in terms that will resonate more broadly.
I’ve just seen your comment further down:
which answers my final paragraph in the parent comment, and suggests that we are not too far apart.
Yes, I think so!
And thanks again for making this point (and to weeatquince as well). I’ve written a new paragraph emphasising a more reasonable, less conservative estimate of benefit-cost ratios. I expect it’ll probably go in the final draft, and I’ll edit the post here to include it as well (just waiting on Carl’s approval).
I think this is right (and I must admit that I don’t know that much about the mechanics and success-rates of international agreements) but one cause for optimism here is Cass Sunstein’s view about why the Montreal Protocol was such a success (see Chapter 2): cost-benefit analysis suggested that it would be in the US’s interest to implement unilaterally and that the benefit-cost ratio would be even more favourable if other countries signed on as well. In that respect, the Montreal Protocol seems akin to prospective international agreements to share the cost of GCR-reducing interventions.
‘Democratically unacceptable’
Like Richard, I was quite confused while reading the piece about the use of this term. I agree with him that if you are trying to get normative mileage out of this idea, then it doesn’t work. In representative democracies like the US, we elect people to represent us in the highest levels of national decision-making. They are allowed to act in our narrow self-interest, but also in furtherance of our ideals. Indeed, they are aren’t constrained to leading from the back — to leading people to where they already want to go. They are also allowed to to lead from the front — to use good judgment to see things the public hadn’t yet seen, to show why it is an attractive course of action and to take it. Indeed, when we think of great Presidents or Prime Ministers it is often this quality that we most admire. So it seems entirely fine to me to appeal to people in government on moral grounds — including on longtermist grounds such as the effects on future generations. To say otherwise would involve some surprising judgments about other acts of moral leadership that are widely celebrated (such as in the ending of slavery).
I think the best version of this argument about democratic acceptability is not a moral or political philosophy argument (that it is wrong or unjust or undemocratic for governments to implement policies on these grounds) because there is little evidence it would be any of those things. Before a funding allocation reached those levels it would likely become politically unacceptable — i.e. it couldn’t practically be implemented and you would get less in the long run than if you asked for a more modest amount. I think there is broad agreement among longtermists that political feasibility is a key constraint, so the question is then about where it kicks in and what we can do to loosen the constraint by raising moral awareness.
At times it sounds like you agree with this, such as when you explicitly say “democratically unacceptable, by which we mean it could not be adopted and maintained by a democratic government”, but at other times you seem to trade on the idea that there is something democratically tainted about political advocacy on behalf of the people of the future — this is something I strongly reject. I think the paper would be better if it were clearer on this. I think using the term ‘democratically infeasible’ would have been more apposite and would dispel the confusion.
(I should add that part of this is the question of whether you mean infeasible vs normatively problematic, and part of it is the question of whether you are critiquing use of longtermist considerations (over and above CBA considerations) vs whether you are critiquing attempts to impose the policies one would come up with on strong longtermism + ignoring feasibility. I’ll address that more in another comment, but the short version is that I agree that imposing wildly unpopular policies on voters might be normatively problematic as well as infeasible, but think that comparison is a straw man.)
I reject that too. We don’t mean to suggest that there is anything democratically tainted about that kind of advocacy. Indeed, we say that longtermists should advocate on behalf of future generations, in order to increase the present generation’s altruistic willingness to pay for benefits to future generations.
What we think would be democratically unacceptable is governments implementing policies that go significantly beyond the present generation’s altruistic willingness to pay. Getting governments to adopt such policies is infeasible, but we chose ‘unacceptable’ because we also think there would be something normatively problematic about it.
So I think we are addressing your strawman here. Certainly, we don’t take ourselves to be arguing against anyone in particular in saying that there would be something wrong with governments placing heavy burdens on the present generation for the sake of small reductions in existential risk. But it seems worth saying in any case, because I think it helps ward off misunderstandings from people not so familiar with longtermism (and we’re hoping to reach such people—especially policymakers—with this paper). For suppose that someone knows only of the following argument for longtermism: the expected future population is enormous, the lives of future people are good in expectation, and it is better if the future contains more good lives. Then that person might mistakenly think that the goal of longtermists in the political sphere must be something like a strong longtermist policy.
I’m not so sure about that. I agree with you that it would be normatively problematic in the paradigm case of a policy that imposed extreme costs on current society for very slight reduction in total existential risk — let’s say, reducing incomes by 50% in order to lower risk by 1 part in 1 million.
But I don’t know that it is true in general.
First, consider a policy that was inefficient but small — e.g. one that cost $10 million to the US govt, but reduced the number of statistical lives lost in the US by only 0.1, I don’t think I’d say that this was democratically unacceptable. Policies like this are enacted all the time in safety contexts and are often inefficient and ill-thought-out, and I’m not generally in favour of them, but I don’t find them to be undemocratic. I suppose one could argue that all US policy that doesn’t pass a CBA is undemocratic (or democratically unacceptable), but that seems a stretch to me. So I wonder whether it is correct to count our intuitions on the extreme example as counting against all policies that are inefficient in traditional CBA terms or just against those that impose severe costs.
I wouldn’t call a small policy like that ‘democratically unacceptable’ either. I guess the key thing is whether a policy goes significantly beyond citizens’ willingness to pay not only by a large factor but also by a large absolute value. It seems likely to be the latter kinds of policies that couldn’t be adopted and maintained by a democratic government, in which case it’s those policies that qualify as democratically unacceptable on our definition.
Overshooting:
I disagree with this.
First, I think that many moral views are compelled to find the possibility that their generation permanently eradicates all humans from the world to be especially bad and worthy of much extra effort to avoid. As I detailed in Chapter 2 of the Precipice, this can be based on considerations about the past or about the future. While longtermism is often associated with the future-directed reasons, I favour a broader definition. If someone is deeply moved by the Burkean partnership of the generations over an unbroken chain stretching back 10,000 generations and thinks this gives additional reason not to be the generation who breaks it, then I’m inclined to say they are a longtermist too. But whether it counts or doesn’t, my arguments in Chapter 2 still imply that many moral views are already committed to a special badness of extinction (and often other existential risks). This means there is a wide set of views that go beyond traditional CBA and I can’t see a good argument why they should all overshoot.
And what about for a longtermist view that is more typical of our community? Suppose we are committed to the idea that each person matters equally no matter when they would live. It doesn’t follow from this that the best policy is one that demands vast sacrifices from the current generation, anymore than this follows from the widely held view that all people matter equally regardless of race or place of birth. One could still have places in ethics or political philosophy where there are limits placed on the sacrifices that can be demanded of you, and especially limits placed on the sacrifices you can force others to endure in order to produce a larger amount of benefits for others. Theories with such limits could still be impartial in time and could definitely qualify as longtermist.
One could also have things tempered by moral uncertainty or political beliefs about pluralism or non-coercion.
And that is before we get to the fact that longtermist policy drafters don’t have to ignore the feasibility of their proposals — another clear way to stop before you overshoot.
I really don’t think it is clear that there are any serious policy suggestions from longtermists that do overshoot here. e.g. in The Precipice (p. 186) my advice on budget is:
And this doesn’t seem too different from your own advice ($400B spending by the US is 2% of a year’s GDP).
A different take might be that I and others could be commended for not going too far, but that in doing so we are being inconsistent with our stated principles. That is an interesting angle, and one raised by Jim Holt in his very good NYT book review. But I ultimately don’t think it works either: I can’t see any strong arguments that longtermism lacks the theoretical resources to consistently avoid overshooting.
The argument we mean to refer to here is the one that we call the ‘best-known argument’ elsewhere: the one that says that the non-existence of future generations would be an overwhelming moral loss because the expected future population is enormous, the lives of future people are good in expectation, and it is better if the future contains more good lives. We think that this argument is liable to overshoot.
I agree that there are other compelling longtermist arguments that don’t overshoot. But my concern is that governments can’t use these arguments to guide their catastrophe policy. That’s because these arguments don’t give governments much guidance in deciding where to set the bar for funding catastrophe-preventing interventions. They don’t answer the question, ‘By how much does an intervention need to reduce risks per $1 billion of cost in order to be worth funding?’.
This seems like a good target to me, although note that $400b is our estimate for how much it would cost to fund our suite of interventions for a decade, rather than for a year.
Thanks for the clarifications!
Thanks, these comments are great! I’m planning to work through them later this week.
I agree with pretty much all of your bulletpoints. With regards to the last one, we didn’t mean to suggest that arguing for greater concern about existential risks is undemocratic. Instead, we meant to suggest that (in the world as it is today) it would be undemocratic for governments to implement polices that place heavy burdens on the present generation for the sake of small reductions in existential risk.
The target of your comparisons:
At times it seems like you are comparing traditional CBA justifications for government work on existential risk vs longtermist justifications. At other times it seems like you are comparing the policy prescriptions and funding that would pass a traditional CBA vs the policy prescriptions and funding that would come out of strong longtermist reasoning (deliberately setting aside political feasibility). I have different views on each of these comparisons.
On the first, I think we should use both traditional CBA justifications as well as longtermist considerations and don’t think the piece really offers much argument against including the latter at all (for one thing, it doesn’t seem to recognise that they can actually be popular and motivating with the public and with policy makers in a way that CBA isn’t, widening the feasibility window).
On the second, I agree that we should generally not use strong longtermist justifications (though the piece usually frames itself as arguing against regular longtermist justifications). For one thing, I don’t actually endorse strong longtermism anyway. I also agree that we shouldn’t set aside political feasibility in our policy recommendations. But it seems something of a straw man to suggest that the choice under discussion is to ignore effects on future generations or to consider all such effects on a total utilitarian basis and ignore the political feasibility. Are there any serious advocates of that position? The inadequacy of the second of these options doesn’t really help you conclude that we should deliberately restrict ourselves to the CBA option, as there are so many other options that were not considered.
I agree with this. What we’re arguing for is a criterion: governments should fund all those catastrophe-preventing interventions that clear the bar set by cost-benefit analysis and altruistic willingness to pay. One justification for funding these interventions is the justification provided by CBA itself, but it need not be the only one. If longtermist justifications help us get to the place where all the catastrophe-preventing interventions that clear the CBA-plus-AWTP bar are funded, then there’s a case for employing those justifications too.
We think that longtermists in the political sphere should (as far as they can) commit themselves to only pushing for policies that can be justified on CBA-plus-AWTP grounds (which need not entirely ignore effects on future generations). We think that, in the absence of such a commitment, the present generation may worry that longtermists would go too far. From the paper: