Regarding your “outside view” point: I agree with what you say here, but think it cannot directly undermine my original “outside view” argument. These clarifications may explain why:
My original outside view argument appealed to the process by which certain global health interventions such as distributing bednets have been selected rather than their content. The argument is not “global health is a different area from economic growth, therefore a health intervention is unlikely to be optimal for accelerating growth”; instead it is “an intervention that has been selected to be optimal according to some goal X is unlikely to also be optimal according to a different goal Y”.
In particular, if GiveWell had tried to identify those interventions that best accelerate growth, I think my argument would be moot (no matter what interventions they had come up with, in particular in the hypothetical case where distributing bednets had been the result of their investigation).
In general, I think that selecting an intervention that’s optimal for furthering some goal needs to pay attention to all of importance, tractability, and neglectedness. I agree that it would be bad to exclusively rely on the heuristics “just focus on the most important long-term outcome/risk” when selecting longtermist interventions, just as it would be bad to just rely on the heuristics “work on fighting whatever disease has the largest disease burden globally” when selecting global health interventions. But I think these would just be bad ways to select interventions, which seems orthogonal to the question when an intervention selected for X will also be optimal for Y. (In particular, I don’t think that my original outside view argument commits me to the conclusion that in the domain of AI safety it’s best to directly solve the largest or most long-term problem, whatever that is. I think it does recommend to try to deliberately select an intervention optimized for reducing AI risk, but this selection process should also take into account feedback loops and all the other considerations you raised.)
The main way I can see to undermine this argument would be to argue that a certain pair of goals X and Y is related in such a way that interventions optimal for X are also optimal for Y (e.g., X and Y are positively correlated, though this in itself wouldn’t be sufficient). For example, in this case, such an argument could be of the type “our best macroeconomic models predict that improving health in currently poor countries would have a permanent rate effect on growth, and empirically it seems likely that the potential for sustained increases in the growth rate is largest in currently poor countries” (I’m not saying this claim is true, just that I would want to see something like this).
The “inside view” point is that Christiano’s estimate only takes into account the “price of a life saved”. But in truth GiveWell’s recommendations for bednets or deworming are to a large measure driven by their belief, backed by some empirical evidence, that children who grow up free of worms or malaria become adults who can lead more productive lives. This may lead to better returns than what his calculations suggest. (Micronutrient supplementation may also be quite efficient in this respect.)
I think this is a fair point. Specifically, I agree that GiveWell’s recommendations are only partly (in the case of bednets) or not at all (in the case of deworming) based on literally averting deaths. I haven’t looked at Paul Christiano’s post in sufficient detail to say for sure, but I agree it’s plausible that this way of using “price of a life saved” calculations might effectively ignore other benefits, thus underestimating the benefits of bednet-like interventions compared to GiveWell’s analysis.
I would need to think about this more to form a considered view, but my guess is this wouldn’t change my mind on my tentative belief that global health interventions selected for their short-term (say, anything within the next 20 years) benefits aren’t optimal growth interventions. This is largely because I think the dialectical situation looks roughly like this:
The “beware suspicious convergence” argument implies that it’s unlikely (though not impossible) that health interventions selected for maximizing certain short-term benefits are also optimal for accelerating long-run growth. The burden of proof is thus with the view that they are optimal growth interventions.
In addition, some back-of-the-envelope calculations suggest the same conclusion as the first bullet point.
You’ve pointed out a potential problem with the second bullet point. I think it’s plausible to likely that this significantly to totally removes the force of the second bullet point. But even if the conclusion of the calculations were completely turned on their head, I don’t think they would by themselves succeed in defeating the first bullet point.
As I said in another comment, one relevant complication seems to be that risk and growth interact. In particular, the interaction might be such that speeding up growth could actually have negative value. This has been debated for a long time, and I don’t think the answer is obvious. It might something we’re clueless about.
(See Paul Christiano’s How useful is “progress”? for an ingenious argument for why either
(a) “People are so badly mistaken (or their values so misaligned with mine) that they systematically do harm when they intend to do good, or”
(b) “Other (particularly self-interested) activities are harmful on average.”
Conditional on (b) we might worry that speeding up growth would work via increasing the amount or efficiency of various self-interested activities, and thus would be harmful.
I’m not sure if I buy the argument, though. It is based on “approximat[ing] the changes that occur each day as morally neutral on net”. But on longer timescales it seems that we should be highly uncertain about the value of changes. It thus seems concerning to me to look at a unit of time for which the magnitude of change is unintuitively small, round it to zero, and extrapolate from this to a large-scale conclusion.)
You say that:
I will [...] focus instead on a handful of simple model cases. [...] These models will be very simple. In my opinion, nothing of value is being lost by proceeding in this way.
I agree in the sense that I think your simple models succeed in isolating an important consideration that wouldn’t itself be qualitatively altered by looking at a more complex model.
However, I do think (without implying that this contradicts anything you have said in the OP) that there are other crucial premises for the argument concluding that reducing existential risk is the best strategy for most EAs. I’d like to highlight three, without implying that this list is comprehensive.
One important question is how growth and risk interact. Specifically, it seems that we face existential risks of two different types: (a) ‘exogenous’ risks with the property that their probability per wall-clock time doesn’t depend on what we do (perhaps a freak physics disaster such as vacuum decay); and (b) ‘endogenous’ risks due to our activities (e.g. AI risk). The probability of such endogenous risks might correlate with proxies such as economic growth or technological progress, or more specific kinds of these trends. As an additional complication, the distinction between exogenous and endogenous risks may not be clear-cut, and arguably is itself endogenous to the level of progress—for example, an asteroid strike could be an existential risk today but not for an intergalactic civilization. Regarding growth, we might thus think that we face a tradeoff where faster growth would on one hand reduce risk by allowing us to more quickly reach thresholds that would make us invulnerable to some risks, but on the other hand might exacerbate endogenous risks that increase with the rate of growth. (A crude model for why there might be risks of the latter kind: perhaps ‘wisdom’ increases at fixed linear speed, and perhaps the amount of risk posed by a new technology decreases with wisdom.)
I think “received wisdom” is roughly that most risk is endogenous, and that more fine-grained differential intellectual or technological progress aimed at specifically reducing such endogenous risk (e.g. working on AI safety rather then generically increasing technological progress) is therefore higher-value than shortening the window of time during which we’re exposed to some exogenous risks.
See for example Paul Christiano, On Progress and Prosperity
A somewhat different lense is to ask how growth will affect the willingness of impatient actors—i.e., those that discount future resources at a higher rate than longtermists—to spend resources on existential risk reduction. This is part of what Leopold Aschenbrenner has examined in his paper on Existential Risk and Economic Growth.
More generally, the value of existential risk reduction today depends on the distribution of existential risk over time, including into the very long-run future, and on whether todays effort would have permanent effects on that distribution. This distribution might in turn depend on the rate of growth, e.g. for the reasons mentioned in the previous point. For an excellent discussion, see Tom Sittler’s paper on The expected value of the long-term future. In particular, the standard argument for existential risk reduction requires the assumption that we will eventually reach a state with much lower total risk than today.
A somewhat related issue is the distribution of opportunities to improve the long-term future over time. Specifically, will there be more efficient longtermist interventions in, say, 50 years? If yes, this would be another reason to favor growth over reducing risk now. Though more specifically it would favor growth, not of the economy as a whole, but of the pool of resources dedicated to improving the long-term future—for example, through ‘EA community building’ or investing to give later. Relatedly, the observation that longtermists are unusually patient (i.e. discount future resources at a lower rate) is both a reason to invest now and give later, when longtermists control a larger share of the pie—and a consideration increasing the value of “ensuring that the future proceeds without disruptions”, potentially by using resources now to reduce existential risk. For more, see e.g.:
Toby Ord, The timing of labour aimed at reducing existential risk
Owen Cotton-Barratt, Allocating risk mitigation across time
Will MacAskill, Are we living at the most influential time in history?
Phil Trammel, Philanthropic timing and the Hinge of History
You describe the view you’re examining as:
cause areas related to existential risk reduction, such as AI safety, should be virtually infinitely preferred to other cause areas such as global poverty
You then proceed by discussing considerations that are somewhat specific to the specific types of interventions you’re comparing—i.e., reducing extinction risk versus speeding up growth.
You might be interested in another type of argument questioning this view. These arguments attack the “virtually infinitely” part of the view, in a way that’s agnostic about the interventions being compared. For such arguments, see e.g.:
Brian Tomasik, Why Charities Usually Don’t Differ Astronomically in Expected Cost-Effectiveness
Tobias Baumann, Uncertainty smooths out differences in impact
Thank you, I think this is an excellent post!
I also sympathize with your confusion. - FWIW, I think that a fair amount of uncertainty and confusion about the issues you’ve raised here is the epistemically adequate state to be in. (I’m less sure whether we can reliably reduce our uncertainty and confusion through more ‘research’.) I tentatively think that the “received longtermist EA wisdom” is broadly correct—i.e. roughly that the most good we can do usually (for most people in most situations) is by reducing specific existential risks (AI, bio, …) -, but I think that
(i) this is not at all obvious or settled, and involves judgment calls on my part which I could only partly make explicit and justify; and
(ii) the optimal allocation of ‘longtermist talent’ will have some fraction of people examining whether this “received wisdom” is actually correct, and will also have some distribution across existential risk reduction, what you call growth interventions, and other plausible interventions aimed at reducing the long-term future (e.g. “moral circle expansion”) - for basically the “switching cost” and related reasons you mention [ETA: see also sc. 2.4 of GPI’s research agenda].
One thing in your post I might want to question is that, outside of your more abstract discussion, you phrase the question as whether, e.g., “AI safety should be virtually infinitely preferred to other cause areas such as global poverty”. I’m worried that this is somewhat misleading because I think most of your discussion rather concerns the question whether, to improve the long-term future, it’s more valuable to (a) speed up growth or to (b) reduce the risk of growth stopping. I think AI safety is a good example of a type-(b) intervention, but that most global poverty interventions likely aren’t a good example of a type-(a) intervention. This is because I would find it surprising if an intervention that has been selected to maximize some measure of short-term impact also turned out to be optimal for speeding up growth in the long-run. (Of course, this is a defeatable consideration, and I acknowledge that there might be economic arguments that suggest that accelerating growth in currently poor countries might be particularly promising to increase overall growth.) In other words, I think that the optimal “growth intervention” Alice would want to consider probably isn’t, say, donating to distribute bednets; I don’t have a considered view on what it would be instead, but I think it might be something like: doing research in a particularly dynamic field that might drive technological advances; or advocating changes in R&D or macroeconomic policy. (For some related back-of-the-envelope calculations, see Paul Christiano’s post on What is the return to giving?; they suggest “that good traditional philanthropic opportunities have a return of around 10 and the best available opportunities probably have returns of 100-1000, with most of the heavy hitters being research projects that contribute to long term tech progress and possibly political advocacy”, but of course there is a lot of room for error here. See also this post for how maximally increasing technological progress might look like.)
Lastly, here are some resources on the “increase growth vs. reduce risk” question, which you might be interested in if you haven’t seen them:
Paul Christiano’s post on (literal) Astronomical waste, where he considers the permanent loss of value from delayed growth due to cosmological processes (expansion, stars burning down, …). In particular, he also mentions the possibility that “there is a small probability that the goodness of the future scales exponentially with the available resources”, though he ultimately says he favors roughly what you called the plateau view.
In an 80,000 Hours podcast, economist Tyler Cowen argues that “our overwhelming priorities should be maximising economic growth and making civilization more stable”.
For considerations about how to deal with uncertainty over how much utility will grow as a function of resources, see GPI’s research agenda, in particular the last bullet point of section 1.4. (This one deals with the possibility of infinite utilities, which raises somewhat similar meta-normative issues. I thought I remembered that they also discuss the literal point you raised—i.e. what if utility will in the long-run grow exponentially? -, but wasn’t able to find it.)
I might follow up in additional comments with some pointers to issues related to the one you discuss in the OP.
Very interesting, thank you!
Potential typo: In the following paragraph, I think “likely yes” should probably read “very probably yes”.
Although incomplete, there is direct evidence that individuals of these taxa exhibit features which, according to expert agreement, seem to be necessary –although not sufficient– for consciousness (Bateson, 1991; Broom, 2013; EFSA, 2005; Elwood, 2011; Fiorito, 1986; Sneddon et al., 2014; Sneddon, 2017) (see the first criterion of the ‘likely yes’ category);
(Super minor: It also seems slightly inconsistent that the section title “Very Probably Yes” is capitalized, while others aren’t.)
Thanks to Sanjay and you, I found this very interesting to follow!
Just a quick note on your point 2:
On the counterfactual impact of funds. I agree that this is in principle a gap in the CEA. However, this criticism also applies to almost all CEAs I have ever seen. Accounting for all counterfactuals in CEA models is very hard.
It’s quite possible that I’m missing something here, because I’m not familiar with the context, and haven’t tried to dig into the details. - But FWIW, my initial reaction was that this response doesn’t quite speak to what I understood to be the specific concern in the OP, i.e.:
To be clear, this section is *not* considering the opportunity cost of your (the donor’s) money going to CfRN. Rather, CfRN enables other funds to be raised and donated to REDD+ – this section is considering the opportunity cost on those funds.
I agree with you (Halstead) that the opportunity cost of the donor’s money is almost never accounted for in a CEA. (This also wouldn’t be required conceptually—choosing the donation target with maximal cost-effectiveness across all CEAs would be sufficient to minimize opportunity cost, conditional on the CEAs being individually correct and jointly comprehensive.)
I also agree that “accounting for all counterfactuals in CEA models is very hard”.
However, from my (possibly uninformed) perspective, the force of the OP’s argument is due to an appeal to a somewhat specific, contingent property of the intervention under consideration—namely that this CEA is assessing the cost-effectiveness of a donation the primary purpose of which is to cause a change in the allocation of other (here, largely government) funds.
I think this situation is not at all analogous to “money to [FHI] could have gone to global development”. I in fact think it’s similar to why e.g. 80,000 Hours considers plan changes as their main metric. Or, to give a hypothetical example, consider a health intervention aiming to make people buy more vegetables; when assessing the impact of this intervention, I would want to know whether people who end up buying more vegetables have reduced their expenses for grains or chocolate or clothing, whether they’ve taken out a loan to buy more vegetables etc. - And this concern is quite distinct from the concern that “donations to fund this intervention could also have been donated elsewhere”.
Thanks for mentioning this. I had meant to refer to this accident, but after spending 2 more minutes looking into got the impression that there is less consensus on what happened than I thought.
Specifically, the Wikipedia article says:
However, Michael H. Maggelet and James C. Oskins, authors of Broken Arrow: The Declassified History of U.S. Nuclear Weapons Accidents, dispute this claim, citing a declassified report. They point out that the arm-ready switch was in the safe position, the high-voltage battery was not activated (which would preclude the charging of the firing circuit and neutron generator necessary for detonation), and the Rotary Safing Switch was destroyed, preventing energisation of the X-Unit (which controlled the firing capacitors). The tritium reservoir used for fusion boosting was also full and had not been injected into the weapon primary. This would have resulted in a significantly reduced primary yield and would not have ignited the weapon’s fusion secondary stage.
One of the Wikipedia references is a blog post by one of the authors mentioned above, with the title Goldsboro- 19 Steps Away from Detonation. Some quotes:
Bomb 2, the object of the Goldsboro controversy, was not “one step” away from detonation. [...]
In Bomb 2, the High Voltage Thermal Battery was not activated, so no electrical power could reach any components necessary to fire the weapon and produce a nuclear explosion. [...]
While the Ready/Safe Switch in Bomb 2 showed “armed” after recovery, it was actually safe [...]. Most importantly, the high voltage necessary to fire bomb components was not present for bomb 2.
How close was the Goldsboro bomb to producing a nuclear explosion? Not at all.
I didn’t attempt to understand the specific technical claims (not even if there is a dispute about technical facts, or just a different interpretation of how to describe the same facts in terms of how far away the bombs was from detonating), and so can’t form my own view.
Do you have any sense what source to trust here?
In any case, my understanding is that nuclear weapons usually had many safety features, and that it’s definitely true that one or a few of them failed in several instances.
Wars are not caused by weapons or arms races
My impression is that there has been a lot of both theoretical and empirical research on arms races in the field of international relations, and that this claim is still contested. I therefore find it hard to be confident in this claim.
For example, Siverson and Diehl (p. 214 in Midlarsky, ed., 1989) sardonically note that “[i]f there is any consensus among arms race studies, it is that some arms races lead to war and some do not.” Fifteen years later, Glaser (2004) still opens with:
Are arms races dangerous? This basic international relations question has received extensive attention. A large quantitative empirical literature addresses the consequences of arms races by focusing on whether they correlate with war, but remains divided on the answer.
On one hand, there are several theoretical models that posit mechanisms how arms buildups could causally contribute to wars.
Security dilemma/spiral model: If states can’t distinguish offensive from defensive military capabilities and have incomplete information about each other’s goals—in particular, whether they face a “revisionist” state that would seize an opportunity to attack because it wants to acquire more territory -, their desire for security will compel them to engage in a spiral of arming (e.g. Jervis 1978, 2017). [While commonly cited as a way how arms races could cause wars, I think this idea is somewhat muddy, and in particular it often remains unclear whether the posited mechanism is an irrational stimulus-response cascade or some reason why rational actors would engage in an arms race culminating in a situation where war is a rational response to an external threat. See e.g. Glaser 2000, 2004. Similarly, it’s unclear whether even in this model the arms race is a cause of war or rather a mere symptom of underlying structural causes such as incomplete information or states’ inability to commit to more cooperative policies; see Fearon 1995 and Diehl & Crescenzi 1998.] A different approach of explaining escalation dynamics culminating in war is Vasquez’s (1993) “steps-to-war” theory.
Costly deterrence: If the opportunity cost of military expenditures required for deterrence becomes too large, and if military spending could be reduced after a successful war, then it can be rational to take one’s chances and attack (e.g. Powell 1993, Fearon 2018).
Preventive war: If a state anticipates that the balance of power would change in an adversary’s favor, they might want to attack now (e.g. Levy 1987). Allison (2017) has popularized this idea as Thucidydes’s trap and applied it to current US-China relations. The worry that an adversary could acquire a new weapons technology arguably is a special case; as you suggest, the 2003 Iraq War is often seen as an instance, which has inspired a wave of recent scholarship (e.g. Bas & Coe 2012).
On the other hand, there has been extensive empirical research on the arms race-conflict relationship. Stoll (2017) and Mitchell & Pickering (2017) provide good surveys. My takeaway is that the conclusions of early research (e.g. Wallace 1979) should be discarded due to methodological flaws , but that some more recent research is interesting. For example, several studies suggest a change in the arms race-war relationship post-WW2, contra your suggestion that the relationship has been similar since at least WW1. Of course, a major limitation is that conclusions are mostly about correlations rather than causation. Some examples (emphases mine):
Gibler, Rider, and Hutchison (2005) add to the literature by addressing a potential selection bias present in many studies. They attribute this to the unit of analysis—a dispute—which presupposes that deterrence has already failed. In an attempt to resolve this, they identify arms races independently of dispute occurrence and use this to test if arms races either deter or escalate MIDs. Using a sample of strategic rival dyads between 1816 and 1993, it was shown that arms races increase the probability of both disputes and war. [Mitchell & Pickering 2017]
Gibler, Rider, and Hutchison (2005) study conventional arms races and war. [...] Only 13 of 79 wars identified by the Correlates of War Project from 1816 through 1992 were preceded by an arms race. As well, only 25 of the 174 strategic rivals identified by Thompson (2001) had an arms race before a war. [Stoll 2017]
Important empirical literature has also placed arms racing in a broader theoretical context to improve comprehension. The “steps-to-war” approach introduced by Vasquez (1993) includes arms races as one of a number factors that contribute to an escalation of violence between states. A good deal of empirical work has tested this approach in the decades since it was first introduced (Colaresi & Thompson, 2005; Senese & Vasquez, 2008; Vasquez, 1996, 2000, 2004, 2009; Vasquez & Henehan, 2001).
Beginning with Sample’s (2000) multivariate analysis, research on the arms race–war relationship has accounted for territorial disputes and other factors that may influence the outbreak of war. The literature had not, however, examined the relationship between arms races and other steps to war. Vasquez (2004) and Senese and Vasquez (2005, 2008) address this and find that other power politics practices (e.g., alliances and rivalry) do not eliminate the arms race–war relationship.
Building upon these earlier findings, Sample (2012) conducted another analysis that divided the temporal domain into three separate eras: 1816 to 2001, 1816 to 1944, and 1945 to 2001. She further controlled for state rivalries—dividing the data into disputes within rivalry and disputes outside of rivalry—and used three different measures of rivalry to compare the findings. The results showed that mutual military buildups had a substantial impact on conflict escalation to war, between both rivals and non-rivals. This suggests the relationship between arms races and war is not an artifact of rivalry (see Rider et al., 2011, for a contrary view). [Mitchell & Pickering 2017]
Rider, Findley, and Diehl (2011) [...] also study the relationship between rivalries, arms races, and war. The time period of their study is 1816–2000. They use Diehl’s operationalization of an arms race. They find that taking rivalries into account is important to understanding that relationship. In particular, locked-in rivalries (those rivalries that have experienced a large number of disputes) that experience an arms race are more likely to experience a war. [Stoll 2017]
 E.g. Stoll (2017), emphasis mine:
The broader issue is about the basic research design used by Wallace. He did not examine whether arms races lead to war. He looked at dyads that engaged in militarized interstate disputes and asked whether if prior to the dispute the dyad engaged in rapid military growth. If so, Wallace predicted (and his results—with the caveats of other studies noted above—supported this) that the states were very likely to engage in war.
For the moment let us accept Wallace’s findings. Understanding the conditions under which a dyad that engages in a militarized interstate dispute is more likely to end in war is a contribution to understanding why wars happen. But it does not explain the relationship between arms races and war. Even if we accept Wallace’s index as a valid indicator his research design does not allow for the possibility that there may be many arms races that are not associated with disputes. Including these cases may produce very different conclusions about the linkage between arms races and war.
Allison, G. (2017). Destined for war: can America and China escape Thucydides’s trap?. Houghton Mifflin Harcourt.
Bas, M. A., & Coe, A. J. (2012). Arms diffusion and war. Journal of Conflict Resolution, 56(4), 651-674.
Diehl, P. F., & Crescenzi, M. J. (1998). Reconfiguring the arms race-war debate. Journal of Peace Research, 35(1), 111-118.
Fearon, J. D. (1995). Rationalist explanations for war. International organization, 49(3), 379-414.
Fearon, J. D. (2018). Cooperation, conflict, and the costs of Anarchy. International Organization, 72(3), 523-559.
Glaser, C. L. (2000). The causes and consequences of arms races. Annual Review of Political Science, 3(1), 251-276.
Glaser, C. L. (2004). When are arms races dangerous? Rational versus suboptimal arming. International Security, 28(4), 44-84.
Jervis, R. (1978). Cooperation under the security dilemma. World politics, 30(2), 167-214.
Jervis, R. (2017). Perception and Misperception in International Politics: New Edition. Princeton University Press.
Levy, J. S. (1987). Declining power and the preventive motivation for war. World Politics, 40(1), 82-107.
Powell, R. (1993). Guns, butter, and anarchy. American Political Science Review, 87(1), 115-132.
Siverson, R., Diehl, P., & Midlarsky, M. (1989). Handbook of War Studies.
Vasquez, J. A. (1993). The war puzzle (No. 27). Cambridge University Press.
Wallace, M. D. (1979). Arms races and escalation: Some new evidence. Journal of Conflict Resolution, 23(1), 3-16.
On the deterrence effect of nuclear weapons, I think the following empirical finding is interesting. (Though not conclusive, as this kind of research design cannot establish causality.)
Sample (2000) found that arms races increase the chances of both MIDs [militarized interstate disputes] and the likelihood that an MID will escalate to full-scale war. However, she discovered that this was only the case in disputes that occurred before World War II. Similarly, territorial disputes were no longer found to be associated with escalation in the post–World War II era. Sample suggested that the presence of nuclear weapons was a possible explanation for why arms races in the post-war era were found to be less likely to result in the outbreak of war than those that occurred prior. She introduced a nuclear weapons variable to test this and found that the probability of war decreased to .05 when nuclear weapons were present during a mutual military buildup. Sample’s discovery of the potential pacifying effect of nuclear weapons was an important contribution to our understanding of how quantitative and qualitative arms race–war relationships differ.
Quote from Mitchell and Pickering, 2017, an encyclopedia article reviewing work on arms races (emphasis mine). On the impact of nukes, they continue (emphasis mine):
The advent of nuclear weapons thus appears to have changed the arms race–conflict relationship. It is important to note in this regard that many policymakers seem to place nuclear weapons in a different conceptual category than conventional weapons. As Sagan (1996, p. 55) has argued, nuclear weapons “are more than tools of national security; they are political objects of considerable importance in domestic debates and internal bureaucratic struggles and can also serve as international normative symbols of modernity and identity.” There have also been attempts to explain “nuclear reversal” cases by which states forgo or give up on their programs (Campbell et al., 2004; Levite, 2003; Paul, 2000; Reiss, 1995; Rublee, 2009). Research has shown that the possession of such weapons is contingent upon both willingness and opportunity (Jo & Gartzke, 2007). While security concerns and technological capabilities are significant determinants of whether states pursue the development of nuclear weapons, the possession of such weapons is dependent upon such factors as domestic politics and international considerations (Jo & Gartzke, 2007). Furthermore, states are heavily dependent upon sensitive nuclear assistance from more advanced nuclear states when attempting to develop a nuclear arsenal (Kroenig, 2009a, 2009b).2 The nature of nuclear weapons acquisition is thus multifaceted and may not always be motivated by arms races. Once acquired, however, nuclear capabilities seem to impact the likelihood of conflict escalation and disputes between states.
[Gibler et al. 2005] controlled for several variables previously demonstrated to be predictors of conflict in a dyad. Among these was the joint presence of nuclear weapons, which was shown to prevent the outbreak of war (as no war has occurred in a dyad where both states possessed nuclear weapons). However, if both states had nuclear weapons, this was found to actually increase the probability of MID onset. Subsequent research has shown that nuclear dyads have engaged in a large number of militarized disputes short of war and may be even more likely to engage in MIDs than non-nuclear states or asymmetric pairs of states (see, e.g., Beardsley & Asal, 2009; Rauchhaus, 2009).
Gibler et al.’s (2005) discovery that nuclear dyads are less likely to engage in all-out war between rivals but more likely to engage in MIDs and hostile action short of war contributes to the broader understanding of the role nuclear weapons play in state decisions to use military force. Although a detailed discussion of nuclear deterrence is outside the scope of this article, it is important to highlight a key debate within this context. Among those who believe that nuclear weapons can serve as a deterrent (often referred to as “proliferation optimists”), some have argued that possession can deter aggression at all levels (Jervis, 1989; Waltz, 1990). Others, meanwhile, have contested that possession secures states from high-level conflict escalation (e.g., war) but increasingly contributes to lower-level hostile action (e.g., MIDs) (Snyder, 1965; Snyder & Diesing, 1977). This concept is known as the “stability-instability paradox,” which states that “to the extent that the military balance is stable at the level of all-out nuclear war, it will become less stable at lower levels of violence” (Jervis, 1984, p. 31).
If Iran actually does develop something of an atomic arsenal, it will likely find, following the experience of all other states so armed, that the bombs are essentially useless and a very considerable waste of money and effort
This claim strikes me as particularly dubious intuitively. I don’t have specific evidence in favor of my intuition, but I think I would want to see quite substantial evidence for Mueller’s claim to believe it, as I think my prior is driven by the following considerations:
At first glance, it seems that Iran’s adversaries are also concerned about the prospect of Iran acquiring nukes. For example, the US seems to be willing to pay a substantial cost in terms of tensions with European allies in order to take a tougher stance toward Iran, e.g. the Trump administration cancelling the nuclear deal. Similarly, there clearly were risks involved in deploying the Stuxnet cyber weapon against Iran. (This is an interesting case because Stuxnet was targeted specifically at Iran’s nuclear program; so the potential response “Iran’s adversaries are using the prospect of a nuclear Iran merely as a pretext to push through policies that hurt Iran more generally, e.g. economically” does not work in this case.)
More broadly, Mueller essentially seems to claim that there is some very widespread delusion: While in fact nuclear weapons are just a waste of money, all of the following actors are making the same epistemic error of believing the opposite (as indicated by their revealed preferences): most Democrats in the US; most Republicans in the US; most people across the political spectrum in Israel; the government of Saudi Arabia; both “moderate” and conservative politicians in Iran; the government of Russia, etc. What is more, incentives to correct this epistemic error surely aren’t super great, but they are not zero either: If, say, a Democratic US President is making a big foreign policy blunder by accepting considerable cost to prevent Iran from acquiring nuclear weapons, why aren’t there more Republicans who jump onto this opportunity to embarrass the government? Why is the prospect of a nuclear Iran able to—at least to some extent—unite a diverse set of actors such as the US, the EU, Israel, Saudi Arabia, and Russia behind a common foreign policy objective, i.e., to prevent Iran from acquiring nuclear weapons?
I also think the claim flies in the face of common sense. In particular, Israel is a tiny country in a vulnerable geographic position, it has been attacked several times since its inception, and Iran has consistently taken a very hostile stance toward it (and not just via cheap talk, but also by e.g. sponsoring insurgent groups in Lebanon). At first glance, I find the suggestion that the additional option of (explicit or implicit) nuclear threats against Israel would not hurt Israel’s interests hard to believe. Similarly, the US has a history of recent interference in Middle Eastern countries via conventional wars, see Afghanistan and Iraq. I think an American attack on Iran with the objective of regime change within the next decade is at least plausible, and everyone knows this. On the other hand, I don’t think anyone has ever tried to attack a nuclear weapons state with a regime change objective. (AFAIK the only direct military conflicts between nuclear weapons states were a 1969 border conflict between China and the USSR, and the 1999 Kargil War between India and Pakistan—both cases in which all sides clearly had much more limited objectives.) Again, I think the idea that the US would invade an Iran armed with nuclear weapons is on its face implausible. If this is true, then possessing nuclear weapons would decrease one of the arguably major risks to Iranian sovereignty—so how can they be a “considerable waste of money and effort”?
(2) I continue to worry that the so-far (apparently) perfect safety and security record for nuclear weapons will eventually end, which could (but probably won’t) have global catastrophic effects
My guess is you’re very likely aware of this, but for other readers it might be worth pointing out that the safety record is “perfect” only if the outcome of interest is a nuclear detonation.
There were, however, several accidents where the conventional explosives (that would trigger a nuclear detonation in intended use cases) in a nuclear weapon detonated (but where safety features prevented a nuclear detonation). E.g., accidents involving bombers in 1958 and 1968, the latter also causing radioactive contamination of an uninhabited part of Greenland; and some accidents involving missiles, e.g. in 1980. See also Wikipedia’s list of military nuclear accidents.
More broadly, the sense I got e.g. from Schlosser’s book Command and Control is that within the US government and military it was a contested issue how much to weigh safety versus cost and desired military capabilities such as readiness. The book mentions several individuals working in government or at nuclear weapon manufacturers campaigning for additional safety measures or changes of risky policies, with mixed success—overall it seemed to me that the US arsenal did for decades contain weapons for which we at least couldn’t rule out an accidental nuclear detonation with extreme confidence.
(Similar remarks apply to security. I forgot the details, but it doesn’t inspire confidence that senior US decision-makers on some occasions worried about scenarios in which European allies such as Turkey might seize scarcely guarded US nuclear weapons during a crisis.)
It is likely that no “loose nukes” — nuclear weapons missing from their proper storage locations and available for purchase in some way — exist
This squares well with my weakly held prior, based on crude beliefs such that most dangers around terrorism are exaggerated.
However, I’m wondering how Mueller treats the question of whether we would know. E.g., during a 2007 incident in the US, several nuclear weapons were mistakenly loaded onto a bomber that was unguarded for hours at both its start and target locations; no-one realized the weapons were missing for about 36 hours, and the whole problem was only discovered once someone discovered the nukes in the bomber.
My guess is that nuclear weapons handling procedures would probably have uncovered eventually that some warheads were missing at the storage location. But as this incident illustrates it’s (i) unclear when, and (ii) there is room for human error (according to Schlosser’s Command and Control, the incident was only possible because four different crews failed to check whether the relevant missiles were loaded with nuclear warheads, even though all of them were supposed to).
Also note that there were a very small number (2-5 based on a loose memory) of accidents in which nuclear weapons were lost and, as far as we know, never recovered. E.g., over Canada in 1950, and in the sea near Japan in 1965. Of course, most likely these weapons haven’t been discovered by anyone, and thus are not “available for purchase”.
So while “likely” seems plausible to me, I find it hard to have extreme confidence in there being no “loose nukes”.
More relevantly, I’d hope that Mueller discusses all of these cases, or else I’d decrease my confidence in his claims.
I understand that you mostly just provide a summary rather than giving reasons to believe the claims in the book, but FWIW I find some of the claims hard to believe. I’ll give more detail in other comments.
For now, some general questions:
What kind of evidence is the book based on? (E.g. archival research, interviews with decision-makers, theoretical models, …)
Does Mueller have a credible debunking explanation for why most people in the national security community (as well as fields such as international relations, nuclear strategy etc., AFAICT) disagree with him?
Thanks, that’s good to know.
I guess I’m mostly comparing the two EA orgs I’ve worked with, and my memory of informal conversations in the community, with my experience elsewhere, e.g. at a student-run global poverty nonprofit I volunteered for while at university. It’s possible that my sample of EA conversations was unrepresentative, or that I’m comparing to an unusually high baseline.
Thanks for making this available!
My impression from having worked at 2 EA organizations in the last few years, and the conversations I’ve had in the community more generally, is that paying more attention to and sharing more thoughts about how to build orgs and other organizational issues could be very valuable. E.g. on: management, hiring, how to scale, communication, feedback, pros and cons of different decision-making structures, how to productively confront interpersonal conflict, accountability/oversight mechanisms, …
I think one of the strengths of the EA community is that improving our reasoning as individuals has really become part of our ‘cultural DNA’. I’m thinking of all the discussions on cognitive biases, forecasting, how to weigh different types of evidence, etc. If the cultural status and available knowledge on ‘organizational improvement’ could become as strong as for ‘self-improvement’, that would make me really optimistic about what we can achieve.
Here are some overviews:
My guess is that ultimately you’ll just find yourself in an irresolvable standoff of differing intuitions with people who favor a different view of value. Philosophers have debated this question for millennia to decades (depending on how we count) and haven’t reached agreement, so I think in the absence of some methodological revolution settling this question is hopeless. (Though of course, you clarifying your own thinking, or arriving at a view you feel more confident in yourself, seem feasible.)
I agree with your points, but from my perspective they somewhat miss the mark.
Specifically, your discussion seems to assume that we have a fixed, exogenously given set of propositions or factors X, Y, …, and that our sole task is to establish relations of correlation and causation between them. In this context, I agree on preferring “wide surveys” etc.
However, in fact, doing research also requires the following tasks:
Identify which factors X, Y, … to consider in the first place.
Refine the meaning of the considered factors X, Y, … by clarifying their conceptual and hypothesized empirical relationships to other factors.
Prioritize which of the myriads of possible correlational or causal relationships between the factors X, Y, … to test.
I think that depth can help with these three tasks in ways in which breadth can’t.
For instance, in Will’s example, my guess is that the main value of considering the history of Objectivism does not come from moving my estimate for the strength of the hypothesis “X = romantic involvement between movement leaders → Y = movement collapses”. Rather, the source of value is including “romantic involvement between movement leaders” into the set of factors I’m considering in the first place. Only then am I able to investigate its relation to outcomes of interests, whether by a “wide survey of cases” or otherwise. Moreover, I might only have learned about the potential relevance of “romantic involvement between movement leaders” by looking at some depth into the history of Objectivism. (I know very little about Objectivism, and so don’t know if this is true in this instance; it’s certainly possible that the issue of romantic involvement between Objectivist leaders is so well known that it would be mentioned in any 5-sentence summary one would encounter during a breadth-first process. But it also seems possible that it’s not, and I’m sure I could come up with examples where the interesting factor was buried deeply.)
My model here squares well with your observation that a “common feature among superforecasters is they read a lot”, and in fact makes a more specific prediction: I expect that we’d find that superforecasters read a fair amount (say, >10% of their total reading) of deep, small-n case studies—for example, historical accounts of a single war, economic policy, or biographies.
[My guess is that my comment is largely just restating Will’s points from his above comment in other words.]
(FWIW, I think some generators of my overall model here are:
Frequently experiencing disagreements I have with others, especially around AI timelines and takeoff scenarios, as noticing a thought like “Uh… I just think your overall model of the world lacks depth and detail.” rather than “Wait, I’ve read about 50 similar cases, and only 10 of them are consistent with your claim”.
Semantic holism, or at least some of the arguments usually given in its favor.
Some intuitive and fuzzy sense that, in the terminology of this Julia Galef post, being a “Hayekian” has worked better for me than being a “Planner”, including for making epistemic progress.
Some intuitive and fuzzy sense of what I’ve gotten out of “deep” versus “broad” reading. E.g. my sense is that reading Robert Caro’s monumental, >1,300-page biography of New York city planner Robert Moses has had a significant impact on my model of how individuals can attain political power, albeit by adding a bunch of detail and drawing my attention to factors I previously wouldn’t have considered rather than by providing evidence for any particular hypothesis.)
I agree. However, your reply makes me think that I didn’t explain my view well: I do, in fact, believe that it is not obvious that, say, setting up seed banks is “better than doing nothing”—and more generally, that nothing is obviously better than doing nothing.
I suspect that my appeal to “diverting attention and funding” as a reason for this view might have been confusing. What I had in mind here was not an argument about opportunity cost: while true, I did not want to say that an actor that set up a seed bank could perhaps have done better by doing something else instead (say, donating to ALLFED).
Instead, I was thinking of effects on future decisions (potentially by other actors), as illustrated by the following example:
Compare the world in which, at some time t0, some actor A decides to set up a seed bank (say, world w1) with the world w2 in which A decides to do nothing at t0.
It could be the case that, in w2, at some later time t1, a different actor B makes a decision that:
Causes a reduction in the risk of extinction from nuclear war that is larger than the effect of setting up a seed bank at t0. (This could even be, say, the decision to set up two seed banks.)
Happened only because A did not set up a seed bank at t0, and so in particular does not occur in world w1. (Perhaps a journalist in w2 wrote a piece decrying the lack of seed banks, which inspired B—who thus far was planning to become an astronaut—to devote her career to setting up seed banks.)
Of course, this particular example is highly unlikely. And worlds w1 and w2 would differ in lots of other aspects. But I believe considering the example is sufficient to see that extinction risk from nuclear war might be lower in world w2 than in w1, and thus that setting up a seed bank is not obviously better than doing nothing.