Conflict of interest statement: I am the aforementioned human.
This might not quite “belong” in this list. But one could classify risks by which of the different “paths” they might follow (e.g., those that would vs wouldn’t “pass through” a distinct collapse stage).
Personally, I think the model/classification scheme in Defence in Depth is probably the most useful. But I think at least a quick skim of the above sources is useful; I think they each provide an additional useful angle or tool for thought.
I intend to add to this list over time. If you know of other relevant work, please mention it in a comment.
Wait, exactly what are you actually collecting here?
The scope of this collection is probably best revealed by checking out the above sources.
But to further clarify, here are two things I don’t mean, which aren’t included in the scope:
Classifications into things like “AI risk vs biorisk”, or “natural vs anthropogenic”
Such categorisation schemes are clearly very important, but they’re also well-established and you probably don’t need a list of sources that show them.
Classifications into different “types of catastrophe”, such as Ord’s distinction between extinction, unrecoverable collapse, and unrecoverable dystopia
This is also very important, and maybe I should make such a collection at some point, but it’s a separate matter to this.
Initial thoughts on anti-ageing and societal resilience to pandemics
[Epistemic status: fairly speculative. Practicing throwing out less-polished ideas as shortform. Might tidy up and promote this to a frontpage post in the future.]
I’m hardly the first to note that COVID-19 is a disease of ageing: from the beginning of the outbreak it’s been clear that older people were dramatically more likely to experience severe illness or die as a result of infection, while death rates among under-30s have been extremely low. In an important sense[1], COVID-19 falls into the same category as cancer, heart disease, and dementia: a symptom of ageing, which is currently treated independently, but which in the longer term could be much more efficiently treated by tackling ageing itself[2].
The same observation can be generalised to respiratory diseases more generally, and (less strongly) to infectious diseases as a whole. Seasonal influenza is overwhelmingly an affliction of older people, with >90% of fatalities occurring in individuals over 65. The original SARS outbreak showed a similar age-skewing to the current pandemic (though with higher CFRs across the board), with <1% mortality among cases under 24 years old and >55% mortality in over-65s. Case fatality rates in adults increase with age for ebola, tuberculosis, and most other disease you might want to check[3]. As a general rule, if a disease is novel (i.e. there is no acquired immunity in the population), older adults[4] who catch it will fare much worse[5]. This isn’t surprising: older people have weaker and less adaptable immune systems than young adults and show much higher general frailty.
I’m interested in whether this phenomenon provides a good additional argument for anti-ageing research as a long-termist cause area. In general, the healthier and more robust the population is, the less we have to fear from catastrophic pandemics. Of course, a truly existential pandemic would kill the young as well as the old: trivially, for a pandemic to kill everyone, it must kill young people. But I’d expect that many sub-extinction catastrophic pandemics would kill a much higher proportion of older adults than younger ones. Since our population is ageing, this will only become a bigger and bigger problem with time.
The worse a pandemic is, the bigger we can expect its effects to be on society. COVID-19, a much milder disease than a hypothetical GCBR, has already caused huge changes and is widely expected to leave long-lasting effects on politics and policy; effects that would almost certainly never have happened in the absence of ageing. The dislocation caused by a worse pandemic could be far more drastic. If anti-ageing reduced a future biological catastrophe to “only” COVID-19 levels that would be a big win. Generalising from this, ending ageing could act as an important “anti-risk factor”, turning borderline existential biological risks into non-existential GCBRs and borderline GCBRs into “mere catastrophes”.
Similar (but shakier) arguments can be made about some other kinds of catastrophe: if any sort of physical health and endurance is required at a population level to survive them and avoid large negative trajectory changes, ending ageing could be very helpful.
I’m not claiming this in itself would be sufficient to justify the huge research expense required to defeat ageing; many pandemics would probably not be as age-skewed as COVID-19, and if our only goal were to defeat pandemics then broad-spectrum antivirals etc. would be more obvious low-hanging fruit to pick. But I think it might militate towards being substantially more favourable to anti-ageing research, from a long-termist perspective, than one otherwise would be.
Obviously, in another important sense COVID-19 is very different: unlike, say, heart disease, COVID-19 has a single, clear, exogenous cause which can be effectively fought with a variety of short-term measures like social distancing. It would be silly to suggest we attack COVID-19 with anti-ageing research in the short term; my claim is that, over the span of decades required to do so, solving ageing would dramatically cut the death toll from both endemic and emerging respiratory diseases. ↩︎
More generally, the reference class of “pandemics similar to COVID-19” falls into this category. ↩︎
The stand-out exception to this trend is the 1918 flu pandemic, which showed three peaks of mortality: early childhood, very old age, and young adulthood. In this pandemic the death rate of 25-year-olds was higher than that of 65-year-olds. There seem to be several competing explanations for this, some of which suggest a biological vulnerability of younger adults and others of which are more circumstantial; I’m not going to dig into it here except to say that this is a very important exception to the general rule I’m discussing here. The 2009 flu pandemic, also H1N1, also killed unusually large numbers of younger adults. ↩︎
Of course, young children, who have both weaker immune systems and minimal acquired immunity, often fare worst of all. ↩︎
This is not always obvious from the raw mortality breakdown, if older people are much rarer than young adults in a society, or if the spread of the disease is concentrated among particular groups that are age-skewed. ↩︎
This is a really interesting point. An additional consideration is that global leaders tend to be older, and hence more at risk (cf. Boris Johnson). You could imagine that their deaths are especially destabilizing.
If the longtermist argument for preventing pandemics is that they trigger destabilization which leads to, say, nuclear war, the age impacts could be an important factor.
Working titles of things I plan/vaguely hope to write
Note: If you might be interested in writing about similar ideas, feel very free to reach out to me. It’s very unlikely I’ll be able to write all of these posts by myself, so potentially we could collaborate, or I could just share my thoughts and notes with you and let you take it from there.
Defining existential risks and existential catastrophes
My thoughts on Toby Ord’s policy & research recommendations
Existential security
Civilizational collapse and recovery: Toby Ord’s views and my doubts
The Terrible Funnel: Estimating odds of each step on the x-risk causal path (working title)
The idea here would be to adapt something like the “Great Filter” or “Drake Equation” reasoning to estimating the probability of existential catastrophe, using how humanity has fared in prior events that passed or could’ve passed certain “steps” on certain causal chains to catastrophe.
E.g., even though we’ve never faced a pandemic involving a bioengineered pathogen, perhaps our experience with how many natural pathogens have moved from each “step” to the next one can inform what would likely happen if we did face a bioengineered pathogen, or if it did get to a pandemic level.
This idea seems sort of implicit in the Precipice, but isn’t really spelled out there. Also, as is probably obvious, I need to do more to organise my thoughts on it myself.
This may include discussion of how Ord distinguishes natural and anthropogenic risks, and why the standard arguments for an upper bound for natural extinction risks don’t apply to natural pandemics. Or that might be a separate post.
Developing—but not deploying—drastic backup plans
“Macrostrategy”: Attempted definitions and related concepts
This would relate in part to Ord’s concept of “grand strategy for humanity”
Collection of notes
A post summarising the ideas of existential risk factors and existential security factors?
I suspect I won’t end up writing this, but I think someone should. For one thing, it’d be good to have something people can reference/link to that explains that idea (sort of like the role EA Concepts serves).
Do emergency universal pass/fail policies improve or worsen student well-being and future career prospects?
I think a natural experiment is in order. Many colleges are adopting universal pass/fail grading for this semester in response to the COVID-19 pandemic, while others aren’t. Someone should study the impact this will have on students to inform future university pandemic response policy.
When suggestions of this type come up, especially for causes that don’t have existing EA research behind them, my recommended follow-up is to look for people who study this as normal academics (here, “this” would be “ways that grades and grading policy influence student outcomes”). Then, write to professors who do this work and ask if they plan on taking advantage of the opportunity (here, the natural experiment caused by new grading policies).
There’s a good chance that the people you write to will have had this idea already (academics who study a subject are frequently on the lookout for opportunities of this kind, and the drastic changes wrought by COVID-19 should be increasing the frequency with which people think about related studies they could run). And if they haven’t, you have the chance to inspire them!
Writing to random professors could be intimidating, but in my experience, even when I’ve written emails like this as a private citizen without a .edu email address, I frequently get some kind of response; people who’ve made research their life’s work are often happy to hear from members of the public who care about the same odd things they do.
Thanks for the suggestion! I imagine that most scholars are reeling from the upheavals caused by the pandemic response, so right now doesn’t feel like the right time to ask professors to do anything. What do you think?
I think that’s probably true for those working directly on the pandemic, but I’m not sure education researchers would mind being bothered. If anything they might welcome the distraction.
If a typical mammalian species survives for ~1 million years, should a 200,000 year old species expect another 800,000 years, or another million years?
tl;dr I think it’s “another million years”, or slightly longer, but I’m not sure.
How much of this future might we live to see? The fossil record provides some useful guidance. Mammalian species typically survive for around one million years before they go extinct; our close relative, Homo erectus, survived for almost two million.[38] If we think of one million years in terms of a single, eighty-year life, then today humanity would be in its adolescence—sixteen years old, just coming into our power; just old enough to get ourselves into serious trouble.
(There are various extra details and caveats about these estimates in the footnotes.)
Ord also makes similar statements on the FLI Podcast, including the following:
If you think about the expected lifespan of humanity, a typical species lives for about a million years [I think Ord meant “mammalian species”]. Humanity is about 200,000 years old. We have something like 800,000 or a million or more years ahead of us if we play our cards right and we don’t lead to our own destruction. The analogy would be 20% of the way through our life[...]
I think this is a strong analogy from a poetic perspective. And I think that highlighting the typical species’ lifespan is a good starting point for thinking about how long we might have left. (Although of course we could also draw on many other facts for that analysis, as Ord discusses in the book.)
But I also think that there’s a way in which the lifespan analogy might be a bit misleading. If a human is 70, we expect they have less time less to live than if a human is 20. But I’m not sure whether, if a species if 700,000 years old, we should expect that species to go extinct sooner than a species that is 200,000 years old will.
My guess would be that a ~1 million year lifespan for a typical mammalian species would translate into a roughly 1 in a million chance of extinction each year, which doesn’t rise or fall very much in a predictable way over most of the species’ lifespan. Specific events, like changes in a climate or another species arriving/evolving, could easily change the annual extinction rate. But I’m not aware of an analogy here to how ageing increases the annual risk of humans dying from various causes.
I would imagine that, even if a species has been around for almost or more than a million years, we should still perhaps expect a roughly 1 in a million chance of extinction each year. Or perhaps we should even expect them to have a somewhat lower annual chance of extinction, and thus a higher expected lifespan going forwards, based on how long they’ve survived so far?
(But I’m also not an expert on the relevant fields—not even certain what they would be—and I didn’t do extra research to inform this shortform comment.)
I don’t think that Ord actually intends to imply that species’ “lifespans” work like humans’ lifespans do. But the analogy does seem to imply it. And in the FLI interview, he does seem to briefly imply that, though of course there he was speaking off the cuff.
I’m also not sure how important this point is, given that humans are very atypical anyway. But I thought it was worth noting in a shortform comment, especially as I expect that, in the wake of The Precipice being great, statements along these lines may be quoted regularly over the coming months.
If I were to read one of EA-related books (e.g. Doing Good Better, The Most Good You Can Do, The Life You Can Save, The Precipice, Superintelligence, etc.), I would consider writing/improving a summary of the book in wikipedia while reading it, in a way that conveys main points well. It could help you to digest the book better and help others to understand the ideas a bit. You could do it in english as well as maybe in some other language. To see whether it’s worth putting in the effort, you can check out Wikipedia pageview statistics of the books I mentioned and others here (it doesn’t include some views that come from redirects though). It seems that the page Superintelligence is the most viewed one out of these with an average of 4,597 monthly visitors.
Epistemic status: speculative; arm-chair thinking; non-expert idea; unfleshed idea
Proposal: Have nuclear powers insure each other that they won’t nuke each other for mutually assure destruction (ie. destroying my infrastructure means you will destroy your economy). Not accepting an offered of mutual insurances should be seen as extremely hostile and uncooperative, and possible even be severely sanctioned internationally.
Also: what about just explicitly criminalizing a) a first strike, b) a nuclear attack? The idea is to make it more likely that the individuals who participated in a nuclear strike would be punished—even if they considered it to be morally justified.
(Someone will certainly think this is “serious April Fool’s stuff”)
Does anyone have any idea / info on what proportion of the infected cases are getting Covid19 inside hospitals?
(Epistemic status: low, but I didin’t find any research on that, so the hypothesis deserves a bit more of attention)
1. Nosocomial infections are serious business. Hospitals are basically big buildings full of dying people and the stressed personel who goes from one bed to another try to avoid it. Throw a deadly and very contagious virus in it, and it becomes a slaughterhouse.
2. Previous coronavirus were rapidly spread in hospitals and other care units. That made South East Asia kinda prepared for possibly similar epidemics (maybe I’m wrong, but in news their medical staff is always in Hazmat suits, unlike most Health workers in the West). Maybe this is a neglected point in the successful approach in South East Asia?
3. I know hospitals have serious protocols to avoid it… but it takes only a few careless cleaning staff, or a patient’s relatives going to cafeteria, or a badly designed airflow, to ruin everything. Just one Hospital chain in Brazil concentrates most of deaths in Sao Paulo, and 40% of the total national.
The Good Judgement Open forecasting tournament gives a 66% chance for the answer to “Will the UN declare that a famine exists in any part of Ethiopia, Kenya, Somalia, Tanzania, or Uganda in 2020?”
I think that the 66% is a slight overestimate. But nonetheless, if a famine does hit, it would be terrible, as other countries might not be able to spare enough attention due to the current pandemic.
Collection of sources related to dystopias and “robust totalitarianism”
The Precipice—Toby Ord (Chapter 5 has a full section on Dystopian Scenarios)
The Totalitarian Threat—Bryan Caplan (a link to a Word doc version can be found on this page) (some related discussion on the 80k podcast here; use the “find” function)
A shift in arguments for AI risk—Tom Sittler (this has a brief but valuable section on robust totalitarianism) (discussion of the overall piece here)
Existential Risk Prevention as Global Priority—Nick Bostrom (this discusses the concepts of “permanent stagnation” and “flawed realisation”, and very briefly touches on their relevance to e.g. lasting totalitarianism)
I intend to add to this list over time. If you know of other relevant work, please mention it in a comment.
Mind-readers as a neglected life extension strategy
Last updated: 2020-03-30
Status: idea to integrate in a longer article
Assuming that:
Death is bad
Lifelogging is a bet worth taking as a life extension strategy
It seems like a potentially really important and neglected intervention is improving mind readers as this is by far the most important part of our experience that isn’t / can’t be captured at the moment.
We don’t actually need to be able to read the mind right now, just to be able to record the mind with sufficiently high resolution (plausibly along text and audio recording to be able to determine which brain patterns correspond to what kind of thoughts).
Questions:
Assuming we had extremely good software, how much could we read minds with our current hardware? (ie. how much is it worth recording your thoughts right now?)
How inconvenient would it be? How much would it cost?
To do:
Ask on Metaculus some operationalisation of the first question
My review of Tom Chivers’ review of Toby Ord’s The Precipice
I thought The Precipice was a fantastic book; I’d highly recommend it. And I agree with a lot about Chivers’ review of it for The Spectator. I think Chivers captures a lot of the important points and nuances of the book, often with impressive brevity and accessibility for a general audience. (I’ve also heard good things about Chivers’ own book.)
But there are three parts of Chivers’ review that seem to me to like they’re somewhat un-nuanced, or overstate/oversimplify the case for certain things, or could come across as overly alarmist.
I think Ord is very careful to avoid such pitfalls in The Precipice, and I’d guess that falling into such pitfalls is an easy and common way for existential risk related outreach efforts to have less positive impacts than they otherwise could, or perhaps even backfire. I understand that a review gives on far less space to work with than a book, so I don’t expect anywhere near the level of nuance and detail. But I think that overconfident or overdramatic statements of uncertain matters (for example) can still be avoided.
I’ll now quote and comment on the specific parts of Chivers’ review that led to that view of mine.
An alleged nuclear close call
Firstly, in my view, there are three flaws with the opening passage of the review:
Humanity has come startlingly close to destroying itself in the 75 or so years in which it has had the technological power to do so. Some of the stories are less well known than others. One, buried in Appendix D of Toby Ord’s splendid The Precipice, I had not heard, despite having written a book on a similar topic myself. During the Cuban Missile Crisis, a USAF captain in Okinawa received orders to launch nuclear missiles; he refused to do so, reasoning that the move to DEFCON 1, a war state, would have arrived first.
Not only that: he sent two men down the corridor to the next launch control centre with orders to shoot the lieutenant in charge there if he moved to launch without confirmation. If he had not, I probably would not be writing this — unless with a charred stick on a rock.
First issue: Toby Ord makes it clear that “the incident I shall describe has been disputed, so we cannot yet be sure whether it occurred.” Ord notes that “others who claimed to have been present in the Okinawa missile bases at the time” have since challenged this account, although there is also “some circumstantial evidence” supporting the account. Ultimately, Ord concludes “In my view this alleged incident should be taken seriously, but until there is further confirmation, no one should rely on it in their thinking about close calls.” I therefore think Chivers should’ve made it clear that this is a disputed story.
Second issue: My impression from the book is that, even in the account of the person claiming this story is true, the two men sent down the corridor did not turn out to be necessary to avert the launch. (That said, the book isn’t explicit on the point, so I’m unsure.) Ord writes that Bassett “telephoned the Missile Operations Centre, asking the person who radioed the order to either give the DEFCON 1 order or issue a stand-down order. A stand-down order was quickly given and the danger was over.” That is the end of Ord’s retelling of the account itself (rather than discussion of the evidence for or against it).
Third issue: I think it’s true that, if a nuclear launch had occurred in that scenario, a large-scale nuclear war probably would’ve occurred (though it’s not guaranteed, and it’s hard to say). And if that happened, it seems technically true that Chivers probably would’ve have written this review. But I think that’s primarily because history would’ve just unfolded very, very difficulty. Chivers seems to imply this is because civilization probably would’ve collapsed, and done so so severely than even technologies such as pencils would be lost and that they’d still be lost all these decades on (such that, if he was writing this review, he’d do so with “a charred stick on a rock”).
This may seem like me taking a bit of throwaway rhetoric or hyperbole too seriously, and that may be so. But I think among the key takeaways of the book were vast uncertainties around whether certain events would actually lead to major catastrophes (e.g., would a launch lead to a full-scale nuclear war?), whether catastrophes would lead to civilizational collapse (e.g., how severe and long-lasting would the nuclear winter be, and how well would we adapt?), how severe collapses would be (e.g., to pre-industrial or pre-agricultural levels?), and how long-lasting collapses would be (from memory, Ord seems to think recovery is in fact fairly likely).
So I worry that a sentence like that one makes the book sound somewhat alarmist, doomsaying, and naive/simplistic, whereas in reality it seems to me quite nuanced and open about the arguments for why existential risk from certain sources may be “quite low”—and yet still extremely worth attending to, given the stakes.
To be fair, or to make things slightly stranger, Chivers does later say:
Perhaps surprisingly, [Ord] doesn’t think that nuclear war would have been an existential catastrophe. It might have been — a nuclear winter could have led to sufficiently dreadful collapse in agriculture to kill everyone — but it seems unlikely, given our understanding of physics and biology.
(Also, as an incredibly minor point, I think the relevant appendix was Appendix C rather than D. But maybe that was different in different editions or in an early version Chivers saw.)
“Numerically small”
Secondly, Chivers writes:
[Ord] points out that although the difference between a disaster that kills 99 per cent of us and one that kills 100 per cent would be numerically small, the outcome of the latter scenario would be vastly worse, because it shuts down humanity’s future.
I don’t recall Ord ever saying something like that the death of 1 percent of the population would be “numerically small”. Ord very repeatedly emphasises and reminds the reader that something really can count as deeply or even unprecedently awful, and well worth expending resources to avoid, even if it’s not an existential catastrophe. This seems to me a valuable thing to do, otherwise the x-risk community could easily be seen as coldly dismissive of any sub-existential catastrophes. (Plus, such catastrophes really are very bad and well worth expending resources to avoid—this is something I would’ve said anyway, but seems especially pertinent in the current pandemic.)
I think saying “the difference between a disaster that kills 99 per cent of us and one that kills 100 per cent would be numerically small” cuts against that goal, and again could paint Ord as more simplistic or extremist than he really is.
“Blowing ourselves up”
Finally (for the purpose of my critiques), Chivers writes:
We could live for a billion years on this planet, or billions more on millions of other planets, if we manage to avoid blowing ourselves up in the next century or so.
To me, “avoid blowing ourselves up” again sounds quite informal or naive or something like that. It doesn’t leave me with the impression that the book will be a rigorous and nuanced treatment of the topic. Plus, Ord isn’t primarily concerned with us “blowing ourselves up”—the specific risks he sees as the largest are unaligned AI, engineered pandemics, and “unforeseen anthropogenic risk”.
And even in the case of nuclear war, Ord is quite clear that it’s the nuclear winter that’s the largest source of existentialrisk, rather than the explosions themselves (though of course the explosions are necessary for causing such a winter). In fact, Ord writes “While one often hears the claim that we have enough nuclear weapons to destroy the world may times over, this is loose talk.” (And he explains why this is loose talk.)
So again, this seems like a case where Ord actively separates his clear-headed analysis of the risks from various naive, simplistic, alarmist ideas that are somewhat common among some segments of the public, but where Chivers’ review makes it sound (at least to me) like the book will match those sorts of ideas.
All that said, I should again note that I thought the review did a lot right. In fact, I have no quibbles at all with anything from that last quote onwards.
This was an excellent meta-review! Thanks for sharing it.
I agree that these little slips of language are important; they can easily compound into very stubborn memes. (I don’t know whether the first person to propose a paperclip AI regrets it, but picking a different example seems like it could have had a meaningful impact on the field’s progress.)
What are the implications of the offence-defence balance for trajectories of violence?
Questions: Is a change in the offence-defence balance part of why interstate (and intrastate?) conflict appears to have become less common? Does this have implications for the likelihood and trajectories of conflict in future (and perhaps by extension x-risks)?
Epistemic status: This post is unpolished, un-researched, and quickly written. I haven’t looked into whether existing work has already explored questions like these; if you know of any such work, please comment to point me to it.
Background/elaboration: Pinker argues in The Better Angels of Our Nature that many types of violence have declined considerably over history. I’m pretty sure he notes that these trends are neither obviously ephemeral nor inevitable. But the book, and other research pointing in similar directions, seems to me (and I believe others?) to at least weakly support the ideas that:
if we avoid an existential catastrophe, things will generally continue to get better
apart from the potential destabilising effects of technology, conflict seems to be trending downwards, somewhat reducing the risks of e.g. great power war, and by extension e.g. malicious use of AI (though of course a partial reduction in risks wouldn’t necessarily mean we should ignore the risks)
It is well-understood that technological progress can impact offense-defense balances. In fact, perhaps the primary motivation for developing the concept has been to understand the distinctions between different eras of military technology.
For instance, European powers’ failure to predict the grueling attrition warfare that would characterize much of the First World War is often attributed to their failure to recognize that new technologies, such as machine guns and barbed wire, had shifted the European offense-defense balance for conquest significantly toward defense.
And:
holding force sizes fixed, the conventional wisdom holds that a conflict with mid-nineteenth century technology could be expected to produce a better outcome for the attacker than a conflict with early twentieth century technology. See, for instance, Van Evera, ‘Offense, Defense, and the Causes of War’.
The paper tries to use these sorts of ideas to explore how emerging technologies will affect trajectories, likelihood, etc. of conflict. E.g., the very first sentence is: “The offense-defense balance is a central concept for understanding the international security implications of new technologies.”
But it occurs to me that one could also do historical analysis of just how much these effects have played a role in the sort of trends Pinker notes. From memory, I don’t think Pinker discusses this possible factor in those trends. If this factor played a major role, then perhaps those trends are substantially dependent on something “we” haven’t been thinking about as much—perhaps we’ve wondered about whether the factors Pinker discusses will continue, whereas they’re less necessary and less sufficient than we thought for the overall trend (decline in violence/interstate conflict) that we really care about.
And at a guess, that might mean that that trend is more fragile or “conditional” than we might’ve thought. It might mean that we really really can’t rely on that “background trend” continuing, or at least somewhat offsetting the potentially destabilising effects of new tech—perhaps a lot of the trend, or the last century or two of it, was largely about how tech changed things, so if the way tech changes things changes, the trend could very easily reverse entirely.
I’m not at all sure about any of that, but it seems it would be important and interesting to explore. Hopefully someone already has, in which case I’d appreciate someone pointing me to that exploration.
(Also note that what the implications of a given offence-defence balance even are is apparently somewhat complicated/debatable matter. Eg., Garfinkel and Dafoe write: “While some hold that shifts toward offense-dominance obviously favor conflict and arms racing, this position has been challenged on a number of grounds. It has even been suggested that shifts toward offense-dominance can increase stability in a number of cases.”)
Should reducing partisanship be a higher priority cause area (for me)?
I think political polarization in the US produces a whole heap of really bad societal/policy outcomes and makes otherwise good policy outcomes ~impossible. It has always seemed relatively important to me, because when things go wrong in the US, they often have global consequences. I haven’t put that many of my actual resources here though because it’s a draining cause to work on and didn’t feel that tractable. I also suspected myself of motivated reasoning: I get deep joy from inter-group cooperation and am very distressed by inter-group conflict.
Then I read things like the thread below and feel like not paying more attention to this is foolish, like I’ve gone too far in the other direction and underweighted the importance of this barrier to global coordination. I imagine others have written about similar questions and I would be interested in more thoughts.
Something else in the vein of “things EAs and rationalists should be paying attention to in regards to Corona.”
There’s a common failure mode in large human systems where one outlier causes us to create a rule that is a worse equilibrium. In the PersonalMBA, Josh Kaufman talks about someone taking advantage of a “buy any book you want” rule that a company has—so you make it so that you can no longer get any free books.
This same pattern has happened before in the US, after 9-11 - We created a whole bunch of security theater, that caused more suffering for everyone, and gave government way more power and way less oversight than is safe, because we over-reacted to prevent one bad event, not considering the counterfactual invisible things we would be losing.
This will happen again with Corona, things will be put in place that are maybe good at preventing pandemics (or worse, making people think they’re safe from pandemics), but create a million trivial conveniences every day that add up to more strife than they’re worth.
These types of rules are very hard to repeal after the fact because of absence blindness—someone needs to do the work of calculating the cost/benefit ratio BEFORE they get implemented, then build a convincing enough narrative to what seems obvious/common sense measures given the climate/devastation.
I’ve been thinking more lately about how I should be thinking about causal effects for cost-effectiveness estimates, in order to clarify my own skepticism of more speculative causes, especially longtermist ones, and better understand how skeptical I ought to be. Maybe I’m far too skeptical. Maybe I just haven’t come across a full model for causal effects that’s convincing since I haven’t been specifically looking. I’ve been referred to this in the past, and plan to get through it, since it might provide some missing pieces for the value of research.
Suppose I have two random variables, X and Y, and I want to know the causal effect of manipulating X on Y, if any.
1. If I’m confident there’s no causal relationship between the two, say due to spatial separation, I assume there is no causal effect, and Y conditional on the manipulation of X to take value A (possibly random), Y|do(X=A), is identical to Y, i.e. Y|do(X=A)=Y. (The do notation is Pearl’s do-calculus notation.)
2. If X could affect Y, but I know nothing else,
a. I might assume, based on symmetry (and chaos?) for Y, that Y|do(X=A) and Y are identical in distribution, but not necessarily literally equal as random variables. They might be slightly “shuffled” or permuted versions of each other (see symmetric decreasing rearrangements for specific examples of such a permutation). The difference in expected values is still 0. This is how I think about the effects of my every day decisions, like going to the store, breathing at particular times, etc. on future populations. I might assume the same for variables that depend on Y.
b. Or, I might think that manipulating X just injects noise into Y, possibly while preserving some of its statistics, e.g. the mean or median. A simple case is just adding random symmetric noise with mean and median 0 to Y. However, whether or not a statistic is preserved with the extra noise might be sensitive to the scale on which Y is measured. For example, if Y is real-valued, and f:R→R is strictly increasing, then for the median, med(f(Y))=f(med(Y)), but the same is not necessarily true for the expected value of Y, or for other variables that depend on Y.
c. Or, I might think that manipulating X makes Ycloser to a “default” distribution over the possible values of Y, often but not always uninformed or uniform. This can shift the mean, median, etc., of Y. For example, Y could be the face of the coin I see on my desk, and X could be whether I flip the coin or not, with X being not by default. So, if I do flip the coin and hence manipulate X, this randomizes the value of Y, making my probability distribution for its value uniformly random instead of a known, deterministic value. You might think that some systems are the result of optimization and therefore fragile, so random interventions might return them to prior “defaults”, e.g. naive systemic change or changes to ecosystems. This could be (like) regression to the mean.
I’m not sure how to balance these three possibilities generally. If I do think the effects are symmetric, I might go with a or b or some combination of them. In particular asymmetric cases, I might also combine c.
3. Suppose I have a plausible argument for how X could affect Y in a particular way, but no observations that can be used as suitable proxies, even very indirect, for counterfactuals with which to estimate the size of the effect. I lean towards dealing with this case as in 2, rather than just making assumptions about effect sizes without observations.
For example, someone might propose a causal path through which X affects Y with a missing estimate of effect size at at least one step along the path, but an argument to that this should increase the value of Y. It is not enough to consider only one such path, since there may be many paths from X to Y, e.g. different considerations for how X could affect Y, and these would need to be combined. Some could have opposite effects. By 2, those other paths, when combined with the proposed causal path, reduce the effects of X on Y through the proposed path. The longer the proposed path, the more unknown alternate paths.
I think this is where I am now with speculative longtermist causes. Part of this may be my ignorance of the proposed causal paths and estimates of effect sizes, since I haven’t looked too deeply at the justifications for these causes, but the dampening from unknown paths also applies when the effect sizes along a path are known, which is the next case.
4. Suppose I have a causal path through some other variable Z, X→Z→Y, so that X causes Z and Z causes Y, and I model both the effects of X→Z and Z→Y, based on observations. Should I just combine the two for the effect of X on Y? In general, not in the straightforward way. As in 3, there could be another causal path, X→Z′→Y (and it could be longer, instead of with just a single intermediate variable).
As in case 3, you can think of X→Z′→Y as dampening the effect of X→Z→Y, and with long proposed causal paths, we might expect the net effect to be small, consistently with the intuition that the predictable impacts on the far future decrease over time due to ignorance/noise and chaos, even though the actual impacts may compound due to chaos.
Maybe I’ll write this up as a full post after I’ve thought more about it. I imagine there’s been writing related to this, including in the EA and rationality communities.
I think EA hasn’t sufficiently explored the use of different types of empirical studies from which we can rigorously estimate causal effects, other than randomized controlled trials (or other experiments). This leaves us either relying heavily on subjective estimates of the magnitudes of causal effects based on weak evidence, anecdotes, expert opinion or basically guesses, or being skeptical of interventions whose cost-effectiveness estimates don’t come from RCTs. I’d say I’m pretty skeptical, but not so skeptical that I think we need RCTs to conclude anything about the magnitudes of causal effects. There are methods to do causal inference from observational data.
2. Relying too much on guesses and poor studies in the effective animal advocacy space (especially in the past), for example overestimating the value of leafletting. I think things have improved a lot since then, and I thought the evidence presented in the work of Rethink Priorities, Charity Entrepreneurship and Founders Pledge on corporate campaigns was good enough to meet the bar for me to donate to support corporate campaigns specifically. Humane League Labs and some academics have done and are doing research to estimate causal effects from observational data that can inform EAA.
Let’s consider a given preference from the point of view of a given outcome after choosing it, in which the preference either exists or does not, by cases:
1. The preference exists:
a. If there’s an outcome in which the preference exists and is more satisfied, and all else is equal, it would have been irrational to have chosen this one (over it, and at all).
b. If there’s an outcome in which the preference exists and is less satisfied, and all else is equal, it would have been irrational to have chosen the other outcome (over this one, and at all).
c. If there’s an outcome in which the preference does not exist, and all else is equal, the preference itself does not tell us if either would have been irrational to have chosen.
2. The preference doesn’t exist:
a. If there’s an outcome in which the preference exists, regardless of its degree of satisfaction, and all else equal, the preference itself does not tell us if either would have been irrational to have chosen.
So, all else equal besides the existence or degree of satisfaction of the given preference, it’s always rational to choose an outcome in which the preference does not exist, but it’s irrational to choose an outcome in which the preference exists but is less satisfied than in another outcome.
(I made a similar argument in the thread starting here.)
I also think that antifrustrationism in some sense overrides interests less than symmetric views. Consider the following two options for interests within one individual:
A. Interest 1 exists and is fully satisfied
B. Interest 1 exists and is not fully satisfied, and interest 2 exists and is (fully) satisfied.
A symmetric view would sometimes choose B, so that the creation of interests can take priority over interests that would exist regardless. In particular, the proposed benefit comes from satisfying an interest that would not have existed in the alternative, so it seems like we’re overriding the interests the individual would have in A with a new interest, interest 2. For example, we make someone want something and satisfy that want, at the expense of their other interests.
On the other hand, consider:
A. Interest 1 exists and is partially unsatisfied
B. Interest 1 exists and is fully satisfied, and interest 2 exists and is partially unsatisfied.
In this case, antifrustrationism would sometimes choose A, so that the removal or avoidance of an otherwise unsatisfied interest can take priority over (further) satisfying an interest that would exist anyway. But in this case, if we choose A because of concerns for interest 2, at least interest 2 would exist in the alternative A, so the benefit comes from the avoidance of an interest that would have otherwise existed. In A, compared to B, I wouldn’t say we’re overriding interests, we’re dealing with an interest, interest 2, that would have existed otherwise.
Some related writings, although not making the same point I am here:
Then, if you extend these comparisons to satisfy the independence of irrelevant alternatives by stating that in comparisons of multiple choices in an option set, all permissible options are strictly better than all impermissible options regardless of option set, extending these rankings beyond the option set, the result is antifrustrationism. To show this, you can use the set of the following three options, which are identical except in the ways specified:
A: a preference exists and is fully satisfied,
B: the same preference exists and is not fully satisfied, and
C: the preference doesn’t exist,
and since B is impermissible because of the presence of A, this means C>B, and so it’s always better for a preference to not exist than for it to exist and not be fully satisfied, all else equal.
I also think this argument isn’t specific to preferences, but could be extended to any interests, values or normative standards that are necessarily held by individuals (or other objects), including basically everything people value (see here for a non-exhaustive list). See Johann Frick’s paper and thesis which defend the procreation asymmetry, and my other post here.
- [ETA: this point is likely wrong, cf. Khorton’s comment below. However, I believe the conclusion that the timing of WHO declarations by itself doesn’t provide a significant argument against epistemic modesty still stands, as I explain in a follow-up comment below.] The WHO declaring a pandemic has a bunch of major legal and institutional consequences. E.g. my guess is that among other things it affects the amounts of resources the WHO and other actors can utilize, the kind of work the WHO and others are allowed to do, and the kind of recommendations the WHO can make.
- The optimal time for the WHO to declare a pandemic is primarily determined by these legal and institutional consequences. Whether COVID-19 is or will in fact be a pandemic in the everyday or epidemiological sense is an important input into the decision, but not a decisive one.
- Without familiarity with the WHO and the legal and institutional system it is a part of, it is very difficult to accurately assess the consequences of the WHO declaring a pandemic. Therefore, it is very hard to evaluate the timing of the WHO’s declaration without such familiarity. And being even maximally well-informed about COVID-19 itself isn’t even remotely sufficient for an accurate evaluation.
- The bottom line is that the WHO officially declaring that COVID-19 is a pandemic is a totally different thing from any individual persuasively arguing that COVID-19 is or will be a pandemic. In a language that would accurately reflect differences in meaning, me saying that COVID-19 is a pandemic and the WHO declaring COVID-19 is a pandemic would be done using different words. It is simply not the primary purpose of this WHO speech act to be an early, accurate, reliable, or whatever indicator of whether “COVID-19 is a pandemic”, to predict its impact, or any other similar thing. It isn’t primarily epistemic in any sense.
- If just based on information about COVID-19 itself someone confidently thinks that the WHO ought to have declared a pandemic earlier, they are making a mistake akin to the mistake reflected by answering “yes” to the question “could you pass me the salt?” without doing anything.
So did the WHO make a mistake by not declaring COVID-19 to be a pandemic earlier, and if so how consequential was it? Well, I think the timing was probably suboptimal just because my prior is that most complex institutions aren’t optimized for getting the timing of such things exactly right. But I have no idea how consequential a potential mistake was. In fact, I’m about 50-50 on whether the optimal time would have been slightly earlier or slightly later. (Though substantially earlier seems significantly more likely optimal than substantially later.)
“The WHO declaring a pandemic has a bunch of major legal and institutional consequences. E.g. my guess is that among other things it affects the amounts of resources the WHO and other actors can utilize, the kind of work the WHO and others are allowed to do, and the kind of recommendations the WHO can make.”
Are you sure about this? I’ve read that there aren’t major implications to it being officially declared a pandemic.
[Note by me: The International Health Regulations include multiple instances of “public health emergency of international concern”. By contrast, they include only one instance of “pandemic”, and this is in the term “pandemic influenza” in a formal statement by China rather than the main text of the regulation.]
The WHO declared a PHEIC due to COVID-19 on January 30th.
The OP was prompted by a claim that the timing of the WHO using the term “pandemic” provides an argument against epistemic modesty. (Though I appreciate this was less clear in the OP than it could have been, and maybe it was a bad idea to copy my Facebook comment here anyway.) From the Facebook comment I was responding to:
For example, to me, the WHO taking until ~March 12 to call this a pandemic*, when the informed amateurs I listen to were all pretty convinced that this will be pretty bad since at least early March, is at least some evidence that trusting informed amateurs has some value over entirely trusting people usually perceived as experts.
Since the WHO declaring a PHEIC seems much more consequential than them using the term “pandemic”, the timing of the PHEIC declaration seems more relevant for assessing the merits of the WHO response, and thus for any argument regarding epistemic modesty.
Since the PHEIC declaration happened significantly earlier, any argument based on the premise that it happened too late is significantly weaker. And whatever the apparent initial force of this weaker argument, my undermining response from the OP still applies.
So overall, while the OP’s premise appealing to major legal/institutional consequences of the WHO using the term “pandemic” seems false, I’m now even more convinced of the key claim I wanted to argue for: that the WHO response does not provide an argument against epistemic modesty in general, nor for the epistemic superiority of “informed amateurs” over experts on COVID-19.
For example, to me, the WHO taking until ~March 12 to call this a pandemic*, when the informed amateurs I listen to were all pretty convinced that this will be pretty bad since at least early March, is at least some evidence that trusting informed amateurs has some value over entirely trusting people usually perceived as experts.
Also, predicting that something will be pretty bad or will be a pandemic is not the same as saying it is now a pandemic. When did it become a pandemic according to the WHO’s definition?
Dr Fukuda: An easy way to think about pandemic – and actually a way I have some times described in the past – is to say: a pandemic is a global outbreak. Then you might ask yourself: “What is a global outbreak”? Global outbreak means that we see both spread of the agent – and in this case we see this new A(H1N1) virus to most parts of the world – and then we see disease activities in addition to the spread of the virus. Right now, it would be fair to say that we have an evolving situation in which a new influenza virus is clearly spreading, but it has not reached all parts of the world and it has not established community activity in all parts of the world. It is quite possible that it will continue to spread and it will establish itself in many other countries and multiple regions, at which time it will be fair to call it a pandemic at that point. But right now, we are really in the early part of the evolution of the spread of this virus and we will see where it goes.
About declaring it a “pandemic,” I’ve seen the WHO reason as follows (me paraphrasing):
«Once we call it a pandemic, some countries might throw up their hands and say “we’re screwed,” so we should better wait before calling it that, and instead emphasize that countries need to try harder at containment for as long as there’s still a small chance that it might work.»
So overall, while the OP’s premise appealing to major legal/institutional consequences of the WHO using the term “pandemic” seems false, I’m now even more convinced of the key claim I wanted to argue for: that the WHO response does not provide an argument against epistemic modesty in general, nor for the epistemic superiority of “informed amateurs” over experts on COVID-19.
Yeah, I think that’s a good point.
I’m not sure I can have updates in favor or against modest epistemology because it seems to me that my true rejection is mostly “my brain can’t do that.” But if I could have further updates against modest epistemology, the main Covid-19-related example for me would be how long it took some countries to realize that flattening the curve instead of squishing it is going to lead to a lot more deaths and tragedy than people seem to have initially thought. I realize that it’s hard to distinguish between what’s actual government opinion versus what’s bad journalism, but I’m pretty confident there was a time when informed amateurs could see that experts were operating under some probably false or at least dubious assumptions. (I’m happy to elaborate if anyone’s interested.)
Thank you for pointing this out! It sounds like my guess was probably just wrong.
My guess was based on a crude prior on international organizations, not anything I know about the WHO specifically. I clarified the epistemic status in the OP.
Collection of ways of classifying existential risk pathways/mechanisms
Each of the following works show or can be read as showing a different model/classification scheme/taxonomy:
Defence in Depth Against Human Extinction:Prevention, Response, Resilience, and Why They All Matter—Cotton-Barratt, Daniel, and Sandberg, 2020
The same model is also discussed in Toby Ord’s The Precipice.
Classifying global catastrophic risks—Avin et al., 2018
Causal diagrams of the paths to existential catastrophe—Michael Aird, 2020
Conflict of interest statement: I am the aforementioned human.
This might not quite “belong” in this list. But one could classify risks by which of the different “paths” they might follow (e.g., those that would vs wouldn’t “pass through” a distinct collapse stage).
Typology of human extinction risks—Alexey Turchin, ~2015
Related LessWrong post
Personally, I think the model/classification scheme in Defence in Depth is probably the most useful. But I think at least a quick skim of the above sources is useful; I think they each provide an additional useful angle or tool for thought.
I intend to add to this list over time. If you know of other relevant work, please mention it in a comment.
Wait, exactly what are you actually collecting here?
The scope of this collection is probably best revealed by checking out the above sources.
But to further clarify, here are two things I don’t mean, which aren’t included in the scope:
Classifications into things like “AI risk vs biorisk”, or “natural vs anthropogenic”
Such categorisation schemes are clearly very important, but they’re also well-established and you probably don’t need a list of sources that show them.
Classifications into different “types of catastrophe”, such as Ord’s distinction between extinction, unrecoverable collapse, and unrecoverable dystopia
This is also very important, and maybe I should make such a collection at some point, but it’s a separate matter to this.
Initial thoughts on anti-ageing and societal resilience to pandemics
[Epistemic status: fairly speculative. Practicing throwing out less-polished ideas as shortform. Might tidy up and promote this to a frontpage post in the future.]
I’m hardly the first to note that COVID-19 is a disease of ageing: from the beginning of the outbreak it’s been clear that older people were dramatically more likely to experience severe illness or die as a result of infection, while death rates among under-30s have been extremely low. In an important sense[1], COVID-19 falls into the same category as cancer, heart disease, and dementia: a symptom of ageing, which is currently treated independently, but which in the longer term could be much more efficiently treated by tackling ageing itself[2].
The same observation can be generalised to respiratory diseases more generally, and (less strongly) to infectious diseases as a whole. Seasonal influenza is overwhelmingly an affliction of older people, with >90% of fatalities occurring in individuals over 65. The original SARS outbreak showed a similar age-skewing to the current pandemic (though with higher CFRs across the board), with <1% mortality among cases under 24 years old and >55% mortality in over-65s. Case fatality rates in adults increase with age for ebola, tuberculosis, and most other disease you might want to check[3]. As a general rule, if a disease is novel (i.e. there is no acquired immunity in the population), older adults[4] who catch it will fare much worse[5]. This isn’t surprising: older people have weaker and less adaptable immune systems than young adults and show much higher general frailty.
I’m interested in whether this phenomenon provides a good additional argument for anti-ageing research as a long-termist cause area. In general, the healthier and more robust the population is, the less we have to fear from catastrophic pandemics. Of course, a truly existential pandemic would kill the young as well as the old: trivially, for a pandemic to kill everyone, it must kill young people. But I’d expect that many sub-extinction catastrophic pandemics would kill a much higher proportion of older adults than younger ones. Since our population is ageing, this will only become a bigger and bigger problem with time.
The worse a pandemic is, the bigger we can expect its effects to be on society. COVID-19, a much milder disease than a hypothetical GCBR, has already caused huge changes and is widely expected to leave long-lasting effects on politics and policy; effects that would almost certainly never have happened in the absence of ageing. The dislocation caused by a worse pandemic could be far more drastic. If anti-ageing reduced a future biological catastrophe to “only” COVID-19 levels that would be a big win. Generalising from this, ending ageing could act as an important “anti-risk factor”, turning borderline existential biological risks into non-existential GCBRs and borderline GCBRs into “mere catastrophes”.
Similar (but shakier) arguments can be made about some other kinds of catastrophe: if any sort of physical health and endurance is required at a population level to survive them and avoid large negative trajectory changes, ending ageing could be very helpful.
I’m not claiming this in itself would be sufficient to justify the huge research expense required to defeat ageing; many pandemics would probably not be as age-skewed as COVID-19, and if our only goal were to defeat pandemics then broad-spectrum antivirals etc. would be more obvious low-hanging fruit to pick. But I think it might militate towards being substantially more favourable to anti-ageing research, from a long-termist perspective, than one otherwise would be.
Obviously, in another important sense COVID-19 is very different: unlike, say, heart disease, COVID-19 has a single, clear, exogenous cause which can be effectively fought with a variety of short-term measures like social distancing. It would be silly to suggest we attack COVID-19 with anti-ageing research in the short term; my claim is that, over the span of decades required to do so, solving ageing would dramatically cut the death toll from both endemic and emerging respiratory diseases. ↩︎
More generally, the reference class of “pandemics similar to COVID-19” falls into this category. ↩︎
The stand-out exception to this trend is the 1918 flu pandemic, which showed three peaks of mortality: early childhood, very old age, and young adulthood. In this pandemic the death rate of 25-year-olds was higher than that of 65-year-olds. There seem to be several competing explanations for this, some of which suggest a biological vulnerability of younger adults and others of which are more circumstantial; I’m not going to dig into it here except to say that this is a very important exception to the general rule I’m discussing here. The 2009 flu pandemic, also H1N1, also killed unusually large numbers of younger adults. ↩︎
Of course, young children, who have both weaker immune systems and minimal acquired immunity, often fare worst of all. ↩︎
This is not always obvious from the raw mortality breakdown, if older people are much rarer than young adults in a society, or if the spread of the disease is concentrated among particular groups that are age-skewed. ↩︎
This is a really interesting point. An additional consideration is that global leaders tend to be older, and hence more at risk (cf. Boris Johnson). You could imagine that their deaths are especially destabilizing.
If the longtermist argument for preventing pandemics is that they trigger destabilization which leads to, say, nuclear war, the age impacts could be an important factor.
List of things I’ve written or may write that are relevant to The Precipice
Things I’ve written
My thoughts on Toby Ord’s existential risk estimates
My Google Play review
My review of Tom Chivers’ review of Toby Ord’s The Precipice
If a typical mammalian species survives for ~1 million years, should a 200,000 year old species expect another 800,000 years, or another million years?
Working titles of things I plan/vaguely hope to write
Note: If you might be interested in writing about similar ideas, feel very free to reach out to me. It’s very unlikely I’ll be able to write all of these posts by myself, so potentially we could collaborate, or I could just share my thoughts and notes with you and let you take it from there.
Defining existential risks and existential catastrophes
My thoughts on Toby Ord’s policy & research recommendations
Existential security
Civilizational collapse and recovery: Toby Ord’s views and my doubts
The Terrible Funnel: Estimating odds of each step on the x-risk causal path (working title)
The idea here would be to adapt something like the “Great Filter” or “Drake Equation” reasoning to estimating the probability of existential catastrophe, using how humanity has fared in prior events that passed or could’ve passed certain “steps” on certain causal chains to catastrophe.
E.g., even though we’ve never faced a pandemic involving a bioengineered pathogen, perhaps our experience with how many natural pathogens have moved from each “step” to the next one can inform what would likely happen if we did face a bioengineered pathogen, or if it did get to a pandemic level.
This idea seems sort of implicit in the Precipice, but isn’t really spelled out there. Also, as is probably obvious, I need to do more to organise my thoughts on it myself.
This may include discussion of how Ord distinguishes natural and anthropogenic risks, and why the standard arguments for an upper bound for natural extinction risks don’t apply to natural pandemics. Or that might be a separate post.
Developing—but not deploying—drastic backup plans
“Macrostrategy”: Attempted definitions and related concepts
This would relate in part to Ord’s concept of “grand strategy for humanity”
Collection of notes
A post summarising the ideas of existential risk factors and existential security factors?
I suspect I won’t end up writing this, but I think someone should. For one thing, it’d be good to have something people can reference/link to that explains that idea (sort of like the role EA Concepts serves).
Some selected Precipice-related works by others
80,000 Hours’ interview with Toby Ord
Slate Star Codex’s review of the book
FLI Podcast interview with Toby Ord
Do emergency universal pass/fail policies improve or worsen student well-being and future career prospects?
I think a natural experiment is in order. Many colleges are adopting universal pass/fail grading for this semester in response to the COVID-19 pandemic, while others aren’t. Someone should study the impact this will have on students to inform future university pandemic response policy.
When suggestions of this type come up, especially for causes that don’t have existing EA research behind them, my recommended follow-up is to look for people who study this as normal academics (here, “this” would be “ways that grades and grading policy influence student outcomes”). Then, write to professors who do this work and ask if they plan on taking advantage of the opportunity (here, the natural experiment caused by new grading policies).
There’s a good chance that the people you write to will have had this idea already (academics who study a subject are frequently on the lookout for opportunities of this kind, and the drastic changes wrought by COVID-19 should be increasing the frequency with which people think about related studies they could run). And if they haven’t, you have the chance to inspire them!
Writing to random professors could be intimidating, but in my experience, even when I’ve written emails like this as a private citizen without a .edu email address, I frequently get some kind of response; people who’ve made research their life’s work are often happy to hear from members of the public who care about the same odd things they do.
Thanks for the suggestion! I imagine that most scholars are reeling from the upheavals caused by the pandemic response, so right now doesn’t feel like the right time to ask professors to do anything. What do you think?
Maybe a better question for late May or early June, when classes are over.
I think that’s probably true for those working directly on the pandemic, but I’m not sure education researchers would mind being bothered. If anything they might welcome the distraction.
If a typical mammalian species survives for ~1 million years, should a 200,000 year old species expect another 800,000 years, or another million years?
tl;dr I think it’s “another million years”, or slightly longer, but I’m not sure.
In The Precipice, Toby Ord writes:
(There are various extra details and caveats about these estimates in the footnotes.)
Ord also makes similar statements on the FLI Podcast, including the following:
I think this is a strong analogy from a poetic perspective. And I think that highlighting the typical species’ lifespan is a good starting point for thinking about how long we might have left. (Although of course we could also draw on many other facts for that analysis, as Ord discusses in the book.)
But I also think that there’s a way in which the lifespan analogy might be a bit misleading. If a human is 70, we expect they have less time less to live than if a human is 20. But I’m not sure whether, if a species if 700,000 years old, we should expect that species to go extinct sooner than a species that is 200,000 years old will.
My guess would be that a ~1 million year lifespan for a typical mammalian species would translate into a roughly 1 in a million chance of extinction each year, which doesn’t rise or fall very much in a predictable way over most of the species’ lifespan. Specific events, like changes in a climate or another species arriving/evolving, could easily change the annual extinction rate. But I’m not aware of an analogy here to how ageing increases the annual risk of humans dying from various causes.
I would imagine that, even if a species has been around for almost or more than a million years, we should still perhaps expect a roughly 1 in a million chance of extinction each year. Or perhaps we should even expect them to have a somewhat lower annual chance of extinction, and thus a higher expected lifespan going forwards, based on how long they’ve survived so far?
(But I’m also not an expert on the relevant fields—not even certain what they would be—and I didn’t do extra research to inform this shortform comment.)
I don’t think that Ord actually intends to imply that species’ “lifespans” work like humans’ lifespans do. But the analogy does seem to imply it. And in the FLI interview, he does seem to briefly imply that, though of course there he was speaking off the cuff.
I’m also not sure how important this point is, given that humans are very atypical anyway. But I thought it was worth noting in a shortform comment, especially as I expect that, in the wake of The Precipice being great, statements along these lines may be quoted regularly over the coming months.
If I were to read one of EA-related books (e.g. Doing Good Better, The Most Good You Can Do, The Life You Can Save, The Precipice, Superintelligence, etc.), I would consider writing/improving a summary of the book in wikipedia while reading it, in a way that conveys main points well. It could help you to digest the book better and help others to understand the ideas a bit. You could do it in english as well as maybe in some other language. To see whether it’s worth putting in the effort, you can check out Wikipedia pageview statistics of the books I mentioned and others here (it doesn’t include some views that come from redirects though). It seems that the page Superintelligence is the most viewed one out of these with an average of 4,597 monthly visitors.
Nuke insurance
Category: Intervention idea
Epistemic status: speculative; arm-chair thinking; non-expert idea; unfleshed idea
Proposal: Have nuclear powers insure each other that they won’t nuke each other for mutually assure destruction (ie. destroying my infrastructure means you will destroy your economy). Not accepting an offered of mutual insurances should be seen as extremely hostile and uncooperative, and possible even be severely sanctioned internationally.
Also: what about just explicitly criminalizing a) a first strike, b) a nuclear attack? The idea is to make it more likely that the individuals who participated in a nuclear strike would be punished—even if they considered it to be morally justified.
(Someone will certainly think this is “serious April Fool’s stuff”)
Good point. My implicit idea was to have the money in an independent trust, so that the “punishment” is easier to enforce.
Does anyone have any idea / info on what proportion of the infected cases are getting Covid19 inside hospitals?
(Epistemic status: low, but I didin’t find any research on that, so the hypothesis deserves a bit more of attention)
1. Nosocomial infections are serious business. Hospitals are basically big buildings full of dying people and the stressed personel who goes from one bed to another try to avoid it. Throw a deadly and very contagious virus in it, and it becomes a slaughterhouse.
2. Previous coronavirus were rapidly spread in hospitals and other care units. That made South East Asia kinda prepared for possibly similar epidemics (maybe I’m wrong, but in news their medical staff is always in Hazmat suits, unlike most Health workers in the West). Maybe this is a neglected point in the successful approach in South East Asia?
3. I know hospitals have serious protocols to avoid it… but it takes only a few careless cleaning staff, or a patient’s relatives going to cafeteria, or a badly designed airflow, to ruin everything. Just one Hospital chain in Brazil concentrates most of deaths in Sao Paulo, and 40% of the total national.
https://www.theguardian.com/world/2020/mar/24/woman-first-uk-victim-die-coronavirus-caught-hospital-marita-edwards
CoronaVirus and Famine
The Good Judgement Open forecasting tournament gives a 66% chance for the answer to “Will the UN declare that a famine exists in any part of Ethiopia, Kenya, Somalia, Tanzania, or Uganda in 2020?”
https://www.gjopen.com/questions/1559-will-the-un-declare-that-a-famine-exists-in-any-part-of-ethiopia-kenya-somalia-tanzania-or-uganda-in-2020
I think that the 66% is a slight overestimate. But nonetheless, if a famine does hit, it would be terrible, as other countries might not be able to spare enough attention due to the current pandemic.
https://ourworldindata.org/what-does-a-famine-declaration-declare
https://fews.net/
https://www.gjopen.com/questions/1559-will-the-un-declare-that-a-famine-exists-in-any-part-of-ethiopia-kenya-somalia-tanzania-or-uganda-in-2020 (registration needed to see)
It is not clear to me what an altruist who realizes that can do, as an individual:
A famine is likely to hit this region (but hasn’t hit yet)
It is likely to be particularly bad.
Donating to the World Food Programme, which is already doing work on the matter, might be a promising answer, but I haven’t evaluated the programe, nor compared it to other potentially promising options (see here: https://forum.effectivealtruism.org/posts/wpaZRoLFJy8DynwQN/the-best-places-to-donate-for-covid-19, or https://www.againstmalaria.com/)
https://donatenow.wfp.org/wfp/~my-donation
https://www.wfp.org/
https://insight.wfp.org/coronavirus-wfp-ready-to-rise-to-the-challenge-8178a4d0aeac
Did you mean to post this using the Markdown editor? Currently, the formatting looks a bit odd from a reader’s perspective.
Collection of sources related to dystopias and “robust totalitarianism”
The Precipice—Toby Ord (Chapter 5 has a full section on Dystopian Scenarios)
The Totalitarian Threat—Bryan Caplan (a link to a Word doc version can be found on this page) (some related discussion on the 80k podcast here; use the “find” function)
The Centre for the Governance of AI’s research agenda—Allan Dafoe (this contains discussion of “robust totalitarianism”, and related matters)
A shift in arguments for AI risk—Tom Sittler (this has a brief but valuable section on robust totalitarianism) (discussion of the overall piece here)
Existential Risk Prevention as Global Priority—Nick Bostrom (this discusses the concepts of “permanent stagnation” and “flawed realisation”, and very briefly touches on their relevance to e.g. lasting totalitarianism)
I intend to add to this list over time. If you know of other relevant work, please mention it in a comment.
Mind-readers as a neglected life extension strategy
Last updated: 2020-03-30
Status: idea to integrate in a longer article
Assuming that:
Death is bad
Lifelogging is a bet worth taking as a life extension strategy
It seems like a potentially really important and neglected intervention is improving mind readers as this is by far the most important part of our experience that isn’t / can’t be captured at the moment.
We don’t actually need to be able to read the mind right now, just to be able to record the mind with sufficiently high resolution (plausibly along text and audio recording to be able to determine which brain patterns correspond to what kind of thoughts).
Questions:
Assuming we had extremely good software, how much could we read minds with our current hardware? (ie. how much is it worth recording your thoughts right now?)
How inconvenient would it be? How much would it cost?
To do:
Ask on Metaculus some operationalisation of the first question
My review of Tom Chivers’ review of Toby Ord’s The Precipice
I thought The Precipice was a fantastic book; I’d highly recommend it. And I agree with a lot about Chivers’ review of it for The Spectator. I think Chivers captures a lot of the important points and nuances of the book, often with impressive brevity and accessibility for a general audience. (I’ve also heard good things about Chivers’ own book.)
But there are three parts of Chivers’ review that seem to me to like they’re somewhat un-nuanced, or overstate/oversimplify the case for certain things, or could come across as overly alarmist.
I think Ord is very careful to avoid such pitfalls in The Precipice, and I’d guess that falling into such pitfalls is an easy and common way for existential risk related outreach efforts to have less positive impacts than they otherwise could, or perhaps even backfire. I understand that a review gives on far less space to work with than a book, so I don’t expect anywhere near the level of nuance and detail. But I think that overconfident or overdramatic statements of uncertain matters (for example) can still be avoided.
I’ll now quote and comment on the specific parts of Chivers’ review that led to that view of mine.
An alleged nuclear close call
Firstly, in my view, there are three flaws with the opening passage of the review:
First issue: Toby Ord makes it clear that “the incident I shall describe has been disputed, so we cannot yet be sure whether it occurred.” Ord notes that “others who claimed to have been present in the Okinawa missile bases at the time” have since challenged this account, although there is also “some circumstantial evidence” supporting the account. Ultimately, Ord concludes “In my view this alleged incident should be taken seriously, but until there is further confirmation, no one should rely on it in their thinking about close calls.” I therefore think Chivers should’ve made it clear that this is a disputed story.
Second issue: My impression from the book is that, even in the account of the person claiming this story is true, the two men sent down the corridor did not turn out to be necessary to avert the launch. (That said, the book isn’t explicit on the point, so I’m unsure.) Ord writes that Bassett “telephoned the Missile Operations Centre, asking the person who radioed the order to either give the DEFCON 1 order or issue a stand-down order. A stand-down order was quickly given and the danger was over.” That is the end of Ord’s retelling of the account itself (rather than discussion of the evidence for or against it).
Third issue: I think it’s true that, if a nuclear launch had occurred in that scenario, a large-scale nuclear war probably would’ve occurred (though it’s not guaranteed, and it’s hard to say). And if that happened, it seems technically true that Chivers probably would’ve have written this review. But I think that’s primarily because history would’ve just unfolded very, very difficulty. Chivers seems to imply this is because civilization probably would’ve collapsed, and done so so severely than even technologies such as pencils would be lost and that they’d still be lost all these decades on (such that, if he was writing this review, he’d do so with “a charred stick on a rock”).
This may seem like me taking a bit of throwaway rhetoric or hyperbole too seriously, and that may be so. But I think among the key takeaways of the book were vast uncertainties around whether certain events would actually lead to major catastrophes (e.g., would a launch lead to a full-scale nuclear war?), whether catastrophes would lead to civilizational collapse (e.g., how severe and long-lasting would the nuclear winter be, and how well would we adapt?), how severe collapses would be (e.g., to pre-industrial or pre-agricultural levels?), and how long-lasting collapses would be (from memory, Ord seems to think recovery is in fact fairly likely).
So I worry that a sentence like that one makes the book sound somewhat alarmist, doomsaying, and naive/simplistic, whereas in reality it seems to me quite nuanced and open about the arguments for why existential risk from certain sources may be “quite low”—and yet still extremely worth attending to, given the stakes.
To be fair, or to make things slightly stranger, Chivers does later say:
(Also, as an incredibly minor point, I think the relevant appendix was Appendix C rather than D. But maybe that was different in different editions or in an early version Chivers saw.)
“Numerically small”
Secondly, Chivers writes:
I don’t recall Ord ever saying something like that the death of 1 percent of the population would be “numerically small”. Ord very repeatedly emphasises and reminds the reader that something really can count as deeply or even unprecedently awful, and well worth expending resources to avoid, even if it’s not an existential catastrophe. This seems to me a valuable thing to do, otherwise the x-risk community could easily be seen as coldly dismissive of any sub-existential catastrophes. (Plus, such catastrophes really are very bad and well worth expending resources to avoid—this is something I would’ve said anyway, but seems especially pertinent in the current pandemic.)
I think saying “the difference between a disaster that kills 99 per cent of us and one that kills 100 per cent would be numerically small” cuts against that goal, and again could paint Ord as more simplistic or extremist than he really is.
“Blowing ourselves up”
Finally (for the purpose of my critiques), Chivers writes:
To me, “avoid blowing ourselves up” again sounds quite informal or naive or something like that. It doesn’t leave me with the impression that the book will be a rigorous and nuanced treatment of the topic. Plus, Ord isn’t primarily concerned with us “blowing ourselves up”—the specific risks he sees as the largest are unaligned AI, engineered pandemics, and “unforeseen anthropogenic risk”.
And even in the case of nuclear war, Ord is quite clear that it’s the nuclear winter that’s the largest source of existential risk, rather than the explosions themselves (though of course the explosions are necessary for causing such a winter). In fact, Ord writes “While one often hears the claim that we have enough nuclear weapons to destroy the world may times over, this is loose talk.” (And he explains why this is loose talk.)
So again, this seems like a case where Ord actively separates his clear-headed analysis of the risks from various naive, simplistic, alarmist ideas that are somewhat common among some segments of the public, but where Chivers’ review makes it sound (at least to me) like the book will match those sorts of ideas.
All that said, I should again note that I thought the review did a lot right. In fact, I have no quibbles at all with anything from that last quote onwards.
This was an excellent meta-review! Thanks for sharing it.
I agree that these little slips of language are important; they can easily compound into very stubborn memes. (I don’t know whether the first person to propose a paperclip AI regrets it, but picking a different example seems like it could have had a meaningful impact on the field’s progress.)
Agreed.
These seem to often be examples of hedge drift, and their potential consequences seem like examples of memetic downside risks.
What are the implications of the offence-defence balance for trajectories of violence?
Questions: Is a change in the offence-defence balance part of why interstate (and intrastate?) conflict appears to have become less common? Does this have implications for the likelihood and trajectories of conflict in future (and perhaps by extension x-risks)?
Epistemic status: This post is unpolished, un-researched, and quickly written. I haven’t looked into whether existing work has already explored questions like these; if you know of any such work, please comment to point me to it.
Background/elaboration: Pinker argues in The Better Angels of Our Nature that many types of violence have declined considerably over history. I’m pretty sure he notes that these trends are neither obviously ephemeral nor inevitable. But the book, and other research pointing in similar directions, seems to me (and I believe others?) to at least weakly support the ideas that:
if we avoid an existential catastrophe, things will generally continue to get better
apart from the potential destabilising effects of technology, conflict seems to be trending downwards, somewhat reducing the risks of e.g. great power war, and by extension e.g. malicious use of AI (though of course a partial reduction in risks wouldn’t necessarily mean we should ignore the risks)
But How Does the Offense-Defense Balance Scale? (by Garfinkel and Dafoe, of the Center for the Governance of AI; summary here) says:
And:
The paper tries to use these sorts of ideas to explore how emerging technologies will affect trajectories, likelihood, etc. of conflict. E.g., the very first sentence is: “The offense-defense balance is a central concept for understanding the international security implications of new technologies.”
But it occurs to me that one could also do historical analysis of just how much these effects have played a role in the sort of trends Pinker notes. From memory, I don’t think Pinker discusses this possible factor in those trends. If this factor played a major role, then perhaps those trends are substantially dependent on something “we” haven’t been thinking about as much—perhaps we’ve wondered about whether the factors Pinker discusses will continue, whereas they’re less necessary and less sufficient than we thought for the overall trend (decline in violence/interstate conflict) that we really care about.
And at a guess, that might mean that that trend is more fragile or “conditional” than we might’ve thought. It might mean that we really really can’t rely on that “background trend” continuing, or at least somewhat offsetting the potentially destabilising effects of new tech—perhaps a lot of the trend, or the last century or two of it, was largely about how tech changed things, so if the way tech changes things changes, the trend could very easily reverse entirely.
I’m not at all sure about any of that, but it seems it would be important and interesting to explore. Hopefully someone already has, in which case I’d appreciate someone pointing me to that exploration.
(Also note that what the implications of a given offence-defence balance even are is apparently somewhat complicated/debatable matter. Eg., Garfinkel and Dafoe write: “While some hold that shifts toward offense-dominance obviously favor conflict and arms racing, this position has been challenged on a number of grounds. It has even been suggested that shifts toward offense-dominance can increase stability in a number of cases.”)
Should reducing partisanship be a higher priority cause area (for me)?
I think political polarization in the US produces a whole heap of really bad societal/policy outcomes and makes otherwise good policy outcomes ~impossible. It has always seemed relatively important to me, because when things go wrong in the US, they often have global consequences. I haven’t put that many of my actual resources here though because it’s a draining cause to work on and didn’t feel that tractable. I also suspected myself of motivated reasoning: I get deep joy from inter-group cooperation and am very distressed by inter-group conflict.
Then I read things like the thread below and feel like not paying more attention to this is foolish, like I’ve gone too far in the other direction and underweighted the importance of this barrier to global coordination. I imagine others have written about similar questions and I would be interested in more thoughts.
https://twitter.com/CT_Bergstrom/status/1243265011788611584
Something else in the vein of “things EAs and rationalists should be paying attention to in regards to Corona.”
There’s a common failure mode in large human systems where one outlier causes us to create a rule that is a worse equilibrium. In the PersonalMBA, Josh Kaufman talks about someone taking advantage of a “buy any book you want” rule that a company has—so you make it so that you can no longer get any free books.
This same pattern has happened before in the US, after 9-11 - We created a whole bunch of security theater, that caused more suffering for everyone, and gave government way more power and way less oversight than is safe, because we over-reacted to prevent one bad event, not considering the counterfactual invisible things we would be losing.
This will happen again with Corona, things will be put in place that are maybe good at preventing pandemics (or worse, making people think they’re safe from pandemics), but create a million trivial conveniences every day that add up to more strife than they’re worth.
These types of rules are very hard to repeal after the fact because of absence blindness—someone needs to do the work of calculating the cost/benefit ratio BEFORE they get implemented, then build a convincing enough narrative to what seems obvious/common sense measures given the climate/devastation.
I’ve been thinking more lately about how I should be thinking about causal effects for cost-effectiveness estimates, in order to clarify my own skepticism of more speculative causes, especially longtermist ones, and better understand how skeptical I ought to be. Maybe I’m far too skeptical. Maybe I just haven’t come across a full model for causal effects that’s convincing since I haven’t been specifically looking. I’ve been referred to this in the past, and plan to get through it, since it might provide some missing pieces for the value of research.
Suppose I have two random variables, X and Y, and I want to know the causal effect of manipulating X on Y, if any.
1. If I’m confident there’s no causal relationship between the two, say due to spatial separation, I assume there is no causal effect, and Y conditional on the manipulation of X to take value A (possibly random), Y|do(X=A), is identical to Y, i.e. Y|do(X=A)=Y. (The do notation is Pearl’s do-calculus notation.)
2. If X could affect Y, but I know nothing else,
a. I might assume, based on symmetry (and chaos?) for Y, that Y|do(X=A) and Y are identical in distribution, but not necessarily literally equal as random variables. They might be slightly “shuffled” or permuted versions of each other (see symmetric decreasing rearrangements for specific examples of such a permutation). The difference in expected values is still 0. This is how I think about the effects of my every day decisions, like going to the store, breathing at particular times, etc. on future populations. I might assume the same for variables that depend on Y.
b. Or, I might think that manipulating X just injects noise into Y, possibly while preserving some of its statistics, e.g. the mean or median. A simple case is just adding random symmetric noise with mean and median 0 to Y. However, whether or not a statistic is preserved with the extra noise might be sensitive to the scale on which Y is measured. For example, if Y is real-valued, and f:R→R is strictly increasing, then for the median, med(f(Y))=f(med(Y)), but the same is not necessarily true for the expected value of Y, or for other variables that depend on Y.
c. Or, I might think that manipulating X makes Y closer to a “default” distribution over the possible values of Y, often but not always uninformed or uniform. This can shift the mean, median, etc., of Y. For example, Y could be the face of the coin I see on my desk, and X could be whether I flip the coin or not, with X being not by default. So, if I do flip the coin and hence manipulate X, this randomizes the value of Y, making my probability distribution for its value uniformly random instead of a known, deterministic value. You might think that some systems are the result of optimization and therefore fragile, so random interventions might return them to prior “defaults”, e.g. naive systemic change or changes to ecosystems. This could be (like) regression to the mean.
I’m not sure how to balance these three possibilities generally. If I do think the effects are symmetric, I might go with a or b or some combination of them. In particular asymmetric cases, I might also combine c.
3. Suppose I have a plausible argument for how X could affect Y in a particular way, but no observations that can be used as suitable proxies, even very indirect, for counterfactuals with which to estimate the size of the effect. I lean towards dealing with this case as in 2, rather than just making assumptions about effect sizes without observations.
For example, someone might propose a causal path through which X affects Y with a missing estimate of effect size at at least one step along the path, but an argument to that this should increase the value of Y. It is not enough to consider only one such path, since there may be many paths from X to Y, e.g. different considerations for how X could affect Y, and these would need to be combined. Some could have opposite effects. By 2, those other paths, when combined with the proposed causal path, reduce the effects of X on Y through the proposed path. The longer the proposed path, the more unknown alternate paths.
I think this is where I am now with speculative longtermist causes. Part of this may be my ignorance of the proposed causal paths and estimates of effect sizes, since I haven’t looked too deeply at the justifications for these causes, but the dampening from unknown paths also applies when the effect sizes along a path are known, which is the next case.
4. Suppose I have a causal path through some other variable Z, X→Z→Y, so that X causes Z and Z causes Y, and I model both the effects of X→Z and Z→Y, based on observations. Should I just combine the two for the effect of X on Y? In general, not in the straightforward way. As in 3, there could be another causal path, X→Z′→Y (and it could be longer, instead of with just a single intermediate variable).
As in case 3, you can think of X→Z′→Y as dampening the effect of X→Z→Y, and with long proposed causal paths, we might expect the net effect to be small, consistently with the intuition that the predictable impacts on the far future decrease over time due to ignorance/noise and chaos, even though the actual impacts may compound due to chaos.
Maybe I’ll write this up as a full post after I’ve thought more about it. I imagine there’s been writing related to this, including in the EA and rationality communities.
I think EA hasn’t sufficiently explored the use of different types of empirical studies from which we can rigorously estimate causal effects, other than randomized controlled trials (or other experiments). This leaves us either relying heavily on subjective estimates of the magnitudes of causal effects based on weak evidence, anecdotes, expert opinion or basically guesses, or being skeptical of interventions whose cost-effectiveness estimates don’t come from RCTs. I’d say I’m pretty skeptical, but not so skeptical that I think we need RCTs to conclude anything about the magnitudes of causal effects. There are methods to do causal inference from observational data.
I think this has lead us to:
1. Underexploring the global health and development space. See John Halstead’s and Hauke Hillebrandt’s “Growth and the case against randomista development”. I think GiveWell is starting to look beyond RCTs. There’s probably already a lot of research out there they can look to.
2. Relying too much on guesses and poor studies in the effective animal advocacy space (especially in the past), for example overestimating the value of leafletting. I think things have improved a lot since then, and I thought the evidence presented in the work of Rethink Priorities, Charity Entrepreneurship and Founders Pledge on corporate campaigns was good enough to meet the bar for me to donate to support corporate campaigns specifically. Humane League Labs and some academics have done and are doing research to estimate causal effects from observational data that can inform EAA.
Fehige defends the asymmetry between preference satisfaction and frustration on rationality grounds. This is my take:
Let’s consider a given preference from the point of view of a given outcome after choosing it, in which the preference either exists or does not, by cases:
1. The preference exists:
a. If there’s an outcome in which the preference exists and is more satisfied, and all else is equal, it would have been irrational to have chosen this one (over it, and at all).
b. If there’s an outcome in which the preference exists and is less satisfied, and all else is equal, it would have been irrational to have chosen the other outcome (over this one, and at all).
c. If there’s an outcome in which the preference does not exist, and all else is equal, the preference itself does not tell us if either would have been irrational to have chosen.
2. The preference doesn’t exist:
a. If there’s an outcome in which the preference exists, regardless of its degree of satisfaction, and all else equal, the preference itself does not tell us if either would have been irrational to have chosen.
So, all else equal besides the existence or degree of satisfaction of the given preference, it’s always rational to choose an outcome in which the preference does not exist, but it’s irrational to choose an outcome in which the preference exists but is less satisfied than in another outcome.
(I made a similar argument in the thread starting here.)
I also think that antifrustrationism in some sense overrides interests less than symmetric views. Consider the following two options for interests within one individual:
A. Interest 1 exists and is fully satisfied
B. Interest 1 exists and is not fully satisfied, and interest 2 exists and is (fully) satisfied.
A symmetric view would sometimes choose B, so that the creation of interests can take priority over interests that would exist regardless. In particular, the proposed benefit comes from satisfying an interest that would not have existed in the alternative, so it seems like we’re overriding the interests the individual would have in A with a new interest, interest 2. For example, we make someone want something and satisfy that want, at the expense of their other interests.
On the other hand, consider:
A. Interest 1 exists and is partially unsatisfied
B. Interest 1 exists and is fully satisfied, and interest 2 exists and is partially unsatisfied.
In this case, antifrustrationism would sometimes choose A, so that the removal or avoidance of an otherwise unsatisfied interest can take priority over (further) satisfying an interest that would exist anyway. But in this case, if we choose A because of concerns for interest 2, at least interest 2 would exist in the alternative A, so the benefit comes from the avoidance of an interest that would have otherwise existed. In A, compared to B, I wouldn’t say we’re overriding interests, we’re dealing with an interest, interest 2, that would have existed otherwise.
Some related writings, although not making the same point I am here:
Brian Tomasik’s “Does Negative Utilitarianism Override Individual Preferences?”
Simon Knutsson’s “What is the difference between weak negative and non-negative ethical views?” (On Center for Long-Term Risk’s website)
Toby Ord’s “Why I’m Not a Negative Utilitarian”
Then, if you extend these comparisons to satisfy the independence of irrelevant alternatives by stating that in comparisons of multiple choices in an option set, all permissible options are strictly better than all impermissible options regardless of option set, extending these rankings beyond the option set, the result is antifrustrationism. To show this, you can use the set of the following three options, which are identical except in the ways specified:
and since B is impermissible because of the presence of A, this means C>B, and so it’s always better for a preference to not exist than for it to exist and not be fully satisfied, all else equal.
I also think this argument isn’t specific to preferences, but could be extended to any interests, values or normative standards that are necessarily held by individuals (or other objects), including basically everything people value (see here for a non-exhaustive list). See Johann Frick’s paper and thesis which defend the procreation asymmetry, and my other post here.
Collection of EA analyses of how social social movements rise, fall, can be influential, etc.
Movement collapse scenarios—Rebecca Baron
Why do social movements fail: Two concrete examples. - NunoSempere
What the EA community can learn from the rise of the neoliberals—Kerry Vaughan
Some of the Sentience Institute’s research, such as its “social movement case studies” and the post How tractable is changing the course of history?
These aren’t quite “EA analyses”, but Slate Star Codex has several relevant book reviews and other posts, such as https://slatestarcodex.com/2019/03/18/book-review-inventing-the-future/
It appears Animal Charity Evaluators did relevant research, but I haven’t read it, they described it as having been “of variable quality”, and they’ve discontinued it.
Notes
I intend to add to this list over time. If you know of other relevant work, please mention it in a comment.
Also, I’m aware that there are a lot of non-EA analyses of these topics. The reasons I’m collecting only EA analyses here are that:
their precise focuses or methodologies may be more relevant to other EAs than would be the case with non-EA analyses
links to non-EA work can be found in most of the things I list here
I’d guess that many collections of non-EA analyses of these topics already exist (e.g., in reference lists)
[Epistemic status: speculation based on priors about international organizations. I know next to nothing about the WHO specifically.]
[On the WHO declaring COVID-19 a pandemic only (?) on March 12th. Prompted by this Facebook discussion on epistemic modesty on COVID-19.]
- [ETA: this point is likely wrong, cf. Khorton’s comment below. However, I believe the conclusion that the timing of WHO declarations by itself doesn’t provide a significant argument against epistemic modesty still stands, as I explain in a follow-up comment below.] The WHO declaring a pandemic has a bunch of major legal and institutional consequences. E.g. my guess is that among other things it affects the amounts of resources the WHO and other actors can utilize, the kind of work the WHO and others are allowed to do, and the kind of recommendations the WHO can make.
- The optimal time for the WHO to declare a pandemic is primarily determined by these legal and institutional consequences. Whether COVID-19 is or will in fact be a pandemic in the everyday or epidemiological sense is an important input into the decision, but not a decisive one.
- Without familiarity with the WHO and the legal and institutional system it is a part of, it is very difficult to accurately assess the consequences of the WHO declaring a pandemic. Therefore, it is very hard to evaluate the timing of the WHO’s declaration without such familiarity. And being even maximally well-informed about COVID-19 itself isn’t even remotely sufficient for an accurate evaluation.
- The bottom line is that the WHO officially declaring that COVID-19 is a pandemic is a totally different thing from any individual persuasively arguing that COVID-19 is or will be a pandemic. In a language that would accurately reflect differences in meaning, me saying that COVID-19 is a pandemic and the WHO declaring COVID-19 is a pandemic would be done using different words. It is simply not the primary purpose of this WHO speech act to be an early, accurate, reliable, or whatever indicator of whether “COVID-19 is a pandemic”, to predict its impact, or any other similar thing. It isn’t primarily epistemic in any sense.
- If just based on information about COVID-19 itself someone confidently thinks that the WHO ought to have declared a pandemic earlier, they are making a mistake akin to the mistake reflected by answering “yes” to the question “could you pass me the salt?” without doing anything.
So did the WHO make a mistake by not declaring COVID-19 to be a pandemic earlier, and if so how consequential was it? Well, I think the timing was probably suboptimal just because my prior is that most complex institutions aren’t optimized for getting the timing of such things exactly right. But I have no idea how consequential a potential mistake was. In fact, I’m about 50-50 on whether the optimal time would have been slightly earlier or slightly later. (Though substantially earlier seems significantly more likely optimal than substantially later.)
“The WHO declaring a pandemic has a bunch of major legal and institutional consequences. E.g. my guess is that among other things it affects the amounts of resources the WHO and other actors can utilize, the kind of work the WHO and others are allowed to do, and the kind of recommendations the WHO can make.”
Are you sure about this? I’ve read that there aren’t major implications to it being officially declared a pandemic.
This article suggests there aren’t major changes based on ‘pandemic’ status https://www.bbc.co.uk/news/world-51839944
[Epistemic status: info from the WHO website and Wikipedia, but I overall invested only ~10 min, so might be missing something.]
It seems my remarks do apply for “public health emergency of international concern (PHEIC)” instead of “pandemic”. For example, from Wikipedia:
The WHO declared a PHEIC due to COVID-19 on January 30th.
The OP was prompted by a claim that the timing of the WHO using the term “pandemic” provides an argument against epistemic modesty. (Though I appreciate this was less clear in the OP than it could have been, and maybe it was a bad idea to copy my Facebook comment here anyway.) From the Facebook comment I was responding to:
Since the WHO declaring a PHEIC seems much more consequential than them using the term “pandemic”, the timing of the PHEIC declaration seems more relevant for assessing the merits of the WHO response, and thus for any argument regarding epistemic modesty.
Since the PHEIC declaration happened significantly earlier, any argument based on the premise that it happened too late is significantly weaker. And whatever the apparent initial force of this weaker argument, my undermining response from the OP still applies.
So overall, while the OP’s premise appealing to major legal/institutional consequences of the WHO using the term “pandemic” seems false, I’m now even more convinced of the key claim I wanted to argue for: that the WHO response does not provide an argument against epistemic modesty in general, nor for the epistemic superiority of “informed amateurs” over experts on COVID-19.
Also, predicting that something will be pretty bad or will be a pandemic is not the same as saying it is now a pandemic. When did it become a pandemic according to the WHO’s definition?
Expanding a quote I found on the wiki page in the transcript here from 2009:
But see also WHO says it no longer uses ‘pandemic’ category, but virus still emergency from February 24, 2020.
About declaring it a “pandemic,” I’ve seen the WHO reason as follows (me paraphrasing):
«Once we call it a pandemic, some countries might throw up their hands and say “we’re screwed,” so we should better wait before calling it that, and instead emphasize that countries need to try harder at containment for as long as there’s still a small chance that it might work.»
Yeah, I think that’s a good point.
I’m not sure I can have updates in favor or against modest epistemology because it seems to me that my true rejection is mostly “my brain can’t do that.” But if I could have further updates against modest epistemology, the main Covid-19-related example for me would be how long it took some countries to realize that flattening the curve instead of squishing it is going to lead to a lot more deaths and tragedy than people seem to have initially thought. I realize that it’s hard to distinguish between what’s actual government opinion versus what’s bad journalism, but I’m pretty confident there was a time when informed amateurs could see that experts were operating under some probably false or at least dubious assumptions. (I’m happy to elaborate if anyone’s interested.)
Thank you for pointing this out! It sounds like my guess was probably just wrong.
My guess was based on a crude prior on international organizations, not anything I know about the WHO specifically. I clarified the epistemic status in the OP.