Thanks for the comment! Wonder if you, or @Derek Shiller knows of any research on the number or proportion of extinctions caused by humans? Thinking it would be a useful number to use as a prior!
Axby
Karma: 84
Thanks for your comment, which helps me to zoom in on claims 4 and 5 in my own thinking.
I was thinking of another point on intelligence fallibility, specifically whether intelligence really allows the AGI to fully shape the future to its will. Was thinking along the lines of Laplace’s Demon which asks the question: if there is a demon which knows the position of every atom in the universe, and the direction which it travels in, will it be able to predict (and hence shape) the future? I think it is not clear that it will. In fact, Heisenberg’s uncertainty principle suggests that it will not (at least at the quantum level). Similarly, it is not clear that the AGI would be able to do so even if it has complete knowledge of everything.
Happy to comment on your post before/when you publish it!
[Question] Some Quick Notes on Singapore’s Centre for Strategic Futures
My personal view is that it is worth spending some more time to think about cause prioritisation, while at the same time building up career capital (see: https://80000hours.org/articles/career-capital/) that is robustly useful across a range of likely options. One way to narrow down your option set is to think about the type of work which you prefer: policy vs. research vs. operational vs. technical work. If you prefer policy/strategy/managerial kinds of work, a graduate degree in AI or medicine may not be required.
You might be able to get some work experience in one such type of work with your history degree, while thinking about cause prioritisation on the side. If possible, you may also want to try to find a job that is medicine/public health or AI-related, so you can also build up some knowledge of these fields while assessing your own personal interest. Grad school is one of the most useful ways to make a “career switch”, so I think it would be wise not to rush the decision.
(unless you think AGI would be developed in the next 2 years or something and you should therefore aim for impact straightaway)
A Rising Tide Threatens Barriers to Bioweapons
Hi! Was wondering if you have any advice/thoughts on the value of a Security Studies masters (e.g. georgetown’s) for a foreign national who is interested in working on US policy (such as through a think tank)?
An initial thought is that since “security” positions are usually subject to security clearance requirements, a foreign national would find it difficult to approach X-risk-related US policy from a security policy angle. Instead, we would need to approach US policy from a tech policy or health policy angle. A security studies degree, as opposed to an IR/MPP, might make applying for such positions more challenging.
Skimmed through so you might have mentioned this point already.
Seemed to me like maybe rather than to understand religious ideologies better in themselves, your intention is ultimately to answer questions that help with cause prioritisation, like:
what beings matter
does values matter (e.g. suffering, happiness, QALYs)
These are questions that some EAs are already working on.
Religious ideologies may help to ask other relevant questions and provide some plausible hypothesis, but the valid answers would still need to be based on empirical evidence.
I think that’s a much more tractable endeavour than arguing about which religion/belief is correct, from a theological angle, as an aim in itself.
Hi, would be good if you are able to elaborate on the point of Malaria being an environmental (rather than a vaccine) issue, as your title suggests, perhaps by summarising some relevant research. I have absolutely zero knowledge on this but not sure if cleanliness is the issue, or something more specific like stagnant water etc.? Some cited references/sources would also be useful further reading. :)
Personal take on the top biosecurity interventions, after 7 weeks of participation in Bluedot Impact’s Biosecurity Fundamentals reading group.
I have shortlisted these four interventions based on what I think are most impactful for preventing extreme pandemics, and tractable (based on my vague sense of public trust and political will in most countries).
Epidemic intelligence—specifically to estimate how likely an outbreak is going to become a pandemic based on pathogen properties → crucial for informing non-pharmaceutical interventions
Targeted and clinical and metagenomic sequencing could help to collect data points
Digital detection tools as well
Enabling non-pharmaceutical interventions such as contact tracing, quarantines and lockdowns (took many months to be implemented in most countries, if at all)
Vaccine delivery—took almost 2 years for Global South (e.g. African countries, India and Indonesia) to be able to vaccinate 60% of their population
Governing benchtop synthesis machines above a certain number of base pairs, since allowing malicious actors to have access to such machines would pose an irreversible risk.
Reflecting on our COVID-19 experience, there were five key problems that would undermine humanity’s ability to survive extreme pandemics.
Firstly, it took us about two months from the detection of a novel viral pneumonia to the lockdown of Wuhan. Two months is too long a duration to try to contain the virus within the region. By then, cases of COVID-19 were already reported in Thailand and the United States. This seems quite tractable because central governments have an incentive to contain the spread of viruses within one local region before it spreads to other regions.
Relatedly, the second bottleneck is the time it took for each country to implement contact tracine regimes and lockdowns. The United States did not impose any lockdowns, whereas other countries seem to have taken a couple of months after the first local case before lockdowns were imposed.
A key reason for why governments took such a long time to impose adequate movement control measures is that the epidemiological properties of the virus too unclear. As such, there wasn’t enough political will to impose such costly disease control measures. Better epidemic intelligence—specifically, to provide estimates on how fast the disease can spread, and its lethality—is crucial to enable governments to implement non-pharmaceutical interventions as quickly as possible.
The fourth bottleneck was the time it took for clinical trials of the vaccines. The design of the vaccine took just slightly over two months from when the COVID-19 genome sequence was published (on 16 Mar 2020). However, it took another nine months for Moderna’s coronavirus vaccine to be approved for emergency use on December 18, 2020. Much of the time was spent on clinical trials. By then, the peak of the outbreak had passed in most countries, which means that vaccines were not as effective.
The fourth bottleneck is the time it takes for vaccines to be delivered globally. After Moderna’s vaccine was developed and approved on December 18, it is estimated that another two years were needed for the vaccine to be delivered in very populous parts of the world, including Indonesia, India, and the entire African continent. No country would be able to impose lockdowns and disease control measures for such a long time. Part of the reason for why vaccine delivery took such a long time was vaccine nationalism.
From this analysis of the bottlenecks that we face during COVID-19, we can identify the key interventions that would prevent extreme pandemics.
Firstly, and quite uncontroversially, we need better epidemic intelligence. We need to go beyond identifying the genome of novel pathogens to estimating their epidemiological properties such as R0 as quickly as possible. This will help to inform and legitimize non-pharmaceutical interventions such as contract tracing, quarantines and lockdowns. Beyond DNA sequencing methods (including metagenomic sequencing), digital methods could also help to inform epidemiological estimates.
Secondly, governments need to impose non-pharmaceutical interventions such as quarantines and lockdowns more quickly in order to contain novel epidemics within regions. In support of such non-pharmaceutical interventions, digital detection could also enable more effective contact tracing and more targeted interventions.
Thirdly, vaccine equity is crucial for humanity to recover from extreme pandemics; no one is safe until everybody is safe. Speeding up clinical trials is also important, but I would think that pharmaceutical companies already have a strong incentive to speed this up as much as possible. As such, more advocacy and government funding is not as crucial.
Other preventive measures like DNA synthesis screening and governing do use research of concern are also important. However, I think that are less likely to receive public support as there haven’t been any cases of successful bioterror attacks using pandemic-capable viruses. Nevertheless, governing access to benchtop synthesis machines (and hardwiring screening measures into their design) is still important to deny potentially malicious actors the capability to develop pandemic-capable viruses as bioweapons. The proliferation of such capabilities—like the proliferation of fissile material—would be quite irreversible.
(Not quite sure how to evaluate clean-air interventions like far-UVC or air filtration. My view would probably depend most on whether they are effective enough to prevent even the worst pandemics, second would be whether they would be cost effective enough to be deployed widely)
Hi Karthik,
Thanks for writing this! Very useful summary to the economics literature on development for someone unfamiliar with it.
I note in your caveat that “a large body of work emphasizes the importance of good institutions for development; I don’t believe that topic will yield any promising interventions”. You seem to have given alot of thought on this so I’m wondering if there are any clear signs you see which suggest that such institutional development work are indeed not promising.
There seem to be a number of organisations working on this front (e.g. Artha Global as suggested in this post, anti-corruption institutions etc.), though the outcomes may be hard to quantify. The EA community also often faces criticisms that it neglects interventions which aim at systemic change. Without having looked at the evidence, such interventions seem possibly promising to me.
I’m currently trying to develop an estimation of the effectiveness of pursuing a career in mitigating Global Catastrophic Biological Risks (GCBRs). As part of the EA Global in-depth reading group, I read “Existential Risk and Cost-Effective Biosecurity” by Piers Millett and Andrew Snyder-Beattie (2017). The authors’ estimates of the probability of GCBRs over the next century seemed very low (from 1.6 x 10^-6 to 0.02 over the next century, depending on which of the three methodologies used by the authors gives a better estimate)[1].
I could not seem to find any other sources that try to rigorously estimate the risks of GCBRs. Would forum users would be able to point me to some, please? [2] Thanks all in advance!
[1] Even with the highest estimate of 0.02 of GCB events per century, longtermist assumptions (of 10^16 potential lives lost, as indicated by the author) are needed for GCBR-mitigation to be more cost-effective than Givewell’s top charities (taken as $4500/life saved). I would prefer not to make longtermist assumptions, in line with Neel Nanda’s (2021) call to “Simplify EA
Pitches to ‘Holy Shit, X-Risk’”.
[2] With this input, I hope to write a more extensive post clarifying the priority areas in mitigating GCBRs (possibly pointing to risks of GCBRs from emerging biotech that enable bioterrorism as a priority area; in contrast with a focus on state-led biowarfare). If we are focusing on emerging biotech rather than existing biotech, estimates of whether these technologies would even materialise would be an important consideration as well.