A website to deter rogue individuals from developing ideas to ‘destroy the world’ by showing that it’s just a very bad idea given their values (e.g. A failed attempt will only make the world worse). The website would specifically target people visiting pro-extinction websites or searching the internet for things like ‘how to destroy the world’.
This project would require a very delicate approach and ideally would attract very little attention from people uninterested in destroying the world. We don’t want to give anyone bad ideas. The text on it would require careful phrasing. The project probably would need buy-in from knowledgeable stakeholders (FHI, CSER, FLI, EAF, perhaps even the CIA?), a secure website, and very careful targeting.
Cause prioritization tool
Cause prioritization is complex, and going through it takes time. The relevant considerations are often unstructured and require a lot of work to discover. This leads so sub-optimal cause prioritization. This can be done much better.
An interactive cause prioritization tool could help people go through relevant considerations. I am imagining something like the GPP’s flowchart, but where are asked only one question at a time.
Features could include:
Ability to save progress and return later
Insight into how other users answered the question
Ability to link to relevant arguments
It should ideally be community-driven (the community can suggest considerations, and the community can vote on the importance of a particular consideration).
Guided meditation recordings for EAs
I think meditation is a very interesting medium for learning that we aren’t using much yet. Guided meditation recordings are highly scalable and can be used to deal with specific challenges that (effective) altruists encounter, like these by Holly Morgan, or to teach elements from the Replacing Guilt Series.
I would be very interested in seeing someone do a few of these to test the idea. Plausibly, this only becomes effective once the EA has grown substantially in size.
Two other topics suitably for a guided meditation recording:
EA events (retreats, conferences) can be very exciting and very taxing, and the meditation could help to calm people down, refresh, process what’s going on, set intention or reflect on one’s intention, prioritize, etc.
Demotivated moments: sometimes EA feels like running against a wall—the effort just seems a complete waste, and it’d be nice to have some meditation to turn to. It could help people to deal healthily with setbacks by trying to learn from them, and then accepting whatever is.
EA e-learning community:
EA’s are focused a lot on learning, but are doing this inefficiently: they study alone, without a syllabus, often without peers, and without getting feedback and evaluation on their learning outcomes.
There are many e-learning tools, much more than just MOOC videos, that can be used, although I’m not familiar with them. A learning community could be created within an existing e-learning platform.
People in EA are smart; a lot of value can be gained by facilitating people help each other through carefully designed learning programs. Q&A’s with EA researchers whose work is on the syllabus could be arranged to minimize time needed for teaching. The e-learning course/community could also create structure and accountability to get things done, as well as assist students in signalling their EA-relevant knowledge and skills. For example, they could show they passed a course on ‘creating good Fermi estimates’.
A multiple-day event in which people (mostly EA’s) work to solve a set of problems. These problems can be in any cause area, as long as a solution could plausibly be cost-effective.
Organizations could send in problem cases, or participants could identify their own. In fact, this thread might already give a nice list of possible problems/solutions to work on!
The event can be made more complex by adding in rounds, which filter the good solutions from mediocre and bad solutions. In further rounds, promising teams/individuals could be matched with a coach. The best few ideas get funding to take the next step in solving the problem (e.g. creating a minimum viable product).
Thanks for the answer. I look forward to seeing your write-up and how the models evolve!
One thing that seems missing in the current model is how smoke maps to famine (does the location of the smoke matter?), but perhaps assuming a linear relationship between amount of smoke and amount of casualties from famine is a good approximation.
Also on the topic of nuclear war: I found it very surprising to see that, according to the report, the expected deaths of a US-Russia exchange is lower (3 billion) than US-China or (5.5 billion) or Russia-China (7 billion). This implies that the existential risk from nuclear war is higher from a Russia-China exchange than from a US-Russia exchange. How should we translate these expected deaths to existential risk?
I found the Guesstimate models interesting, and figured out that this is largely because a nuclear bombing of China is expected to produce the most smoke. But there’s a factor in TSG1 (“total smoke generated by nuclear weapons detonated in China (Tg)”) that I don’t understand, but that factor drives much of the differences in total smoke generated. In the US-China model it’s 1.58, and the Russia-China model it’s 2.04, while a Russian bombing of the US has only a factor of 0.664. Could you explain what this factor is and where it’s derived from?
Also: the Russia-China model has some broken links due to an extra “]” in MNO2.
About Nuclear Nonproliferation you write:
Candidates should be good at handling nuclear proliferation and threatening states such as North Korea [my emphasis], Iran and Pakistan. These states do not have the nuclear capacity to pose any existential risk, so they get less attention here than the primary powers, but they complicate calculations of international deterrence and might indirectly increase the risk of a major nuclear war.
What do you mean by ‘being good at threatening’? It seems counterproductive to threaten North Korea, as this could cause international conflict, and backfire by triggering action from North Korea against South Korea and possibly Japan. As argued in this article, it seems that the only options are to persuade China to increase economic pressure, or to accept a nuclear North Korea. North Korea has always been consistent and, given their goal, rational: they aim for nuclear weapons for deterrence and know that using nukes will mean assured destruction.
Ah, I assumed the latter was a consequence of the former because they were in the same paragraph, my bad.
However, like Michael, I’m still a bit confused about the role neglectedness is playing in this analysis (and all other analyses). But don’t take that as criticism of your analysis. It often seems that neglectedness and tractability (and scale) are used as independent reasons to support a particular cause area or intervention, rather than that they are used as a coherent framework. It seems to me your argument would have been similarly strong if clean energy R&D was not neglected—if you could just show that additional spending would have big benefits.
My understanding of these terms is roughly as follows:
Scale: general size of the problem. Determines the upper bound of what can be achieved. Determining the scale of a problem is quite arbitrary, because how do you draw the boundaries of ‘the problem’ and when is it completely solved?
Tractability: determines the average or global cost-effectiveness if you don’t know where you are on the curve (i.e. if you don’t know how much of the problem has been solved so far). Higher tractability means that the curve is steeper on average.
Neglectedness: determines the location on the curve and gets you the marginal or local cost-effectiveness. Because we expect the value curve to have diminishing returns, a good heuristic is ‘more neglected --> higher marginal value.’
I think where EA’s go repeatedly wrong with the application of the model is that tractability and neglectedness get confused: tractability should refer to solving the complete problem. If it refers to marginal tractability, then it double counts the neglectedness consideration. The report seems to do this:
Public clean energy R&D is neglected: only $22 billion is spent per year globally. Many advanced economies such as the US could unilaterally increase this substantially i.e. even without international coordination—which makes this policy uniquely politically tractable.
Here, neglectedness is taken as a reason for tractability, while it should be a reason for marginal cost-effectiveness.
The SNT-model is also much more helpful for funding than for career choice, because neglectedness has linear implications for the value of extra funding, but more complex implications for an extra person doing work. Some skills are not very useful in the early stage of a cause area or problem, but become valuable only later. In general, I think personal fit is very important, and the SNT-model does not account for it.
I actually hope to write a longer post explaining all this in more detail, including some nice visual explanation I have of the SNT-model.
I think EA’s believe that this is definitely possible, most likely by the creation of an aligned superintelligence. That could reduce x-risk to infinitessimal levels, if there are no other intelligent actors that we could encounter. I think the general strategy could be summarized as ‘reduce extinction risk as much as possible until we can safely build and deploy an aligned superintelligence, then let the superintelligence (dis)solve all other problems’.
After the creation of an aligned superintelligence, society’s resources could focus on other problems. However, I think some people also think there are no other problems anymore once there is an aligned superintelligence: with superintelligence all the other problems like animal suffering are trivial to solve.
But most people—including myself—seem to not have given very much thought to what other problems might still exist in an era of superintelligence.
If you believe a strong version superintelligence is impossible this complicates the whole picture, but you’d at least have to include the consideration that in the future it is likely we have substantially higher (individual and/or collective) intelligence.
If humanity wipes itself out, those wild animals are going to continue suffering forever.
Not forever. Only until the planet becomes too hot to support complex life (<1 billion years from now). Giving that the universe can support life 1-100 trillion years, this is a relatively short amount of suffering compared to what could be.
And also only on our planet! Which is much less restricted than the suffering that can spread if humanity remains alive. (Although, as I write in my own answer, I don’t think humanity would spread wild animals beyond the solar system.)
Hey Jack, I think this is a great question and I dedicate a portion of my MA philosophy thesis to this. Here are some general points:
It is likely that the expected moral value of the future is dominated by futures in which there is optimization for moral (dis)value. Since we would expect it to be much more likely there will be optimization for value than for disvalue, the expected moral value of the future seems positive (unless you adhere to strict/lexical negative utilitarianism). This claim depends on the difference between possible worlds that are optimized for (dis)value vs. states that are subject to other pressures, like competition: this difference is not obviously large (see Price of Anarchy).
There seems substantial convergence between improving the quality of the long-term future and reducing extinction risk. Things that can bring humanity to extinction (superintelligence, virus, nuclear winter, extreme climate change) can also very bad for the long-term future of humanity if they do not lead to extinction. Therefore, reducing extinction risk also has very positive effects on the quality of the long-term future. Potential suffering risk from misaligned AI is one candidate. In addition, I think global catastrophes, if they don’t lead to extinction, create a negative trajectory change in expectation. Either because civilizational collapse puts us on a worse trajectory, but the most likely outcome is what I call general global disruption: civilization doesn’t quite collapse, but think are shaken up a lot. From my thesis:
Should we expect global disruption to be (in expectation) good or bad for the value of the future? This is speculative, but Beckstead lays out some reasons to expect that global disruption will put humanity most certainly on a worse trajectory: it may reverse social progress, limit the ability to adequately regulate the development of dangerous technologies, open an opportunity for authoritarian regimes to take hold, or increase inter-state conflict (Beckstead, 2015). We can also approach the issue abstractly: disruption can be seen as injecting more noise into a previously more stable global system, increasing the probability that the world settles into a different semi-stable configuration. If there are many more undesirable configurations of the world than desirable ones, increasing randomness is more likely to lead to an undesirable state of the world. I am convinced that, unless we are currently in a particularly bad state of the world, global disruption would have a very negative effect (in expectation) on the value of the long-term future.
I find it unlikely that we would export wild-animal suffering beyond our solar system. It takes a lot of time to move to different solar systems, and I don’t think future civilizations will require a lot of wilderness: it’s a very inefficient use of resources. So I believe the amount of suffering is relatively small from that source. However, I think some competitive dynamics between digital beings could create astronomical amounts of suffering, and this could come about if we focus only on reducing extinction risk.
Whether you want to focus on the quality of the future also depends on your moral views. Some people weigh preventing future suffering much more heavily than enabling the creation of future happiness. For them, part of the value of reducing extinction risk is taken away, and they will have stronger reasons to focus on the quality of the future.
I found the post by Brauner & Grosse-Holz and the post by Beckstead most helpful. I know that Haydn Belfield (CSER) is currently working on a longer article about the long-term significance of reducing Global Catastrophic Risks.
In conclusion, I think reducing extinction risk is a very positive in terms of expected value, even if one expects the future to be negative! However, depending on different parameters, there might be better options than focusing on extinction risk. Candidates involve particular parts of moral circle expansion and suffering risks from AI.
I can send you the current draft of my thesis in case you’re interested, and will post it online once I have finished it.
Research teams seem more likely to realize their full disruptive potential if the researchers do not have to do anything but research and have easy access to all the resources they need.
Note that Richard Hamming disagreed with this. He makes the point that restricted resources force creativity, and the best breakthroughs come through this creativity. I think he has got a point, but I’m not sure what they all-things-considered conclusion should be. I think the ‘constraints breed creativity’ applies more to the tools people work with, and other constraints like teaching, administrative tasks, and grant applications mostly waste time.
Here’s an excerpt form his famous talk on research (for individual researchers, not teams):
What most people think are the best working conditions, are not. Very clearly they are not because people are often most productive when working conditions are bad. One of the better times of the Cambridge Physical Laboratories was when they had practically shacks—they did some of the best physics ever.
I give you a story from my own private life. Early on it became evident to me that Bell Laboratories was not going to give me the conventional acre of programming people to program computing machines in absolute binary. It was clear they weren’t going to. But that was the way everybody did it. I could go to the West Coast and get a job with the airplane companies without any trouble, but the exciting people were at Bell Labs and the fellows out there in the airplane companies were not. I thought for a long while about, ``Did I want to go or not?″ and I wondered how I could get the best of two possible worlds. I finally said to myself, ``Hamming, you think the machines can do practically everything. Why can’t you make them write programs?″ What appeared at first to me as a defect forced me into automatic programming very early. What appears to be a fault, often, by a change of viewpoint, turns out to be one of the greatest assets you can have. But you are not likely to think that when you first look the thing and say, ``Gee, I’m never going to get enough programmers, so how can I ever do any great programming?″
And there are many other stories of the same kind; Grace Hopper has similar ones. I think that if you look carefully you will see that often the great scientists, by turning the problem around a bit, changed a defect to an asset. For example, many scientists when they found they couldn’t do a problem finally began to study why not. They then turned it around the other way and said, ``But of course, this is what it is″ and got an important result. So ideal working conditions are very strange. The ones you want aren’t always the best ones for you.
In the same talk he also criticizes the Institute for Advanced Study, claiming that they only produced good output because they recruited the very best, but that counterfactually these researchers would have produced better output.
I’d like to see every major research article in EA find its way to the Forum, ideally with summaries and authorial responses. This report [on farmed fish welfare] is an excellent example of how to bring interesting work to a broad audience.
That would certainly be great!
Relatedly, there are the concepts of ‘uncertainty’ and ‘insecurity’. I think there’s a risk that uncertainty as perceived, and perhaps even experienced, as insecurity. Interestingly, both concepts are translated into one and the same word in Dutch! (“onzekerheid”)
However, I think stating epistemic uncertainty in a very precise and confident way (e.g. “I believe X, and I am 60% certain my hypothesis is correct”) can show meta-confidence and strong epistemics. I would rather learn to be convince while still communicating uncertainties, than learning to hide my epistemic uncertainty.
Also, experts in any domain face this challenge, and useful lessons could be drawn from literature on it, such as this paper (I only read the abstract, it seems useful).
How big do you expect that fraction to be? (Or: what percentage of those numbers do you expect to be ‘real listeners’?)
Perhaps not the best place to ask this, but:
When I was going through my account options to edit my bio, I saw there’s an option to automatically hide comments generated by GPT-2. Why are we not automatically marking these comments as spam or at least as bot-generated?
provide especially strong evidence w.r.t. the discovery counterfactuals.
I was a bit confused here. Do you mean the rediscovery provides evidence that the idea was ahead of its time, especially when the rediscovery was much later, because we have an actual counterfactual?
That is nice! Here are some other ideas for guided meditation:
For during EA events: they can be very exciting and very taxing, and the meditation could help to calm people down, refresh, process what’s going on, etc.
For demotivated moments: sometimes EA feels like running against a wall—the effort just seems a complete waste, and it’d be nice to have some meditation to turn to. It could help people deal healthily with setbacks by trying to learn from them, and then accepting whatever is.
There’s a bunch of other types that might be interesting to explore via meditation, such as obsession, suffering (of others and oneself), egoism, (dis)connection to EA’s and non-EA’s, value drift, multiple things from the replacing guilt series and other EA self-care stuff, etc.
In general, I think meditation is a very interesting medium for learning that we aren’t using much yet. I would be very interested in seeing someone do a few of these to test the idea, and I’d be interested in getting someone funding to increase the volume if the pilots are successful.