I’m am on a gap year from studying for a Master of Social Entrepreneurship degree at the University of Southern California.
I have thought along EA lines for as long as I can remember, and I recently wrote the first draft of a book “Ways to Save The World” about my top innovative ideas for broad approaches to reduce existential risk.
I am now doing more research on how these approaches interact with AI X-risk.
Yes! This is helpful. I think one of the main places where I get caught up is taking expected value calculations very seriously even though they are wildly speculative; it seems like there is a very small chance that I might make a huge difference on an issue that ends up being absurdly important, and so it is hard to use my intuition on this kind of thing, whereas my intuitions very clearly help me with things that are close by and hence more easier to see I am doing some good but more difficult to make wild speculations that I might be having a hugely positive impact. So I guess part of the issue is to what degree I depend on these wildly speculative EV calculations, I feel like I really want to maximize impact, yet it is always a tenuous balancing act with so much uncertainty.
Anyone else ever feel a strong discordance between emotional response and cognitive worldview when it comes to EA issues?
Like emotionally I’m like “save the animals! All animals deserve love and protection and we should make sure they can all thrive and be happy with autonomy and evolve toward more intelligent species so we can live together in a diverse human animal utopia, yay big tent EA…”
But logically I’m like “AI and/or other exponential technologies are right around the corner and make animal issues completely immaterial. Anything that detracts from progress on that is a distraction and should be completely and deliberately ignored. Optimally we will build an AI or other system that determines maximum utility per unit of matter, possibly including agency as a factor and quite possibly not, so that we can tile the universe with sentient simulations of whatever the answer is.”
OR, a similar discordance between what was just described and the view that we should also co-optimize for agency, diversity of values and experience, fun, decentralization, etc., EVEN IF that means possibly locking in a state of ~99.9999+percent of possible utility unrealized.
Very frustrating, I usually try to push myself toward my rational conclusion of what is best with a wide girth for uncertainty and epistemic humility, but it feels depressing, painful, and self-de-humanizing to do so.
Good question. Like most numbers in this post, it is just a very rough approximation used because it is a round number that I estimate is relatively close (~within an order of magnitude) to the actual number. I would guess that the number is somewhere between $50 and $200.
Thanks Mo! These estimates were very interesting.
As to discount rates, I was a bit confused reading William MacAskill’s discount rate post, it wasn’t clear to me that he was talking about the moral value of lives in the future, it seemed like it might be having something to do with value of resources. In “What We Owe The Future” which is much more recent, I think MacAskill argues quite strongly that we should have a zero discount rate for the moral patienthood of future people.
In general, I tend to use a zero discount rate, I will add this to the background assumptions section, as I do think it is an important point. In my opinion, future people and their experience do not have any more or less valuable than people live today, though of course other people may differ. I try to address this somewhat in the section titled “Inspiration.”
Thank you so much for this reply! I’m glad to know there is already some work on this, makes my job a lot easier. I will definitely look into the articles you mentioned and perhaps just study AI risk / AI safety a lot more in general to get a better understanding of how people think about this. It sounds like what people call “deployment” may be very relevant, so well especially look into this.
Yes, I agree this is somewhat what Bostrom is arguing. As I mentioned in the post, I think there may be solutions which don’t require totalitarianism, i.e. massive universal moral progress. I know this sounds intractable, I might address why I think this maybe mistaken in a future post, but it is a moot point if a vulnerable world induced X-risk scenario is unlikely, hence why I am wondering if there has been any work on this.
Ah yes! I think I see what you mean.
I hope to research topics related to this in the near future, including in-depth research on anthropics, as well as on what likely/desirable end-states of the universe are (including that we may already be in an end-state simulation) and what that implies for our actions.
I think this could be a 3rd reason for acting to create a high amount of well-being for those close to you in proximity, including yourself.
Hey Carl! Thanks for your comment. I am not sure I understand. Are you arguing something like “comparing x-risk interventions to other inventions such as bed nets is invalid because the universe may be infinite, or there may be a lot of simulations, or some other anthropic reason may make other interventions more valuable”?
This short-form supplements a post estimating how many lives x-risk work saves on average.
Following are four alternative pessimistic scenarios, two of which are highly pessimistic, and two of which fall between pessimistic and moderate.
Except where stated, each has the same assumptions as the original pessimistic estimate, and is adjusted from the baseline estimates of 10^16 lives possible and one life saved per hour of work or $100 donated.
It is 100% impossible to prevent existential risk, or it is 100% impossible to accurately predict what will reduce X-risk. In this case, we get an estimate that in expectation, on average, x-risk work may extremely pessimistically have zero positive impact and have the negative impact of wasting resources. I think it is somewhat unreasonable to conclude with absolutely certainty an existential catastrophe is inevitable or unpredictable, but others may disagree.
Humanity lasts as long as the typical mammalian species, ~1 million years. This would lead to three orders of magnitude reduction in expected value from the pessimistic estimate, giving an estimate that over the next 10,000 years, in expectation, on average, x-risk work will very pessimistically save one life for every 1,000 hours of work or every $100,000 donated. *Because humanity goes extinct in a relatively short amount of time in this scenario, x-risk work has not technically sustainably prevented existential risk, but this estimate has the benefit of using other species to give an outside view.
Digital minds are possible, but interstellar travel is impossible. This estimate is highly speculative. My understanding is that Bostrom estimated 15 additional orders of magnitude if digital minds are possible, given that we are able to inhabit other star systems. I have no idea if anything like this holds up if we only inhabit earth. But if it does, assuming a 1⁄10 chance digital minds are possible, the possibility of digital minds gives a 14 orders of magnitude increase from the original pessimistic estimate so that, over the next 10,000 years, in expectation, on average, x-risk work will moderately pessimistically save approximately one trillion lives per minute of work or per dollar donated.
Interstellar travel is possible, but digital minds are impossible. Nick Bostrom estimates that if emulations are not possible and so humans must remain in biological form, there could be 10^37 biological human lives at 100 years per life, or 21 orders of magnitude greater than the original pessimistic estimate. Assuming a 1⁄10 chance interstellar travel is possible, this adds 20 orders of magnitude so that, over the next 10,000 years, in expectation, on average, x-risk work will moderately pessimistically save approximately a billion billion (10^18) lives per minute of work or per dollar donated.
Thanks Spencer, really appreciated the variety of guests, this was a great podcast.
Is this contest still active after the FTX fiasco?
UPDATE: QUESTION HAS BEEN ANSWERED—https://forum.effectivealtruism.org/posts/3kaojgsu6qy2n8TdC/pre-announcing-the-2023-open-philanthropy-ai-worldviews
Fantastic news!!! My main question:
The Future Fund AI Worldview Prize had specific, very bold criteria, such as raising or lowering to certain thresholds the probability estimates of transformative AI timelines or probabilities of an AI related catastrophe, given certain timelines;
Will this AI Worldview Prize have very similar criteria, or do you have any intuitions what these criteria might be?
This would be very helpful for researchers like myself deciding whether to continue on a particular line of research!
This short-form is a minimally edited outtake from a piece estimating how many lives x-risk work saves on average. It conservatively estimates how much better future lives might be with pessimistic, moderate, and optimistic assumptions.
A large amount of text is in footnotes because it was highly peripheral to the original post.
(human remain on earth, digital minds are impossible)
We could pessimistically assume that future lives can only be approximately as good as present lives for some unknown reasons concerning the limits of well-being, or well-being in the long-term future.
(digital minds are possible)
It seems likely future lives will have significantly greater well-being or happiness than present lives, especially if brain emulations are possible.
If humans become digital people, we can edit our source code such that we are much happier. Additionally, by designing our digital environment we will be better able to achieve our desires, experience pleasure and avoid pain, and have the good things in life that support happiness.
Because this is speculative, let’s conservatively assume future people could be 10 times happier than present people, leading to one order of magnitude increase in expected value.[1] I will come back to why I think this is conservative and a higher estimate in the optimistic section.
(digital minds are possible)
In the moderate estimate we assumed future lives would have ten times greater well-being or happiness than the average present human life. Yet there isn’t any reason in principle to assume future people couldn’t be many orders of magnitude happier.[2]
In the moments before a seizure, Dostoyevsky reported feeling:
“A happiness unthinkable in the normal state and unimaginable for anyone who hasn’t experienced it… I am then in perfect harmony with myself and the entire universe”
And he wrote this experience into one of his characters, who stated,
“I would give my whole life for this one instant”
Taken literally, if we assume this experience lasted 10 seconds, that would make it over 100 million times better than his average experience.
I can say that I have personally experienced altered states of consciousness which were many orders of magnitude better than my average experience, and far better than even other very good experiences.[3]
Most people have occasional peak experiences that are extremely good when compared with average experience. We may postulate maximum happiness is a Bostromian utopia of unimaginable ecstasy, (maximized hedonic well-being) or a Karnofskian future of maximum choice and freedom (“‘meta’ option”,) (maximized preference satisfaction), or perhaps gradients of bliss so that we can freely navigate choices while also being in unimaginable ecstasy.
In any case, humans are not currently optimized for happiness. We could optimistically estimate that if future minds were optimized for happiness there could be something like a minimum of one million times more happiness per unit of time than we currently experience, leading to 6 orders of magnitude increase in expected value.[4]
Because I am using a utilitarian expected value framework, I will assume that an individual being 10 times happier is equivalent to 10 times as many well-being adjusted lives;
In other words,
Average happiness * number of lives = well-being adjusted lives
I will not give special attention to the impact on suffering in this analysis.
Although I lean toward prioritarianism (utilitarianism which prioritizes reducing suffering over increasing positive wellbeing,) my naive intuition it that it is insufficiently likely there will be a large enough amount of suffering in the far-future for there to be much impact on expected value (which was the focus of the piece this short-form was originally embedded in.) Admittedly, I have not spent much time studying s-risks and would appreciate other perspectives.
Additionally, I once briefly dated a woman and remember distinctly thinking to myself multiple times “I would trade 1 minute of kissing this woman for every other kiss, maybe even every other sexual experience I have ever had,” and this still feels accurate. I have repeatedly tried (unsuccessfully) to capture this experience in a poem.
Assuming this intuition was accurate, if I had previously had an average of 12 minutes of kissing/sexual experience per day over about 2 aggregated years of dating by that point in my life, that 1 minute was about 10,000 times better than my average kissing/sexual experience—and these, in turn, were worth perhaps 100 average minutes in a day (I like kissing and sex quite a lot), making a minute of kissing this woman one million times better than my average experience, or about 3 years of my life. While this seems absurd, I doubt it is off by more than ~two orders of magnitude.
Concretely, this means that if the happiest minute of your life was worth 100 days of normal experience, this would be one order of magnitude greater happiness than that.
Such a state may seem difficult to maintain long-term. But considering we can already induce and sustain profoundly intense states of wellbeing over several hours with certain chemical substances, it seems plausible this could be the case.
This section is highly speculative, as it deals with aspects of phenomenology that we are not yet able to empirically test. It is possible that optimal experience requires ups and downs, complexity, moving from a worse past to a better future, or can’t be sustained indefinitely.
That said, this could also be underestimating the real possible happiness by many orders of magnitude. Complete scientific understanding of phenomenology and powerful consciousness engineering techniques could unlock levels of happiness we currently have no way to conceive.
Mmm yeah, I really like this compromise, it leaves room for being human, but indeed, I’m thinking more about career currently. Since I’ve struggled to find a career that is impactful and I am good at, I’m thinking I might actually choose a career that is a relatively stable normal job that I like (Like therapist for enlightened people/people who meditate), and then I can use my free time to work on projects that could be maximally massively impactful.