I’m am on a gap year from studying for a Master of Social Entrepreneurship degree at the University of Southern California.
I have thought along EA lines for as long as I can remember, and I recently wrote the first draft of a book “Ways to Save The World” about my top innovative ideas for broad approaches to reduce existential risk.
I am now doing more research on how these approaches interact with AI X-risk.
Fantastic news!!! My main question:
The Future Fund AI Worldview Prize had specific, very bold criteria, such as raising or lowering to certain thresholds the probability estimates of transformative AI timelines or probabilities of an AI related catastrophe, given certain timelines;
Will this AI Worldview Prize have very similar criteria, or do you have any intuitions what these criteria might be?
This would be very helpful for researchers like myself deciding whether to continue on a particular line of research!
Opportunity Cost Ethics
“Every man is guilty of all the good he did not do.”
~Voltaire
Opportunity Cost Ethics is a term I invented to capture the ethical view that failing to do good ought to carry the same moral weight as doing harm.
You could say that in Opportunity Cost Ethics, sins of omission are equivalent to sins of commission.
In this view, if you walk by a child drowning in a pond, and there is zero cost to you to saving the child, it would be equally morally inexcusable for you not to save the child from drowning as it would be to take a non-drowning child and drown them in the pond.
Moreover, if you have good reasons to suspect there are massive opportunities to do good, such as a reasonable probability you could save millions of lives, Opportunity Cost Ethics states that the moral thing to do is to seek out such opportunities and find the best possible opportunities, after accounting for search costs.
Opportunity Cost Ethics is a consequentialist ethical theory, as it states the only morally relevant fact is the ultimate effect of our actions or non-actions.
The strongest version of Opportunity Cost Ethics, which I hold, states that:
“The best moral action at any given time requires considering as many actions as possible, until marginal search costs exceed the expected value of finding better options; at which point an optimal stopping point has been reached. Considerations of proximity also need to be taken into account.”
We could call the above “Strong Opportunity Cost Ethics.”
Possibly the most important implication of Opportunity Cost Ethics is that we should spend a massive amount of time, energy, and resources trying to determine what the best possible actions we can take are.
It is very difficult to know how much time we should spend trying to figure how good various actions are, and how certain we should be before we act. Opportunity Cost Ethics states that it is morally right to continue searching for the best action you can take until you predict the marginal cost of searching for a better action outweighs the amount of additional good that the better action would achieve.
Due to its obvious importance I assume something like this already exists, if so please point me to it, thank!
edit: Forum user tc pointed out that the “doing / allowing” distinction in ethics, whether causing harm is equivalent to preventing harm, which inspired trolley dilemmas, is highly relevant.
This is the kind of thing I was looking for! I am, however, uncertain if it fully captures what I am saying, which is more about the potential to do a massive amount of good, and the ethical responsibility to seek out such opportunities if we suspect they are available and the search costs do not cancel out the good.
At some point I hope to extend this short-form into a more well-formed robust theory.
I agree this seems like a huge problem. I noticed that even though I am extremely committed to longtermism and have been for many years, the fact that I am skeptical of AI risk seems to significantly decrease my status in longtermist circles, there is a significant isularity and resistance to EA gospel being criticized, and little support for those seeking to do so
Hey everyone! Just joined EA a few months ago and was very fortunate to attend EAGx Bostone recently! I could not be more excited about discovering this community!!!
I’m doing two fellowships and working on a marketing project team in my university EA USC group.
I feel very strongly about utilitarianism, am interested in physics, and as a result came to longtermism several years ago on my own. I actually wrote a book called “Ways to Save The World” essentially about innovative broad strategies to sustainably, systemically reduce existntial risk. Really excited to share it with the EA community and have my ideas challenged and improved by fellow highly intelligent, rational do-gooders!
I love this post! Solarpunk was my first intuition as well, I think there is a lot of good evidence that green and natural environments support happiness and productivity, and so I don’t think it is actually out of alignment with utilitarianism or EA at all.
I have a theory of reality which makes aesthetics the fundamental force of the universe. To demonstrate this, if effective altruism is successful in colonizing space and ends up determining the shape of the future of the universe, then this “shape” will be whatever aesthetic shape we have determined creates maximum utility.
I think aesthetics is a much better fundamental for utilitarianism than pleasure, which intuitively seems quite base and basic. Therefore, I agree that aesthetics are exceedingly important in figuring out what future we want to create.
I think this is post is mistaken. (If I remember correctly, not an expert,) estimates that AI will kill us all are put around only 5-10% by AI experts and attendees at an x-risk conference in a paper from Katja Grace. Only AI Safety researchers think AI doom is a highly likely default (presumably due to selection effects.) So from near-termist perspective AI deserves relatively less attention.
Bio-risk and climate change, and maybe nuclear war, on the other hand, I think are all highly concerning from a near-termist perspective, but unlikely to kill EVERYONE, and so relatively low priority for long-termists.
I think that retroactive Impact Markets may be a net negative for many x-risk projects, however, I also think that in general Impact Markets may significantly reduce x-risk.
I think you have to bear in mind that if this project is highly successful, it has the potential to create a revolution in the funding of public goods. If humanity achieves much better funding and incentive mechanisms for public goods, this could create a massive increase in the efficiency of philanthropy.
It is hard to know how beneficial such a system would be, but it is not hard to see how mutilpying the effectiveness of philanthropy and public good provision could make society function much better by improving education, coordination, mental health, moral development, health, etc., and increases in these public goods could broadly improve humanity’s ability to confront many x-risks.
I think it may make sense for Impact Markets to find ways of limiting or excluding x-risk projects, but I think abandoning Impact Markets altogether would be a mistake, and considering their massive upsides I cannot say I agree that they are net negative in expectation, even without excluding x-risk projects.
This was awesome!
Here’s a question I love. Peter Thiel’s favorite interview question to job candidates:
“What important truth do very few people agree with you on?”
Hey Carl! Thanks for your comment. I am not sure I understand. Are you arguing something like “comparing x-risk interventions to other inventions such as bed nets is invalid because the universe may be infinite, or there may be a lot of simulations, or some other anthropic reason may make other interventions more valuable”?
Wow. This is a really great concrete story of the benefit of signaling. Yeah, I find it so fascinating how Effective Altruism has evolved, and I really love all parts of it and think it is a very natural progression which somewhat mirrors my own. It is really unfortunate that not everyone sees this whole context, and I agree worth putting some effort into managing impressions even if it is in a sense a sort of marketing “pacing” which gradually introduces more advanced concepts rather than throwing out some of the crazier sounding bits of EA right off the bat
Thanks Mo! These estimates were very interesting.
As to discount rates, I was a bit confused reading William MacAskill’s discount rate post, it wasn’t clear to me that he was talking about the moral value of lives in the future, it seemed like it might be having something to do with value of resources. In “What We Owe The Future” which is much more recent, I think MacAskill argues quite strongly that we should have a zero discount rate for the moral patienthood of future people.
In general, I tend to use a zero discount rate, I will add this to the background assumptions section, as I do think it is an important point. In my opinion, future people and their experience do not have any more or less valuable than people live today, though of course other people may differ. I try to address this somewhat in the section titled “Inspiration.”
This was amazing. As a professional dropout, I would like to join your organization so that I can immediately quit it.
I have dropped out of college 3 1⁄2 times now, the 1⁄2 time was during Covid when I didn’t quite start the school year before dropping out and deciding to become a homeless vagabond.
I always wanted to become an Ivy League dropout, USC isn’t Ivy League but close enough, it’s way more expensive than most Ivy League schools at least, and now I feel much more confident in whatever I do next. Lots of great entrepreneurs were dropouts from fancy pants schools. I think accomplishment is very closely correlated to dropping out, so just imagine how promising I must be having dropped out such an unusually high number of times.
In all seriousness, I am extremely proud of having dropped out so many times. Each time it was the right decision, and I believe dropping out of things in general when there is a higher expected value option available shows a willingness to kick the sunk-cost fallacy in the pants and follow your dreams. Especially if those dreams include being a homeless Covid vagabond living in a tent in your friend’s backyard.
Dang yeah I did a quick search on creatine and the IQ number right before writing this post, but now it’s looking like that source was not credible. Would have to research more to see if I can find an accurate reliable measure of creatine cognitive improvement, it seems it at least has a significant impact on memory. Anecdotally, I noticed quite a difference when I took a number of supplements while vegan, and I know there’s some research on a number of differences of various nutrients which vegans lack related to cognitive function. Will do a short post on sometime!
I think human alignment is incredibly difficult, but too important to ignore. I have thought about it a very long time so do have some very ambitious ideas that could feasibly start small and scale up.
Yes! I have been very surprised since joining how narrowly longtermism is focused. I think if the community is right about AGI being within a few decades with fast takeoff then broad longtermism may be less appealing, but I think if there is any doubt about this then we are massively underinvested in broad longtermism and putting all eggs in one basket so to speak. Will definitely write more about this!
Right, definitely wouldn’t be exactly analogous to GiveWell, but I think nonetheless it is important to have SOME way of comparing all the longtermist projects to know what a good investment looks like.
Thanks again for all the feedback Aman! Really appreciate it (and everything else you do for the USC group!!) and really excited to write more on some of these topics :)
Thanks William! This feedback is super valuable. Yes I think the massive scalable community building project would be novel and it actually ties in with the donor contest as well. Glad to know this would be useful! And good thought, I think writing about my own story will be easiest as well. And I will definitely write about broad longtermism, it is one of my main areas of interest.
The Proximity Principle
Excerpt from my upcoming book, “Ways To Save The World”
As was noted earlier, this does not mean we should necessarily pursue absolute perfect selflessness, if such a thing is even possible. We might conceive that this would include such activities as not taking any medicine so that those who are more sick can have it, not eating food and giving all of your food away to those who are starving, never sleeping but instead continually working for those who are less fortunate than yourself. As is obvious, all of these would lead imminently to death and so would ultimately not really be that good in the grand scheme of things.
Instead, the most effective way to be selfless is to be intelligently selfish in such a way that enables you to maximize your own capacity to serve others. This I call the “proximity principle,” which says that the most efficient way to do good for others is to prioritize that which is closest in proximity, both because by helping yourself and those closest to you, you are most likely to maximize your power to be effective in the future, and also because, all else being equal, it is simply easiest and most efficient to help those who are closest to you in time and space, rather than those who are on the opposite side of the world or billions of years in the future.
For example, it might be best to educate yourself, become financially independent, have a family and friends that meets your emotional needs, take care of yourself psychologically, and give these much higher prioritization than helping others, at least insofar as is practicable, since if you are falling apart in one or several of these ways, it might make it much more difficult for you to be effective in helping others. Once you have your own house in order, you can be exponentially more effective in helping the world at large and creating long-term impact.
For reference, I originally created this short-form so I could reference in an estimate of lives saved by longtermists. Here is the original context in which I mention it:
The take-away from this post is not that you should agonize over the trillions of trillions of trillions of men, women, and children you are thoughtlessly murdering each time you splurge on a Starbucks pumpkin spice latte or watch cat videos on tik-tok — or in anyway whatsoever commit the ethical sin of making non-optimal use of your time.
The point of this post is not to create a longtermist “dead children currency” analogue. Instead it is meant to be motivating background information, giving us all the more good reason to be thoughtful about our self-care and productivity.
I call the principle of strategically caring for yourself and those closest to you “The Proximity Principle,” something I discovered after several failed attempts to be perfectly purely altruistic. It roughly states that:
It is easiest to affect those closest to you (in time, space, and relatedness)
Taking care of yourself and those closest to you is high leverage for multiplying your own effectiveness in the future
To account for proximity, perhaps in addition to conversion rates for time and money into lives saved, we also need conversion rates for time and money into increases in personal productivity, personal health & wellbeing, mental health, self-development, personal relationships, and EA community culture.
These things may be hard to quantify, but probably less hard than we think, and seem like fruitful research directions for social-science oriented EAs. I think these areas are highly valuable relative to time and money, even if only valued instrumentally.
In general, for those who feel compelled to over-work to a point that feels unhealthy, have had a tendency to burn out in the past, or think this may be a problem for them, I would suggest erring on the side of over-compensating.
This means finding self-care activities that make you feel happy, energized, refreshed, and a sense of existential hope — and, furthermore, doing these activities regularly, more than the minimum you feel you need to in order to work optimally.
I like to think if this as keeping my tank nearly full, rather than perpetually halfway full or nearly empty. From a systems theory perspective, you are creating a continuous inflow and keeping your energy stocks high, rather than waiting til they are fully depleted and panic mode alerts you to refill.
For me, daily meditation, daily exercise, healthy diet, and good sleep habits are most essential. But each person is different, so find what works for you.
Remember, if you want to change the future, you need to be at your best. You are your most valuable asset. Invest in yourself.
Thank you Dony, Denis, and Matt! I really enjoyed reading this post, and excited about the idea. Looking forward to seeing what posts are submitted!
Thank you! Yes, I’m pretty new here, and now that you say that I think you’re right, anthropics makes more sense.
I am inclined to think the main thing required to be an observer would be enough intelligence to ask whether one is likely to be the entity one is by pure chance, and this doesn’t necessarily require consciousness, just the ability to assess likelihood one is in a simulation into one’s decision calculus.
I had not thought about the possibility that future beings are mostly conscious, but very few are intelligent enough to ask the question. This is definitely a possibility. Though if the vast majority of future beings are unintelligent, you might expect there to be far fewer simulations of intelligent beings like ourselves, somewhat cancelling this possibility out.
So yeah, since I think most future beings (or at least a very large number) will most likely be intelligent, I think the selection affects do likely apply.
Ah yes! I think I see what you mean.
I hope to research topics related to this in the near future, including in-depth research on anthropics, as well as on what likely/desirable end-states of the universe are (including that we may already be in an end-state simulation) and what that implies for our actions.
I think this could be a 3rd reason for acting to create a high amount of well-being for those close to you in proximity, including yourself.
Hmm.. I’d have to think more carefully about it. Was very much off-the-cuff. I mostly agree with your criticism, I think I was mainly thinking bio-risk makes most sense as a near-termist priority and so would get most of x-risk funding until solved, since it is much more tractable than AI Risk.
Maybe this is the main point I’m trying to make, and so the spirit of the post seems off, since near-termist x-risky stuff would mostly fund bio-risk and long-termist x-risky stuff would mostly go to AI.
Anyone else ever feel a strong discordance between emotional response and cognitive worldview when it comes to EA issues?
Like emotionally I’m like “save the animals! All animals deserve love and protection and we should make sure they can all thrive and be happy with autonomy and evolve toward more intelligent species so we can live together in a diverse human animal utopia, yay big tent EA…”
But logically I’m like “AI and/or other exponential technologies are right around the corner and make animal issues completely immaterial. Anything that detracts from progress on that is a distraction and should be completely and deliberately ignored. Optimally we will build an AI or other system that determines maximum utility per unit of matter, possibly including agency as a factor and quite possibly not, so that we can tile the universe with sentient simulations of whatever the answer is.”
OR, a similar discordance between what was just described and the view that we should also co-optimize for agency, diversity of values and experience, fun, decentralization, etc., EVEN IF that means possibly locking in a state of ~99.9999+percent of possible utility unrealized.
Very frustrating, I usually try to push myself toward my rational conclusion of what is best with a wide girth for uncertainty and epistemic humility, but it feels depressing, painful, and self-de-humanizing to do so.