I’m interested in how many 2021 $s you’d think it’s rational for EA be willing to trade (or perhaps the equivalent in human capital) against 0.01% (or 1 basis point) of existential risk.
This question is potentially extremely decision-relevant for EA orgs doing prioritization, like Rethink Priorities. For example, if we assign $X to preventing 0.01% of existential risk, and we take Toby Ord’s figures on existential risk (pg. 167, The Precipice) on face value, then we should not prioritize asteroid risk (~1/1,000,000 risk this century), if all realistic interventions we could think of costs >>1% of $X, or prioritize climate change (~1/1,000 risk this century) if realistic interventions costs >>$10X, at least on direct longtermism grounds (though there might still be neartermist or instrumental reasons for doing this research).
To a lesser extent, it may be relevant for individuals considering whether it’s better to earn-to-give vs contribute to existential risk reduction, whether in research or in real-world work.
Assume the money comes from a very EA-aligned (and not too liquidity-constrained) org like Open Phil.
Note: I hereby define existential risk the following way(see discussion in comments for why I used a non-standard definition):
Existential risk – A risk of catastrophe where an adverse outcome would permanently cause Earth-originating intelligent life’s astronomical value to be <50% of what it would otherwise be capable of.
Note that extinction (0%) and maximally bad[1] s-risks (-100%) are special cases of <50%.
[1] assuming symmetry between utility and disutility
Here are my very fragile thoughts as of 2021/11/27:
Speaking for myself, I feel pretty bullish and comfortable saying that we should fund interventions that we have resilient estimates of reducing x-risk ~0.01% at a cost of ~$100M.
I think for time-sensitive grants of an otherwise similar nature, I’d also lean optimistic about grants costing ~$300M/0.01% of xrisk, but if it’s not time-sensitive I’d like to see more estimates and research done first.
For work where we currently estimate ~$1B/0.01% of xrisk, I’d weakly lean against funding them right now, but think research and seed funding into those interventions is warranted, both for value of information reasons and also because we may have reasonable probability on more money flowing into the movement in the future, suggesting we can lower the bar for future funding. I especially believe this for interventions that are relatively clear-cut and the arguments are simple, or otherwise may be especially attractive to non(currently)-EA billionaires and governments, as we may have future access to non-aligned (or semi-aligned) sources of funding, with either additional research or early-stage infrastructure work that can then leverage more funding.
For work where we currently estimate >>$10B/0.01% of xrisk, I’d be against funding them and weakly lean against deep research into them, with important caveats including a) value of information and b) extremely high scale and/or clarity. For example, the “100T guaranteed plan to solve AI risk” seems like a research project worth doing and laying the foundations of, but not a project worth us directly funding, and I’d be bearish on EA researchers spending significant time on doing research for interventions at similar levels of cost-effectiveness for significantly smaller risks.
One update that has probably not propagated enough for me is a (fairly recent, to me) belief that longtermist EA has a much higher stock of human capital than financial capital. To the extent this is true, we may expect a lot of money to flow in in the future. So as long as we’re not institutionally liquidity constrained, we/I may be systematically underestimating how many $s we should be willing to trade off against existential risk.
*How* are you getting these numbers? At this point, I think I’m more interested in the methodologies of how to arrive at an estimate than about the estimates themselves
For the top number, I’m aware of at least one intervention* where people’s ideas of whether it’s a good idea to fund moved from “no” to “yes” in the last few years, without (AFAIK) particularly important new empirical or conceptual information. So I did a backwards extrapolation from that. For the next numbers, I a) considered that maybe I’m biased in favor of this intervention, so I’m rosier on the xrisk reduced here than others (cf optimizer’s curse), and grantmakers probably have lower numbers on xrisks reduced and b) considered my own conception of how much capital may come to longtermist EA in the future, and decided that I’m probably rosier on our capital than these implied numbers will entail. Put another way, I started with a conservative-ish estimate, then updated down on how much xrisk we can realistically buy off for $X at current margins (which increases $/xrisk), then updated upwards on how much $s we can have access to (which also increases $/xrisk).
My friend started with all the funds available in EA, divided by the estimated remaining xrisk, and then applied some discount factor for marginal vs average interventions, and got some similar if slightly smaller numbers.
*apologies for the vagueness
You said ‘I’m aware of at least one intervention* where people’s ideas of whether it’s a good idea to fund moved from “no” to “yes” in the last few years’. Would you be able to provide the source of this please?
No, sorry!
Do you similarly think we should fund interventions that we have resilient estimates of reducing x-risk ~0.00001% at a cost of ~$100,000? (i.e. the same cost-effectiveness)
Yep, though I think “resilient” is doing a lot of the work. In particular:
I don’t know how you can get robust estimates that low.
EA time is nontrivially expensive around those numbers, not just doing the intervention but also identifying the intervention in the first place and the grantmaker time to evaluate it, so there aren’t many times where ~0.00001% risk reductions will organically come up.
The most concrete thing I can think of is in asteroid risk, like if we take Ord’s estimates of 1⁄1,000,000 risk this century literally, and we identify a cheap intervention that we think can a) avert 10% of asteroid risks, b) costs only $100,000 , c) can be implemented by a non-EA with relatively little oversight, and d) has negligible downside risks, then I’d consider this a pretty good deal.
An LTFF grantmaker I informally talked to gave similar numbers
Could you say whether this was Habryka or not? (Since Habryka has now given an answer in a separate comment here, and it seems a bit good to know whether those are the same data point twice or not. Habryka’s number seems a factor of 3-10 off of yours, but I’d call that “similar” in this context.)
(It was not)
To what degree do you think the x-risk research community (of ~~100 people) collectively decreases x-risk? If I knew this, then you would have roughly estimated the value of an average x-risk researcher.
To make that question more precise, we’re trying to estimate xrisk_{counterfactual world without those people} - xrisk_{our world}, with xrisk_{our world}~1/6 if we stick to The Precipice’s estimate.
Let’s assume that the x-risk research community completely vanishes right now (including the past outputs, and all the research it would have created). It’s hard to quantify, but I would personally be at least twice as worried about AI risk that I am right now (I am unsure about how much it would affect nuclear/climate change/natural disasters/engineered pandemics risk and other risks).
Now, how much of the “community” was actually funded by “EA $”? How much of those researchers would not be capable of the same level of output without the funding we currently have? How much of the x-risk reduction is actually done by our impact in the past (e.g. new sub-fields of x-risk research being created, where progress is now (indirectly) being made by people outside of the x-risk community) vs. researcher hours today? What fraction of those researchers would still be working on x-risk on the side even if their work wasn’t fully funded by “EA $”?
EDIT 2022/09/21: The 100M-1B estimates are relatively off-the-cuff and very not robust, I think there are good arguments to go higher or lower. I think the numbers aren’t crazy, partially because others independently come to similar numbers (but some people I respect have different numbers). I don’t think it’s crazy to make decisions/defer roughly based on these numbers given limited time and attention. However, I’m worried about having too much secondary literature/large decisions based on my numbers, since it will likely result in information cascades. My current tentative guess as of 2022/09/21 is that there are more reasons to go higher (think averting x-risk is more expensive) than lower. However, overspending on marginal interventions is more -EV than underspending, which pushes us to bias towards conservatism.
I assume those estimates are for current margins? So if I were considering whether to do earning to give, I should use lower estimates for how much risk reduction my money could buy, given that EA has billions to be spent already and due to diminishing returns your estimates would look much worse after those had been spent?
Yes it’s about marginal willingness to spend, not an assessment of absolute impact so far.
With 7 billion people alive today, and our current most cost effective interventions for saving a life in the ballpark of $5000 USD, then with perfect knowledge we should probably be willing to spend up to ~$3.5 billion USD to reduce the risk of extinction or similar this century by 0.01 percentage points even if you’re not all that concerned about future people. Is my math right?
(Of course, we don’t have perfect knowledge and there are other reasons why people might prefer to invest less.)
Reasons to go higher:
We may believe that existential risk is astronomically bad, such that killing 8 billion people is much worse than 2x as bad as killing 4 billion people.
Reasons to go lower:
Certain interventions in reducing xrisk may save a significantly lower number of present people’s lives than 8 billion, for example much work in civilizational resilience/recovery, or anything that has a timeline of >20 years for most of the payoff.
As a practical matter, longtermist EA has substantially less money than implied by these odds.
Agreed on all points
Did you mean 0.01%?
Yes sorry, edited!
Around $3 billion also sounds intuitively about right compared to other things governments are willing to spend money on.
governments seem way more inefficient to me than this.
My very rough gut estimate says something like $1B. Probably more on the margin, just because we seem to have plenty of money and are more constrained on other problems. I think we are currently funding projects that are definitely more cost-effective than that, but I see very little hope in scaling them up drastically while preserving their cost-effectiveness, and so spending $1B on 0.01% seems pretty reasonable to me.
I thought it would be interesting to answer this using a wrong method (writing the bottom line first). That is, I made an assumption about what the cost-effectiveness of grants aimed at reducing existential risk is, then calculated how much it would cost to reduce existential risk by one basis point if that cost-effectiveness was correct.
Specifically, I assumed that Ben Todd’s expectation that Long-Term Future Fund grants are 10-100x as cost-effective as Global Health and Development Fund grants is correct:
I made the further assumptions that:
Global Health and Development Fund (GH) grants are as cost-effective as GiveWell’s top charities in expectation
Global Health grants are 8x as cost effective as GiveDirectly
GiveWell’s estimate that “donating $4,250 to a charity that’s 8x GiveDirectly is as good as saving the life of a child under five” (Ben Todd’s words) is correct.
All existential catastrophes are extinction events.
Long-Term Future Fund grants do good via reducing the probability of an extinction event in the next 100 years.
The only good counted towards Long-Term Future Fund grant cost-effectiveness is the human lives directly saved from dying in the extinction event in the next century. (The value of preserving the potential for future generations is ignored, as is the value of saving lives from any non-extinction level global catastrophic risk reduction.)
Saving the lives of people then-alive from dying in an extinction event in the next 100 years is as valuable on average as saving the life of a child under five.
If an extinction event happens in the next century, 8 billion people will die.
From these assumptions, it follows that:
Donating to the LTFF is 10-100x as cost-effective as donating to the GHDF and 80-800x as cost-effective as donating to GiveDirectly.
Donating $4,250 to the LTFF does as much good as saving the lives of 10-100 people from dying in an extinction event in the next 100 years.
Reducing the risk of human extinction in the next century by 0.01% is equivalent to saving the lives of 800,000 people.
It costs $34M-$340M (i.e. ~$100M) (donated to the LTFF) to reduce the probability of extinction in the next 100 years by 0.01% (one basis point). (Math: $4,250 * 800,000⁄100 to $4,250 * 800,000⁄10.)
To reiterate, this is not the right way to figure out how much it costs to reduce existential risk.
I worked backwards from Ben Todd’s statement of his expectations about the cost-effectiveness of grants to the LTFF, making the assumption that his cost-effectiveness estimate is correct, and further assuming that those grants are valuable only due to their effect on saving lives from dying in extinction events in the next century.
In reality, those grants may also help people from dying in non-extinction level global catastrophic risks, increasing the estimate of the cost to reduce extinction risk by one basis point. Further, there is also a lot of value in how preventing extinction preserves the potential for having future generations. If Ben Todd’s 10-100x expectation included this preserving-future-generations value, then that too would increase the estimate of the cost to reduce extinction risk by one basis point.
Just to complement Khorton’s answer: With a discount rate of d [1], and a steady-state population of N, and a willingness to pay of $X, the total value of the future is N∗$X/d, so the willingness to pay for 0.01% of it would be 0.01∗N∗$X/d
This discount rate might be because you care about future people less, or because you expect a d% of pretty much unavoidable existential risk going forward.
Some reference values
N=1010 (10 billion), X=$104, r=0.03 means that willingness to pay for 0.01% risk reduction should be 0.0001∗1010∗$104/0.03=333∗109, i.e., $333 billion
N=7∗109 (7 billion), X=$5∗103, r=0.05 means that willingness to pay for 0.01% risk reduction should be 0.0001∗7∗109∗5∗103/0.05=70∗109 i.e., $70 billion.
I notice that from the perspective of a central world planner, my willingness to pay would be much higher (because my intrinsic discount rate is closer to ~0%). Taking d=0.0001
N=1010 (10 billion), X=$104, r=0.0001 means that willingness to pay for 0.01% risk reduction should be 0.0001∗1010∗$104/0.0001=100∗1012, i.e., $100 trillion
To do:
The above might be the right way to model willingness to pay from 0.02% risk per year to 0.01% risk per year. But with, e.g,. 3% remaining per year, willingness to pay is lower, because over the long-run we all die sooner.
E.g., reducing risk from 0.02% per year to 0.01% per year is much more valuable that reducing risk from 50.1% to 50%.
[1]: Where you value the i-th year in the steady-state at (1−d)i of the value of the first year. If you don’t value future people, the discount rate d might be close to 1, if you do value them, it might be close to 0.
Here is a Guesstimate model which addresses the item on the to-do list. Note that in this guesstimate I’m talking about a −0.01% yearly reduction.
Here is a Guesstimate which calculates this in terms of a one-off 0.01% existential risk reduction over a century.
The steady-state population assumption is my biggest objection here. Everything you’ve written is correct yet I think that one premise is so unrealistic as to render this somewhat unhelpful as a model. (And as always, NPV of the eternal future varies a crazy amount even within a small range of reasonable discount rates, as your numbers show.)
For what it’s worth, I don’t disagree with you, though I do think that the steady state is a lower bound of value, not an upper bound.
Thinking more about this, these are more of an upper bound, which don’t bind because you can probably buy a 0.01% risk reduction per year much cheaper. So the parameter to estimate would be more like ‘what are the other cheaper interventions’
In Ajeya Cotra’s interview with 80,000 Hours, she says:
This suggests a funding bar of 200T/10,000 world~= 20B for every 0.01% of existential risk.
These numbers seem very far from my own estimates for what marginal $s can do or the (AFAICT) apparent revealed preferences of Open Phil’s philantropic spending, so I find myself very confused about the discrepancy.
Cotra seems to be specifically referring to the cost-effectiveness of “meta R&D to make responses to new pathogens faster.” She/OpenPhil sees this as “conservative,” a “lower bound” on the possible impact of marginal funds, and believes “AI risk is something that we think has a currently higher cost effectiveness.” I think they studied this intervention just because it felt robust and could absorb a lot of money.
So I expect Cotra is substantially more optimistic than $20B per basis point. That number is potentially useful as a super-robust minimum-marginal-value-of-longtermist-funds to compare to short-term interventions, and that’s apparently what OpenPhil wanted it for.
This is correct.
Thanks! Do you or others have have any insight on why having a “lower bound” is useful for a “last dollar” estimation? Naively having a tight upper bound is much more useful (so we’re definitely willing to spend $$s on any intervention that’s more cost-effective than that).
I don’t think it’s generally useful, but at the least it gives us a floor for the value of longtermist interventions. $20B per basis point is far from optimal but still blows eg GiveWell out of the water, so this number at least tells us that our marginal spending should be on longtermism over GiveWell.
Can you elaborate on your reasoning here?
~20 billion * 10, 000 /~8 billion ~= $25,000 $/life saved. This seems ~5x worse than AMF (iirc) if we only care about present lives, done very naively. Now of course xrisk reduction efforts save older people, happier(?) people in richer countries, etc, so the case is not clearcut that AMF is better. But it’s like, maybe similar OOM?
(Though this very naive model would probably point to xrisk reduction being slightly better than GiveDirectly, even at 20B/basis point, even if you only care about present people, assuming you share Cotra’s empirical beliefs and you don’t have intrinsic discounting for uncertainty)
Now I will probably prefer 20B/basis point over AMF, because I like to take ideas seriously and longtermism is one of the ideas I take seriously. But this seems like a values claim, not an empirical one.
Hmm, fair. I guess the kind of people who are unpersuaded by speculative AI stuff might also be unswayed by the scope of the cosmic endowment.
So I amend my takeaway from the OpenPhil number to: people who buy that the long-term future matters a lot (mostly normative) should also buy that longtermism can absorb at least $10B highly effectively (mostly empirical).
I don’t understand. Total willingness to pay, and the belief of the marginal impact of $, are different things.
Also, when buying things, we often spend a small fraction of our total willingness to pay (e.g. imagine paying even 10% of the max value for transportation or healthcare each time).
We are accustomed to paying a fraction of our willingness to pay and also it’s how usually things work out. For preventative measures like x-risk, we might also expect this fraction to be low, because it benefits from planning and optimization.
Your own comment here, and Ben Todd’s messaging says that EA spending is limited by top talent that can build and deploy new large scale institutions in x-risk.
I feel like I’m missing something or talking past you?
Maybe I’m missing something, but I’m assuming a smooth-ish curve for things people are just barely willing to fund vs are just barely unwilling to fund.
My impression is that longtermist things just below that threshold have substantially higher naive cost-effectiveness than 20B/0.01% of xrisk.
It’s easier for me to see this for longtermist interventions other than AI risk, since AI risk is very confusing. A possible response for the discrepancy is that maybe grantmakers are much less optimistic (like by 2 orders of magnitude on average) about non-AI xrisk reduction measures than I am. For example, one answer that I sometimes hear (~never from grantmakers) is that people don’t really think of xrisks outside of AI as a thing. If this is the true rejection, I’d a) consider it something of a communications failure in longtermism and b) would note that we are pretty inefficiently allocating human capital then.
You’re the economist here, but my understanding is that the standard argument for this is that people spend $s until marginal spending per unit of goods equals marginal utility of that good. So if we value the world at 200T EA dollars, we absolutely should spend $s until it costs 20B in $s (or human capital-equivalents) per basis point of xrisks averted.
Ok, I am confused and I don’t immediately know where the above reply fits in.
Resetting things and zooming out to the start:
Your question seems to be talking about two different things:
Willingness to pay to save the world (“E.g. 0.01% of 200T”)
The actual spending and near-terms plans of 2021 EA’s willingness to spend on x-risk related programs, e.g. “revealed preferences of Open Phil’s philantropic spending” and other grantmakers as you mentioned.
Clearly these are different things because EA is limited by money. Importantly, we believe EA might have $50B (and as little as $8B before Ben Todd’s 2021 update). So that’s not $200T and not overly large next to the world. For example, I think annual bank overdraft fees in the US alone are like ~$10B or something.
Would the following help to parse your discussion?
If you want to talk about spending priorities right now with $50B (or maybe much more as you speculate, but still less than 200T), that makes sense.
If you want to talk about what we would spend if Linch or Holden were the central planner of the world and allocating money to x-risk reduction, that makes sense too.
I was referring to #1. I don’t think fantasizing about being a central planner of the world makes much sense. I also thought #1 was what Open Phil was referring to when they talk about the “last dollar project”, though it’s certainly possible that I misunderstood (for starters I only skim podcasts and never read them in detail).
Ok, this makes perfect sense. Also this is also my understanding of “last dollar”.
My very quick response, which may be misinformed , is that Open Phil is solving some constrained spending problem with between $4B and $50B of funds (e.g. the lower number being half of the EA funds before Ben Todd’s update and the higher number being estimates of current funds).
Basically, in many models, the best path is going to be some fraction, say 3-5% of the total endowment each year (and there are reasons why it might be lower than 3%).
There’s no reason why this fraction or $ amount rises with the total value of the earth, e.g. even if we add the entire galaxy, we would spend the same amount.
Is this getting at your top level comment “I find myself very confused about the discrepancy”?
I may have missed something else.
I think reducing x-risk is by far the most cost-effective thing we can do, and in an adequate world all our efforts would be flowing into preventing x-risk.
The utility of 0.01% x-risk reduction is many magnitudes greater than the global GDP, and even if you don’t care at all about future people, you should still be willing to pay a lot more than currently is paid for 0.01% x-risk reduction, as Korthon’s answer suggests.
But of course, we should not be willing to trade so much money for that x-risk reduction, because we can invest the money more efficiently to reduce x-risk even more.
So when we make the quite reasonable assumption that reducing x-risk is much more effective than doing anything else, the amount of money we should be willing to trade should only depend on how much x-risk we could otherwise reduce through spending that amount of money.
To find the answer to that, I think it is easier to consider the following question:
How much more likely is an x-risk event in the next 100 years if EA looses X dollars?
When you find the X that causes a difference in x-risk of 0.01%, the X is obviously the answer to the original question.
I only consider x-risk events in the next 100 years, because I think it is extremely hard to estimate how likely x-risk more than 100 years into the future is.
Consider (for simplicity) that EA currently has 50B$.
Now answer the following questions:
How much more likely is an x-risk event in the next 100 years if EA looses 50B$?
How much more likely is an x-risk event in the next 100 years if EA looses 0$?
How much more likely is an x-risk event in the next 100 years if EA looses 20B$?
How much more likely is an x-risk event in the next 100 years if EA looses 10B$?
How much more likely is an x-risk event in the next 100 years if EA looses 5B$?
How much more likely is an x-risk event in the next 100 years if EA looses 2B$?
Consider answering those questions for yourself before scrolling down and looking at my estimated answers for those questions, which may be quite wrong. Would be interesting if you also comment your estimates.
The x-risk from EA loosing 0$ to 2B$ should increase approximately linearly, so if x is the x-risk if EA looses 0$ and y is the x-risk if EA looses 2B$, you should be willing to pay d=0,01%y−x2B$ for a 0.01% x-risk reduction.
(Long sidenote: I think that if EA looses money right now, it does not significantly affect the likelihood of x-risk more than 100 years from now. So if you want to get your answer for the “real” x-risk reduction, and you estimate a z% chance of an x-risk event that happens strictly after 100 years, you should multiply your answer by 1/(1−z%) to get the amount of money you would be willing to spend for real x-risk reduction. However, I think it may even make more sense to talk about x-risk as the risk of an x-risk event that happens in the reasonably soon future (i.e. 100-5000 years), instead of thinking about the extremely long-term x-risk, because there may be a lot we cannot foresee yet and we cannot really influence that anyways, in my opinion.)
Ok, so here are my numbers to the questions above (in that order):
17%,10%,12%,10.8%,10.35%,10.13%
So I would pay 0.01%0.13%2B$=154M$ for a 0.01% x-risk reduction.
Note that I do think that there are even more effective ways to reduce x-risk, and in fact I suspect most things longtermist EA is currently funding have a higher expected x-risk reduction than 0.01% per 154M$. I just don’t think that it is likely that the 50 billionth 2021 dollar EA spends has a much higher effectiveness than 0.01% per 154M$, so I think we should grant everything that has a higher expected effectiveness.
I hope we will be able to afford to spend many more future dollars to reduce x-risk by 0.01%.
I’d guess that quite-well-directed marginal funding can buy a basis point for something like $50M (for example, I’d expect to be able to buy a basis point by putting that money toward a combination of alignment research, AI governance research, and meta-stuff like movement-building around AI). Accounting for not all longtermist funding being so well-directed gives something like $100M of longtermism funding per basis point, or substantially more if we’re talking about all-of-EA funding (insofar as non-longtermism funding buys quite little X-risk reduction).
But on reflection, I think that’s too high. I arrived at $50M by asking myself “what would I feel pretty comfortable saying could buy a basis point.” Considering a reversal test, I would absolutely not take the marginal $100M out of [OpenPhil’s longtermism budget / longtermist organizations] to buy one basis point. Reframing as “what amount would I not feel great trading for a basis point in either direction,” I instinctively go down to more like $25M of quite-well-directed funding or $50M of real-world longtermism funding. EA could afford to lose $50M of longtermism funding, but it would hurt. The Long-Term Future Fund has spent less than $6M in its history, for example. Unlike Linch, I would be quite sad about trading $100M for a single measly basis point — $100M (granted reasonably well) would make a bigger difference, I think.
I suspect others may initially come up with estimates too high due to similarly framing the question as “what would I feel pretty comfortable saying could buy a basis point,” like I originally did. If your answer is $X, I encourage you to make sure that you would take away $X of longtermist funding from the world to buy a single basis point.
Although, one thing to flag is that a lot of the resources in LT organizations is human capital.
Thanks for your thoughts.
To be clear, I also share the intuition that I feel a lot better about taking $s from Open Phil’s coffers than I do taking money from existing LT organizations, which probably is indicative of something.
Suppose there are N people and a baseline existential risk r. There’s an intervention that reduces risk by δ×100% (ie., not percentage points).
Outcome with no intervention: rN people die.
Outcome with intervention: (1−δ)rN people die.
Difference between outcomes: δrN. So we should be willing to pay up to δr⋅u(N) for the intervention, where u(N) is the dollar value of N lives.
[Extension: account for time periods, with discounting for exogenous risks.]
I think this approach makes more sense than starting by assigning $X to 0.01% risk reductions, and then looking at the cost of available interventions.
What does that even mean in this context? cost =/= value, and empirically longtermist EA has nowhere near the amounts of $s as would be implied by this model.
@Linch, I’m curious if you’ve taken an intermediate microeconomics course. The idea of maximizing utility subject to a budget constraint (ie. constrained maximization) is the core idea, and is literally what EAs are doing. I’ve been thinking for a while now about writing up the basic idea of constrained maximization, and showing how it applies to EAs. Do you think that would be worthwhile?
I did in sophomore year of college (Varian’s book I think?), but it was ~10 years ago so I don’t remember much. 😅😅😅. A primer may be helpful, sure.
(That said, I think it would be more worthwhile if you used your model to produce an answer to my question, and then illustrate some implications).
I’d love to read such a post.
Given an intervention of value v, you should be willing to pay for it if the cost c satisfies c≤v (with indifference at equality).
If your budget constraint is binding, then you allocate it across causes so as to maximize utility.
I’m curious about potential methodological approaches to answering this question:
Arrive at a possible lower bound for the value of averting x-risk by thinking about how much one is willing to pay to save present people, like in Khorton’s answer.
Arrive at a possible lower bound by thinking about how much is willing to pay for current and discounted future people
Thinking about what EA is currently paying for similar risk reductions, and arguing that one should be willing to pay at least as much for future risk-reduction opportunities
I’m unsure about this, but I think this is most of what’s going on with Linch’s intuitions.
Overall, I agree that this question is important, but current approaches don’t really convince me.
My intuition about what would convince me would be some really hardcore and robust modeling coming out of e.g., GPI taking into account both increased resources over time and increased risk. Right now the closest published thing that exists might be Existential risk and growth and Existential Risk and Exogenous Growth—but this is inadequate for our purposes because it considers stuff at the global rather than at the movement level—and the closest unpublished thing that exists are some models I’ve heard about that I hope will get published soon.