Thank you, really appreciate the information
Froolow
Karma: 897
[Question] AI Risk Microdynamics Survey
This is absolutely incredible—can’t believe I missed it! Thank you so much
[Question] Questions on databases of AI Risk estimates
ADDENDUM: There is an excellent post here about why the discount rate is probably not zero based on an analysis of existential risk rates. However, I think the post assumes a lot of background familiarity with the theory of discount rates, and doesn’t – for example – explicitly identify that existential risk is probably not the biggest reason to discount the future (even absent time-preference). Would an effortpost which is a deep dive into discount rates as understood by economists be helpful?
Hyperbolic discounting is a ‘time inconsistent’ form of discounting where delays early on are penalised more than delays later on. This results in a ‘fat tail’ where it takes a long time for a hyperbolic function to get near zero. Over a long enough time period, an exponential function (for example growth in happiness driven by population growth) will always be more extreme than a hyperbolic function (for example discount rate in this scenario).
So actually the title should perhaps be reworded; the shape of the discount function matters just as much (if not more so than) the parameterisation of that function. A hyperbolic discount function will always result in longtermism dominating neartermism.
Having said that, I don’t think anyone believes hyperbolic discount rates are anything other than a function of time preference, and the consensus amongst EAs seems to be that time preference should be factored out of philanthropic analysis.
The discount rate effectively determines whether long- or near-termism is the best use of philanthropic resources
Thank you for the kind words—and it is always nice to get follow-up questions!
Further reading
In terms of recommended further reading, almost all UK-based Health Economists swear by ‘the Briggs book’. This contains step-by-step instructions for doing almost everything I describe above, as well as more detail around motivation and assumptions.
If you don’t want to shell out for a textbook, an excellent exploration of uncertainty is Claxton et al 2015 where the authors demonstrated that the value of additional information on the uncertainty of streptokinase following heart attack was so small as to be negligible, which implies that a major shift in health policy could have been undertaken five years earlier and in the absence of several massive expensive trials. Claxton is one of the co-authors of the Briggs book, so knows his stuff inside out.
In terms of EA specific follow-ups, I have always really loved Kwakkel & Pruyt 2013 for their use of uncertainty analysis in a framework that EAs would recognise as longtermist. Their first example is on mineral scarcity in the medium-term future, and they go through a process very similar to that which is done for x-risk type calculations, but with what I regard as a significantly higher degree of rigour and transparency. If someone asked me to model out AI alignment scenarios I would follow this paper almost to the letter, although I would warn anyone casually clicking through that this is pretty hardcore stuff that you can’t just knock together in Excel (see their Fig 1, for example).
I note you also ask for the most speculative use of uncertainty analysis, for which I have a rather interesting answer. I remember once reading a paper on the use of Monte Carlo modelling of parameter uncertainty to resolve the Fermi Paradox (that is, why has no alien intelligence contacted us if the universe is so vast). The paper really entertained me, but I completely forgot the reference until I tracked the paper down to link it for you now—it is Sandberg, Drexler & Ord 2018, and the ‘Ord’ in the third author position is Toby Ord, who I suspect is better known to forum members as one of the founders of EA—what a lovely coincidence!
Model covariance
You are right to raise covariance in Monte Carlo simulations as a clear issue with the way I have presented the topic, but you’ll be pleased to know that this is basically a solved problem in Health Economics which I just skimmed over in the interests of time. The ‘textbook’ method of solving the problem is to use a ‘Cholesky Decomposition’ on the covariance matrix and sample from that. In recent years I’ve also started experimenting with microsimulating the underlying process which generates the correlated results, with some mixed success (but it is cool when it works!).
Risk adjustment
Your comments on risk adjustment are completely correct—amongst many of the problems my approach causes it takes unlikely outcomes (ie high standard deviation away from average) and implicitly turns them into outcomes which are proportionally even more unlikely, sometimes to the point of requiring completely impossible inputs to generate those outputs. I hope I caveated the weakness of the method appropriately, because it isn’t a good model of how humans approach risk (more of a proof of concept)
There is a fairly novel method just breaking into the Health Economics literature called a CERAC, which uses the process you outline of treating a model as a portfolio with an expected return and downside risk of those returns being penalised accordingly. I suspect something like this is the best way to handle risk adjustment in a model without an explicit model of risk-preference specified across all possible outcomes. Unfortunately to use the technique as described you need a cost-effectiveness threshold, which doesn’t exist in EA (and will never exist in EA as a matter of first-principles). As I mentioned, I work in an exclusively expected utility context so I’m not familiar enough with the technique to be confident of adapting it to EA, although if someone with a better maths background than me wanted to give it a shot I suspect that would be a pretty valuable extension of the general principle I outline.
Methods for improving uncertainty analysis in EA cost-effectiveness models
This looks like you are probably right—I’m really sorry I haven’t had time to go through both sets of calculations in detail but they do seem to be getting at the same sort of adjustment so that’s an excellent spot. I suppose regardless, if it isn’t obvious to a user that the same adjustment is being made to two different numbers then this is reason enough to make it more explicit!
Please do use the Refactor however you would like, although please also add enough of a disclaimer to keep the context from this essay that the Refactor is an academic exercise which tries to stay roughly true to GiveWell’s design philosophy and assumptions while improving architecture and calculation conventions. I don’t think as a standalone piece of modelling it is the cutting edge of what EA could aspire to, whereas Nolan / McGuire really do seem to be operating at that level.
But yes, really happy to contribute if I can be of any help—drop me a message!
One of the key takeaways in the body of the text which perhaps I should have brought out more in the summary is that the GiveWell model is basically as reliable as highly professionalised bodies like pharma companies have figured out how to make a cost-effectiveness model. A small number of minor errors are unexceptional for a model of this complexity, even models that we submit to pharma regulators that have had several million dollars of development behind them.
I would say that while the errors are uninteresting and unexceptional, the unusual model design decisions are worth commenting on. The GiveWell team are admirably transparent with their model, and anybody who wants to review it can have access to almost everything at the click of a button (some assumptions are gated to GiveWell staff, but these aren’t central). Given this, it is remarkable the EA community didn’t manage to surface anyone who knew enough about models to flag to GiveWell that there were optimisations in design to be made—the essay above is not really arcane modelling lore but rather something anyone with a few years’ experience in pharma HEOR could have told you. Is this because there are too few quant actors in the EA space? Is it because they don’t think their contributions would be valued so don’t speak up? Is it because criticism of GiveWell makes you unemployable in EA spaces so is heavily incentivised against? Etc etc. That is to say—I think asking why GiveWell missed the improvements is missing the important point, which is that everyone missed these improvements so there’s probably changes that can be made to expert knowledge synthesis in EA right across the board.
Just to add that I think outreach efforts like the Red Team contest are a really good way of doing this—I wouldn’t have heard about the EA Forums had it not been for the plug Scott Alexander gave the contest on Astral Codex Ten (which I read mostly for the stuff on prediction markets).
Ah sorry, I think I might have confused the issue a bit with my footnote. I think I’ve managed to conflate two issues in your mind.
The first is exactly as you say; any intervention worth doing has some effects which are easy to model and some which are difficult (maybe impossible) to model. What GiveWell has done is completely reasonable here; modelling what it can and then making assumptions about how important the other things, like track record, are in comparison to the main cost-effectiveness results.
The second issue is the more subtle one that I was driving at. Imagine you are going to buy a new car, and your friend (who knows about cars) says that modern cars are 10x more fuel efficient than the car you currently drive. Speaking very roughly, there are two strategies you could pick from to choose your next car:
Completely ignore your friend, and pick the car that has the best MPG regardless of any other feature. This would be a good strategy if literally all you care about is fuel efficiency, but a bad strategy otherwise (because it is unlikely the most fuel efficient car is also the most comfortable to drive—especially if fuel efficiency and comfort are sort-of tradeoffs)
Treat your friend as having offered a useful rule of thumb, and so have an idea in your head about what ‘good’ fuel efficiency looks like. This is a good strategy if cars aren’t really directly comparable along a straightforward scale—a Ford F-150 isn’t ‘better’ or ‘worse’ than a Prius, it is just a different kind of thing.
Both GiveWell (implicitly) and me in my fertility days (explicitly) argue that QALYs are like cars—you can end up in a situation where you can generate different kinds of QALYs and your best bet is to compare them with a rule of thumb like GiveWell’s 10x multiplier. However I don’t think GiveWell is correct in making this assumption about charities—there is in fact a single measure like MPG which we want to ruthlessly optimise, and therefore we do actually want to it the F-150 and Prius directly against each other.
However my point in the essay is that GiveWell don’t actually have to choose—they can build their model as if they are in the first world and directly compare charities together, and then make their final decision as though they are in the second world and different charities will offer different profiles of benefit on top of their cost-effectiveness. This is pretty much the commonsense way of choosing a car too—you would look at MPG and directly compare cars in this way, but you might then consider other factors. It would be weird to lump all cars together in your head as ‘better than 10x my previous efficiency’ or ‘worse than 10x my previous efficiency’.
A critical review of GiveWell’s 2022 cost-effectiveness model
Thank you—really helpful additional information and very useful to have it confirmed that GiveWell are considered high quality models by the EA community. Really appreciate it.
Thank you—these are really helpful to help me understand. On the Sam Nolan piece especially quantifying uncertainty was one of the biggest critiques I had of the GiveWell model so I’m glad this has already been considered!
Thank you so much for the links—the Manheim work in this particular one is absolutely spectacular