Our prior strongly punishes MIRI. While the mean of its evidence distribution is 2,053,690,000 HEWALYs/$10,000, the posterior mean is only 180.8 HEWALYs/$10,000. If we set the prior scale parameter to larger than about 1.09, the posterior estimate for MIRI is greater than 1038 HEWALYs/$10,000, thus beating 80,000 Hours.
This suggests that it might be good in the long run to have a process that learns what prior is appropriate, e.g. by going back and seeing what prior would have best predicted previous years’ impact.
My personal take on the issue is that, the better we understand how the updating works (including how to select the prior), the more seriously we should take the results. Currently we don’t seem to have a good understanding (e.g. see Dickens’ discussion: the way of selecting the median based on Give Directly seems reasonable, but there doesn’t seem to be a principled way of selecting the variance, and this seems to be the best effort at it so far), so these updating exercises can be used as heuristics but the results are not to be taken too seriously, and certainly not literally (together with the reason that input values are so speculative in some cases).
This is just my personal view and certainly many people disagree. E.g. my team decided to use the results of Bayesian updating to decide on the grant recipient.
My experience with the project lead me to be not very positive that it’s worth investing too much in improving this quantitative approach for the sake of decision making, if one could instead spend time on gathering qualitative information (or even quantitative information that don’t fit neatly in the framework of cost-effectiveness calculations or updating) that could be much more informative for decision making. This is along the lines of this post and seems to also fit the current approach of the Open Philanthropy Project (of utilizing qualitative evidence rather than relying on quantitative estimates). Of course this is all based on the current state of such quantitative modeling, e.g. how little we understand how updating works as well as how to select speculative inputs for the quantitative models (and my judgment about how hard it would be to try to improve on these fronts). There could be a drastically better version of such quantitative prioritization that I haven’t been able to imagine.
It could be very valuable to construct a quantitative model (or parts of one), think about the inputs and their values, etc., for reasons explained here. E.g. The MIRI model (in particular some inputs by Paul Christiano; see here) has really helped me realize the importance of AI safety. So does the “astronomical waste” argument, which gives one a sense of the scale even if one doesn’t take the numbers literally. Still, when I make a decision of whether to donate to MIRI I wouldn’t rely on a quantitative model (at least one like what I built) and would instead put a lot of weight on qualitative evidence that is likely impossible (for us yet) to model quantitatively.
I’m still undecided on the question of whether quantitative models can actually work better than qualitative analysis. (Indeed, how can you even ever know which works better?) But very few people actually use serious quantitative models to make decisions—even if quantitative models ultimately don’t work as well as well-organized qualitative analysis, they’re still underrepresented—so I’m happy to see more work in this area.
Some suggestions on ways to improve the model:
Account for missing components
Quantitative models are hard, and it’s impossible to construct a model that accounts for everything you care about. I think it’s a good idea to consider which parts of reality you expect to matter most for the impact of a particular thing, and try to model those. Whatever your model is missing, try to figure out which parts of that matter most. You might decide that some things are too hard to model, in which case you should consider how those hard-to-model bits will likely affect the outcome and adjust your decision accordingly.
Examples of major things left out:
80K model only considers impact in terms of new donations to GWWC based on 80K’s own numbers. It would be better to use your own models of the effectiveness of different cause areas and account for how many people 80K moves into/away from these cause areas using your own effectiveness estimates for different causes.
ACE model only looks at the value from moving money among top charities. My own model includes money moved among top charities plus new money moved to top charities plus the value of new research that ACE funds.
Sensitivity analysis
The particular ordering you found (80K > MIRI > ACE > StrongMinds) depends heavily on certain input parameters. For example, for your MIRI model, “expected value of the far future” is doing tons of work. It assumes that the far future contains about 10^17 person-years; I don’t see any justification given. What if it’s actually 10^11? Or 10^50? This hugely changes the outcome. You should do some sensitivity analysis to see which inputs matter the most. If any one input matters too much, break it down into less sensitive inputs.
Suppose it’s 10 years in the future, and we can look back at what ACE and MIRI have been doing for the past 10 years. We now know some new useful information, such as:
Has ACE produced research that influenced our understanding of effective charities?
Has MIRI published new research that moved us closer to making AI safe?
Has ACE moved more money to top animal charities?
But even then, we still don’t know nearly as much as we’d like. We don’t know if ACE really moved money, or if that money would have been donated to animal charities anyway. Maybe MIRI took funding away from other research avenues that would have been more fruitful. We still have no idea how (dis)valuable the far future will be.
If you disagree with our prior parameters, we encourage you to try our own values and see what you come up with, in the style of GiveWell, who provide their parameters as estimated by each staff member.
Do you have these numbers published, broken down by staff member?
It also would be cool to see breakdowns of the HEWALYs/$ for each charity before and after the Bayesian update with the prior.
This suggests that it might be good in the long run to have a process that learns what prior is appropriate, e.g. by going back and seeing what prior would have best predicted previous years’ impact.
My personal take on the issue is that, the better we understand how the updating works (including how to select the prior), the more seriously we should take the results. Currently we don’t seem to have a good understanding (e.g. see Dickens’ discussion: the way of selecting the median based on Give Directly seems reasonable, but there doesn’t seem to be a principled way of selecting the variance, and this seems to be the best effort at it so far), so these updating exercises can be used as heuristics but the results are not to be taken too seriously, and certainly not literally (together with the reason that input values are so speculative in some cases).
This is just my personal view and certainly many people disagree. E.g. my team decided to use the results of Bayesian updating to decide on the grant recipient.
My experience with the project lead me to be not very positive that it’s worth investing too much in improving this quantitative approach for the sake of decision making, if one could instead spend time on gathering qualitative information (or even quantitative information that don’t fit neatly in the framework of cost-effectiveness calculations or updating) that could be much more informative for decision making. This is along the lines of this post and seems to also fit the current approach of the Open Philanthropy Project (of utilizing qualitative evidence rather than relying on quantitative estimates). Of course this is all based on the current state of such quantitative modeling, e.g. how little we understand how updating works as well as how to select speculative inputs for the quantitative models (and my judgment about how hard it would be to try to improve on these fronts). There could be a drastically better version of such quantitative prioritization that I haven’t been able to imagine.
It could be very valuable to construct a quantitative model (or parts of one), think about the inputs and their values, etc., for reasons explained here. E.g. The MIRI model (in particular some inputs by Paul Christiano; see here) has really helped me realize the importance of AI safety. So does the “astronomical waste” argument, which gives one a sense of the scale even if one doesn’t take the numbers literally. Still, when I make a decision of whether to donate to MIRI I wouldn’t rely on a quantitative model (at least one like what I built) and would instead put a lot of weight on qualitative evidence that is likely impossible (for us yet) to model quantitatively.
I’m still undecided on the question of whether quantitative models can actually work better than qualitative analysis. (Indeed, how can you even ever know which works better?) But very few people actually use serious quantitative models to make decisions—even if quantitative models ultimately don’t work as well as well-organized qualitative analysis, they’re still underrepresented—so I’m happy to see more work in this area.
Some suggestions on ways to improve the model:
Account for missing components
Quantitative models are hard, and it’s impossible to construct a model that accounts for everything you care about. I think it’s a good idea to consider which parts of reality you expect to matter most for the impact of a particular thing, and try to model those. Whatever your model is missing, try to figure out which parts of that matter most. You might decide that some things are too hard to model, in which case you should consider how those hard-to-model bits will likely affect the outcome and adjust your decision accordingly.
Examples of major things left out:
80K model only considers impact in terms of new donations to GWWC based on 80K’s own numbers. It would be better to use your own models of the effectiveness of different cause areas and account for how many people 80K moves into/away from these cause areas using your own effectiveness estimates for different causes.
ACE model only looks at the value from moving money among top charities. My own model includes money moved among top charities plus new money moved to top charities plus the value of new research that ACE funds.
Sensitivity analysis
The particular ordering you found (80K > MIRI > ACE > StrongMinds) depends heavily on certain input parameters. For example, for your MIRI model, “expected value of the far future” is doing tons of work. It assumes that the far future contains about 10^17 person-years; I don’t see any justification given. What if it’s actually 10^11? Or 10^50? This hugely changes the outcome. You should do some sensitivity analysis to see which inputs matter the most. If any one input matters too much, break it down into less sensitive inputs.
Retrospective analysis of track record? Looking into Tetlock-style research?
Suppose it’s 10 years in the future, and we can look back at what ACE and MIRI have been doing for the past 10 years. We now know some new useful information, such as:
Has ACE produced research that influenced our understanding of effective charities?
Has MIRI published new research that moved us closer to making AI safe?
Has ACE moved more money to top animal charities?
But even then, we still don’t know nearly as much as we’d like. We don’t know if ACE really moved money, or if that money would have been donated to animal charities anyway. Maybe MIRI took funding away from other research avenues that would have been more fruitful. We still have no idea how (dis)valuable the far future will be.
How do you do the “human equivalent” part?
Do you have these numbers published, broken down by staff member?
It also would be cool to see breakdowns of the HEWALYs/$ for each charity before and after the Bayesian update with the prior.
Found it, thanks!
How do you edit the model code?