I agree with Ben Millwood’s comment that I don’t think this would change many decisions in practice.
To add another point, input parameter uncertainty is larger than you probably think, even for direct-delivery GHD charities (let alone policy or meta orgs). The post Quantifying Uncertainty in GiveWell Cost-Effectiveness Analyses visualises this point particularly vividly; you can see how a 10% change doesn’t really change prioritisation much:
Intervention
GiveWell
Our Mean
95% CI
Diference
Against Malaria Foundation
0.0375
0.0384
0.0234 − 0.0616
+2.4%
GiveDirectly
0.00335
0.00359
0.00167 − 0.00682
+7%
Helen Keller International
0.0541
0.0611
0.0465 − 0.0819
+12.8%
Malaria Consortium
0.031
0.0318
0.0196 − 0.0452
+2.52%
New Incentives
0.0458
0.0521
0.0139 − 0.117
13.8%
(Look at how large those 95% CIs are vs a 10% change.)
I think a useful way to go about this is to ask, what would have to change to alter the decisions (e.g. top-recommended charities, intervention ideas turned into incubated charities, etc)? This gets you into uncertainty analysis, to which I’d point you to froolow’s Methods for improving uncertainty analysis in EA cost-effectiveness models.
Upvoted :)
I agree with Ben Millwood’s comment that I don’t think this would change many decisions in practice.
To add another point, input parameter uncertainty is larger than you probably think, even for direct-delivery GHD charities (let alone policy or meta orgs). The post Quantifying Uncertainty in GiveWell Cost-Effectiveness Analyses visualises this point particularly vividly; you can see how a 10% change doesn’t really change prioritisation much:
(Look at how large those 95% CIs are vs a 10% change.)
I think a useful way to go about this is to ask, what would have to change to alter the decisions (e.g. top-recommended charities, intervention ideas turned into incubated charities, etc)? This gets you into uncertainty analysis, to which I’d point you to froolow’s Methods for improving uncertainty analysis in EA cost-effectiveness models.
@Mo Putera love the graph! Why is something being a small change a reason not to include it?
I agree, this wouldn’t change much probably, but this is a change that applies to a lot of CEAs and is in some way a straightforward and safe change?