To get estimates comparable to each other, GiveWell CEAs use many variables that are the same across different charities. These include moral variables such as the discount rate and the moral weights, as well as a number or adjustment factors.
Because each of our CEAs was built in its separate notebook. Variables that should have more correlated uncertainty do not correlate.
Correlated uncertainty driven by using the same parameters across models: I’d love to see this, I expect it will effect the VOI calculations.
Also could be a good gateway towards the more difficult correlated uncertainty: the correlation between parameters within a model. I expect some of these to be highly correlated, and this might matter a lot. In fact, in the fistula model, some of these might largely be different takes on the same uncertain underlying variable? Here one might need to report on the VOI for something like a ‘cluster of variables’?
But this is more challenging; I guess it would probably start with some calibrated judgment/educated guessing on the correlation parameter.
I think maybe the issue of correlated uncertainty is the biggest limitation to bringing uncertainty into the models. If the uncertainties are ‘super correlated in the same direction’ then the naive ‘worst case/best case’ for everything is closer to being correct. If the uncertainties are ‘super negatively correlated’ then we may need to worry less about uncertainty, or perhaps worry about the largest of the uncertainties.
Correlated uncertainty driven by using the same parameters across models: I’d love to see this, I expect it will effect the VOI calculations.
Also could be a good gateway towards the more difficult correlated uncertainty: the correlation between parameters within a model. I expect some of these to be highly correlated, and this might matter a lot. In fact, in the fistula model, some of these might largely be different takes on the same uncertain underlying variable? Here one might need to report on the VOI for something like a ‘cluster of variables’?
But this is more challenging; I guess it would probably start with some calibrated judgment/educated guessing on the correlation parameter.
I think maybe the issue of correlated uncertainty is the biggest limitation to bringing uncertainty into the models. If the uncertainties are ‘super correlated in the same direction’ then the naive ‘worst case/best case’ for everything is closer to being correct. If the uncertainties are ‘super negatively correlated’ then we may need to worry less about uncertainty, or perhaps worry about the largest of the uncertainties.