I think working this through on guesttimate rather than mulitplying point estimates is really important.
I tried doing it myself with similar figures, and I found the climate change came out ~80x better than global health (even though my point estimate that that global health is better) - which suggests the title of the article could maybe use editing!
When you’re dealing with huge uncertainties like these, the tails of the distribution can drive the EV, so point estimates can be pretty misleading.
Rob’s point that by multiplying together extreme values, your confidence intervals are unreasonably wide.
Some of the confidence intervals you give for the individual parameters also seem too wide (and seem to not be mathematically possible to fit to a lognormal distribution).
I think I’ll just leave the title for now, because it is confusing as it is and I’m not sure if it’s worth it to redo/rewrite the analysis. I should probably have just called it “How to compare the relative effectiveness of development vs. climate interventions”. I’ll make a note in the beginning of the post linking to your guesstimate, saying that you found different results.
I can’t quite follow your analysis from the screenshots (perhaps you could link the models and the assumptions for others). For instance, I’m not sure why the input value of money going to Americans vs. GiveDirectly recipients is 23 to 350.
But generally, I agree that Monte Carlo simulations and minding the distributions can be valuable for better error propagation. Also, I was probably being unclear but my analysis was not supposed to be a confidence intervals but rather my the best guess and extreme scenarios.
Echoing what Greg Lewis said about hobbyists modelling the C19 pandemic being perhaps not super productive, I’m also not sure how productive further empirical work such as this is on the EA forum (I don’t even know how many hits the forum gets generally, and this post in particular, how many climate modellers read it, etc.). I think maybe an org with more research capacity would be better suited to do further analysis on this. Or perhaps one could commission researchers with a background in climate modelling to do this (e.g. the author of this paper might be really qualified to do this: https://www.sciencedirect.com/science/article/pii/S014098831930218X ).
I do think more EA work on this topic would be useful for someone to do, since I don’t think it’s clear from a near-termist perspective that global health is more effective than climate change.
On guesstimate, there was an error and I was unable to save my model. If someone is looking to reproduce this though, I’d suggest they just make their own.
On the value of money to Americans vs. GiveDirectly recipients, my personal estimate was a lower ratio, because I think we should take into account some flow through effects and I think this causes convergence. I don’t think values like 10,000x are plausible for the all-considered tradeoff (even though the ratio could be 10,000x if we’re just considering the welfare of two individuals).
More here:
http://reflectivedisequilibrium.blogspot.com/2014/01/what-portion-of-boost-to-global-gdp.html
I was probably being unclear but my analysis was not supposed to be a confidence intervals but rather my the best guess and extreme scenarios.
I’m still a bit unclear how useful these are due to Rob’s point.
I think working this through on guesttimate rather than mulitplying point estimates is really important.
I tried doing it myself with similar figures, and I found the climate change came out ~80x better than global health (even though my point estimate that that global health is better) - which suggests the title of the article could maybe use editing!
When you’re dealing with huge uncertainties like these, the tails of the distribution can drive the EV, so point estimates can be pretty misleading.
Here’s a screenshot of the model: https://www.dropbox.com/s/adtwlz3k2myv8gc/Screenshot 2020-05-25 20.57.11.png?dl=0
I also tried doing the calculations in a different way that I found more intuitive—where I estimate the ‘utils’ of each intervention: https://www.dropbox.com/s/8uczqc1qhi71lte/Screenshot 2020-05-25 20.58.02.png?dl=0
Some other reasons in favour of this approach:
Rob’s point that by multiplying together extreme values, your confidence intervals are unreasonably wide.
Some of the confidence intervals you give for the individual parameters also seem too wide (and seem to not be mathematically possible to fit to a lognormal distribution).
Thanks for the comment!
I think I’ll just leave the title for now, because it is confusing as it is and I’m not sure if it’s worth it to redo/rewrite the analysis. I should probably have just called it “How to compare the relative effectiveness of development vs. climate interventions”. I’ll make a note in the beginning of the post linking to your guesstimate, saying that you found different results.
I can’t quite follow your analysis from the screenshots (perhaps you could link the models and the assumptions for others). For instance, I’m not sure why the input value of money going to Americans vs. GiveDirectly recipients is 23 to 350.
But generally, I agree that Monte Carlo simulations and minding the distributions can be valuable for better error propagation. Also, I was probably being unclear but my analysis was not supposed to be a confidence intervals but rather my the best guess and extreme scenarios.
For instance, in the cell for cost per tonne of CO2 averted in the pessimistic scenario I intentionally picked the extreme value from the Founder Pledge analysis $0.02 and not their mean value (from the cell note: “A donation to CfRN will avert a tonne of CO2e for $0.12, with a plausible range of $0.02 - $0.72.” https://docs.google.com/spreadsheets/d/12lwvxlWLjwuSuXiciFvnBF2bkfcCkrusdqqT37_QWac/edit#gid=1267972809&range=E35 ).
Echoing what Greg Lewis said about hobbyists modelling the C19 pandemic being perhaps not super productive, I’m also not sure how productive further
empirical work such as this is on the EA forum (I don’t even know how many hits the forum gets generally, and this post in particular, how many climate modellers read it, etc.). I think maybe an org with more research capacity would be better suited to do further analysis on this. Or perhaps one could commission researchers with a background in climate modelling to do this (e.g. the author of this paper might be really qualified to do this: https://www.sciencedirect.com/science/article/pii/S014098831930218X ).
Hey Hauke,
That makes sense.
I do think more EA work on this topic would be useful for someone to do, since I don’t think it’s clear from a near-termist perspective that global health is more effective than climate change.
On guesstimate, there was an error and I was unable to save my model. If someone is looking to reproduce this though, I’d suggest they just make their own.
On the value of money to Americans vs. GiveDirectly recipients, my personal estimate was a lower ratio, because I think we should take into account some flow through effects and I think this causes convergence. I don’t think values like 10,000x are plausible for the all-considered tradeoff (even though the ratio could be 10,000x if we’re just considering the welfare of two individuals). More here: http://reflectivedisequilibrium.blogspot.com/2014/01/what-portion-of-boost-to-global-gdp.html
I’m still a bit unclear how useful these are due to Rob’s point.