Andrew Gelman, who tends to get a lot of things right, has written a post about how they consistently fail at modeling and shrug off peer review on the basis of having so much Gates Foundation money. C19P, which has an excellent modelling record, has a more detailed critique of IHME’s failures.
I haven’t delved deep into this, but assuming it’s correct that IHME has badly bungled covid predictions and ignored corrections on the basis that they’re only accountable to Gates Foundation, that’s a serious problem for civilizational disease response. Can we say with a little more confidence whether this happened? If so, are there people among us who could bring the matter the the Gates Foundation’s attention? I would expect there are people in our circles a small number of trust-hops from relevant GF people.
If funding from the Gates Foundation is having the perverse effect of allowing them to ignore scientific criticism, it also sounds like an interesting case study in charitable spending/inventives gone wrong.
On the bright side, IHME isn’t our only or even our main source of pandemic predictions. The CDC tracks numerous covid prediction projects (although the CDC has generally done badly too, and is definitely not going to be winning any forecasting awards). In recognition of these failures, the CDC is creating a new forecasting center staffed by what seems like a promising crew—they will be led by Marc Lipsitch, a Harvard professor who has given talks at EA events and (IMO) offers very intelligent covid-19 commentary on twitter.
https://fortune.com/2021/08/18/cdc-center-forecasting-and-outbreak-analytics-public-health/
Overall, forecasting and data gathering is one of the few aspects of pandemic response where I’m optimistic that we’ve learned our lesson and will do better next time.
[Question] Is anyone worrying about the Institute for Health Metrics and Evaluation (IHME)?
Andrew Gelman, who tends to get a lot of things right, has written a post about how they consistently fail at modeling and shrug off peer review on the basis of having so much Gates Foundation money. C19P, which has an excellent modelling record, has a more detailed critique of IHME’s failures.
I haven’t delved deep into this, but assuming it’s correct that IHME has badly bungled covid predictions and ignored corrections on the basis that they’re only accountable to Gates Foundation, that’s a serious problem for civilizational disease response. Can we say with a little more confidence whether this happened? If so, are there people among us who could bring the matter the the Gates Foundation’s attention? I would expect there are people in our circles a small number of trust-hops from relevant GF people.
Is anybody already on this?
If funding from the Gates Foundation is having the perverse effect of allowing them to ignore scientific criticism, it also sounds like an interesting case study in charitable spending/inventives gone wrong.
On the bright side, IHME isn’t our only or even our main source of pandemic predictions. The CDC tracks numerous covid prediction projects (although the CDC has generally done badly too, and is definitely not going to be winning any forecasting awards). In recognition of these failures, the CDC is creating a new forecasting center staffed by what seems like a promising crew—they will be led by Marc Lipsitch, a Harvard professor who has given talks at EA events and (IMO) offers very intelligent covid-19 commentary on twitter. https://fortune.com/2021/08/18/cdc-center-forecasting-and-outbreak-analytics-public-health/
Overall, forecasting and data gathering is one of the few aspects of pandemic response where I’m optimistic that we’ve learned our lesson and will do better next time.