With regards to your grim picture of victory, yeah, estimates will span multiple orders of magnitude. But at the very least, we could cut off the paths that “the market”, or “the forecasting system” knows to be dead-ends from the start. Because people have a bunch of biases that better forecasting/evaluation systems could just make apparent in excruciating detail.
For example, consider Open Philanthropy’s spinning off of Just Impact, “[a]fter hundreds of grants totaling more than $130 million over six years” because “we think the top global aid charities recommended by GiveWell (which we used to be part of and remain closely affiliated with) present an opportunity to give away large amounts of money at higher cost-effectiveness than we can achieve in many programs, including CJR, that seek to benefit citizens of wealthy countries”.
I have some pet theories about what happened here, but I’m pretty sure that any decent forecasting/evaluation system would have seen this* coming a mile ahead.
*: this = criminal justice reform being less effective than global health.
With regards to your grim picture of victory, yeah, estimates will span multiple orders of magnitude. But at the very least, we could cut off the paths that “the market”, or “the forecasting system” knows to be dead-ends from the start. Because people have a bunch of biases that better forecasting/evaluation systems could just make apparent in excruciating detail.
For example, consider Open Philanthropy’s spinning off of Just Impact, “[a]fter hundreds of grants totaling more than $130 million over six years” because “we think the top global aid charities recommended by GiveWell (which we used to be part of and remain closely affiliated with) present an opportunity to give away large amounts of money at higher cost-effectiveness than we can achieve in many programs, including CJR, that seek to benefit citizens of wealthy countries”.
I have some pet theories about what happened here, but I’m pretty sure that any decent forecasting/evaluation system would have seen this* coming a mile ahead.
*: this = criminal justice reform being less effective than global health.