Thanks for putting this together! I think criticizing funders is quite valuable, and I commend your doing so. My main object-level thought here is I suspect much of the disagreement with the OP funding decision is based around the 1%-10% estimate of a $2B-$20B campaign leading to a 25%-75% decrease in incarceration. Since, per this article, incarceration rates in the U.S. have declined 20% (per person) between 2008 and 2019, your estimates here seem somewhat pessimistic to me.
My guess is at the outset, OP would have predicted a different order of magnitude for both of those numbers (so I would have estimated something closer to $500M-$5B would produce a 5%-50% chance of a 25%-75% decrease), particularly since (as has been mentioned in other comments) it seemed a particularly tractable moment for criminal justice reform (since crime had declined for a long-time, there was seeming bipartisan support for reform, costs of incarceration were high, and U.S. was so far outside the global norm). By my quick read of the math, that change would put the numbers on par with your estimates for global health stuff.
As someone who’s worked on criminal justice reform (on a volunteer basis, not OP-funded though inspired by OP-funded work like Just Leadership’s Close Rikers campaign), two features of the field that are striking to me are: 1. OP’s original vision was to reduce incarceration while also reducing crime—I don’t think the “reduce crime” half ended up being a main goal of the work, which I think has probably made it less politically robust. 2. A lot of criminal justice reform work (including mine) has stemmed from the thesis that empowering the voices of current and formerly incarcerated people politically would be politically beneficial; I think in retrospect this may have been mistaken (or at least an incomplete hypothesis) and that, more broadly, left identity-politics based strategies of the 2010s have not been as politically (especially electorally) successful as I at least had hoped.
On a meta-level, I think estimating impact of past grantmaking is very important, and EAs should do more of it. (I also think something along these lines could theoretically provide a scalable internship program for EA college students, since estimating impact teaches both cause prioritization skills and skills understanding the operations of organizations trying to achieve EA goals).
Edit: I should clarify that I’ve received significant funding from OP (including from their US Policy side that covered their criminal justice work), so I’m naturally biased in its favor
Thanks Josh, I particularly appreciate your quantified estimates of likelihood/impact.
per this article, incarceration rates in the U.S. have declined 20% (per person) between 2008 and 2019, your estimates here seem somewhat pessimistic to me.
Not sure how that follows; what would be needed for counterfactual/shapley impact would be a further reduction in the absence/reduction of funding. If OP donates $5B and the imprisonment rate goes down another 20%, but would have gone down a 20% (resp 15%) anyways, the impact is 0 (resp 5%).
Yeah it’s unclear how much of the 20% reduction is due to OP’s work or would happen counterfactually. My main point with that number is that reductions of that size are very possible, which implies assuming a 1-10% chance of that level of impact at a funding level 10-100x OP’s amount is overly conservative (particularly since I think OP was funding like 25% of American CJR work—though that number may be a bit off).
Another quick back of the envelope way to do the math would be to say something like: assume 1. 50% of policy change is due to deliberate advocacy, 2. OP’s a funder of average ability that is funding 25% of the field, 3. the 20% 2009-2018 change implies a further 20% change is 50% likely at their level of funding, then I think you get 6.25% (.5*.25*.5) odds of OP’s $25M a year funding level achieving a 20% change in incarceration rates. If I’m looking at your math right (and sorry if not) a 20% reduction for 10 years would be worth like (using point estimates) 4M QALYs (2M people *10 years *1 QALY*20% decrease), which I think would come out to 250K QALYs in expectation (6.25%*4M), which at $25M/year for 10 years would be 1K/QALY—similar to your estimate for GiveDirectly but worse than AMF. (Sorry if any of that math is wrong—did it quickly and haphazardly)
Thanks for putting this together! I think criticizing funders is quite valuable, and I commend your doing so. My main object-level thought here is I suspect much of the disagreement with the OP funding decision is based around the 1%-10% estimate of a $2B-$20B campaign leading to a 25%-75% decrease in incarceration. Since, per this article, incarceration rates in the U.S. have declined 20% (per person) between 2008 and 2019, your estimates here seem somewhat pessimistic to me.
My guess is at the outset, OP would have predicted a different order of magnitude for both of those numbers (so I would have estimated something closer to $500M-$5B would produce a 5%-50% chance of a 25%-75% decrease), particularly since (as has been mentioned in other comments) it seemed a particularly tractable moment for criminal justice reform (since crime had declined for a long-time, there was seeming bipartisan support for reform, costs of incarceration were high, and U.S. was so far outside the global norm). By my quick read of the math, that change would put the numbers on par with your estimates for global health stuff.
As someone who’s worked on criminal justice reform (on a volunteer basis, not OP-funded though inspired by OP-funded work like Just Leadership’s Close Rikers campaign), two features of the field that are striking to me are: 1. OP’s original vision was to reduce incarceration while also reducing crime—I don’t think the “reduce crime” half ended up being a main goal of the work, which I think has probably made it less politically robust. 2. A lot of criminal justice reform work (including mine) has stemmed from the thesis that empowering the voices of current and formerly incarcerated people politically would be politically beneficial; I think in retrospect this may have been mistaken (or at least an incomplete hypothesis) and that, more broadly, left identity-politics based strategies of the 2010s have not been as politically (especially electorally) successful as I at least had hoped.
On a meta-level, I think estimating impact of past grantmaking is very important, and EAs should do more of it. (I also think something along these lines could theoretically provide a scalable internship program for EA college students, since estimating impact teaches both cause prioritization skills and skills understanding the operations of organizations trying to achieve EA goals).
Edit: I should clarify that I’ve received significant funding from OP (including from their US Policy side that covered their criminal justice work), so I’m naturally biased in its favor
Thanks Josh, I particularly appreciate your quantified estimates of likelihood/impact.
Not sure how that follows; what would be needed for counterfactual/shapley impact would be a further reduction in the absence/reduction of funding. If OP donates $5B and the imprisonment rate goes down another 20%, but would have gone down a 20% (resp 15%) anyways, the impact is 0 (resp 5%).
Yeah it’s unclear how much of the 20% reduction is due to OP’s work or would happen counterfactually. My main point with that number is that reductions of that size are very possible, which implies assuming a 1-10% chance of that level of impact at a funding level 10-100x OP’s amount is overly conservative (particularly since I think OP was funding like 25% of American CJR work—though that number may be a bit off).
Another quick back of the envelope way to do the math would be to say something like: assume 1. 50% of policy change is due to deliberate advocacy, 2. OP’s a funder of average ability that is funding 25% of the field, 3. the 20% 2009-2018 change implies a further 20% change is 50% likely at their level of funding, then I think you get 6.25% (.5*.25*.5) odds of OP’s $25M a year funding level achieving a 20% change in incarceration rates. If I’m looking at your math right (and sorry if not) a 20% reduction for 10 years would be worth like (using point estimates) 4M QALYs (2M people *10 years *1 QALY*20% decrease), which I think would come out to 250K QALYs in expectation (6.25%*4M), which at $25M/year for 10 years would be 1K/QALY—similar to your estimate for GiveDirectly but worse than AMF. (Sorry if any of that math is wrong—did it quickly and haphazardly)