The example you gave is about marginal cost-effectiveness (we spend â$1 on adsâ). I agree that then, in this abstract/âidealized case, you should spend the $1 on ads. I think all the uncertainty you would realistically have makes it less obvious, though.
But average cost-effectiveness would be more like, we spent $1,000,000 on an organization that did a bunch of different activities, and we think that led to $1,500,000 counterfactually going to charity. This seems good on average, but thereâs a further question of whether we should give another $1 to the organization. And I think that the 6x figure of the orignal post is referring to average cost-effectiveness (âour current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolioâ). This is at least conceptually coherent with the bar for the marginal $ being closer to 1x.
I think you might find the GWWC impact evals interesting, they go into an enormous amount of depth on all these issues.
Okay, thank you. I aim to take a look at these evals and hopefully learn something and maybe give some useful feedback.
And one more point which maybe is obvious but just to get it out there.
Sure, but these are hard to account for. I agree itâs better to adjust the model when itâs possible, but youâll still be left with a model that has a tonne of uncertainty.
I agree that a large amount of uncertainty will persist, but I suppose we should aim to do the modeling and adjustments is mean zero. E.g., weâd put in a large adjustment for âpotential non-counterfactualityâ for things like âmaybe the people who pledged would have pledged later on anyways and the fact that they pledged and donated now means that theyâre likely to end their pledges earlier.â
I suspect that the impact evaluations indeed consider things like these, and I am looking forward to going over them when I have a moment. Thanks for engaging.
The example you gave is about marginal cost-effectiveness (we spend â$1 on adsâ). I agree that then, in this abstract/âidealized case, you should spend the $1 on ads. I think all the uncertainty you would realistically have makes it less obvious, though.
But average cost-effectiveness would be more like, we spent $1,000,000 on an organization that did a bunch of different activities, and we think that led to $1,500,000 counterfactually going to charity. This seems good on average, but thereâs a further question of whether we should give another $1 to the organization. And I think that the 6x figure of the orignal post is referring to average cost-effectiveness (âour current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolioâ). This is at least conceptually coherent with the bar for the marginal $ being closer to 1x.
I think you might find the GWWC impact evals interesting, they go into an enormous amount of depth on all these issues.
Okay, thank you. I aim to take a look at these evals and hopefully learn something and maybe give some useful feedback.
And one more point which maybe is obvious but just to get it out there.
I agree that a large amount of uncertainty will persist, but I suppose we should aim to do the modeling and adjustments is mean zero. E.g., weâd put in a large adjustment for âpotential non-counterfactualityâ for things like âmaybe the people who pledged would have pledged later on anyways and the fact that they pledged and donated now means that theyâre likely to end their pledges earlier.â
I suspect that the impact evaluations indeed consider things like these, and I am looking forward to going over them when I have a moment. Thanks for engaging.