If it’s average future that still could justify a 1x bar, depending on what we’re averaging over.
I agree with the concerns about uncertainty, displacing less-effective charities, and counterfactuality. But I’d rather see attempts to adjust the estimate for that rather than ~”we’re saying 6x but not really, probably lower after considering this”. This will help avoid temptations towards soldier/promotion mentality, and make it more comparable to other estimates.
(RE “opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc”—if EA people are putting in free labor into these efforts, that should also be factored into the cost estimates, naturally, not just the direct CG investment.)
If it’s average future that still could justify a 1x bar, depending on what we’re averaging over.
I don’t think it does. It’s conceptually coherent for an organization to have a very high average cost-effectiveness while also having a marginal cost-effectiveness below 1x. For this reason, I don’t think you should have a “bar” for average cost-effectiveness. (You might be making the point that if the average cost-effectiveness is above 1, then you are better off making the grant than burning the money, and so it clears a bar in that sense, but it’s not clear it’s worth making the grant vs making a potentially much smaller grant, and so it’s not a helpful ‘bar’ in the sense the term is usually used.)
I agree with the concerns about uncertainty, displacing less-effective charities, and counterfactuality. But I’d rather see attempts to adjust the estimate for that rather than ~”we’re saying 6x but not really, probably lower after considering this”. This will help avoid temptations towards soldier/promotion mentality, and make it more comparable to other estimates.
Sure, but these are hard to account for. I agree it’s better to adjust the model when it’s possible, but you’ll still be left with a model that has a tonne of uncertainty.
(RE “opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc”—if EA people are putting in free labor into these efforts, that should also be factored into the cost estimates, naturally, not just the direct CG investment.)
Yep! I wasn’t trying to suggest you shouldn’t account for that.
I want to make sure we’re talking about the same thing here. I’d be want to know the cost-effectiveness in terms “for each $1 we spend to promote giving” (via starting new orgs or doing more fundraising) “how much do we raise in truly counterfactual donations to the most effective charities” and I’d want this to be net of any donations or effective work that might be crowded out.
E.g., suppose Joe lives in the USA and earns $100k per year. Without our spending Joe, would not give anything to charity and would also not be doing socially-useful work. We spend $1 on ads and this causes Joe (a rich guy) to give $1.50 to The Humane League or The Malaria Consortium, without affecting anyone else’s behavior. From the PoV of ~”the EA community” we have earned our $1 back plus gained an additional 50 cents. Again from the global EA community perspective, wouldn’t we always want to do this?
The example you gave is about marginal cost-effectiveness (we spend “$1 on ads”). I agree that then, in this abstract/idealized case, you should spend the $1 on ads. I think all the uncertainty you would realistically have makes it less obvious, though.
But average cost-effectiveness would be more like, we spent $1,000,000 on an organization that did a bunch of different activities, and we think that led to $1,500,000 counterfactually going to charity. This seems good on average, but there’s a further question of whether we should give another $1 to the organization. And I think that the 6x figure of the orignal post is referring to average cost-effectiveness (“our current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolio”). This is at least conceptually coherent with the bar for the marginal $ being closer to 1x.
I think you might find the GWWC impact evals interesting, they go into an enormous amount of depth on all these issues.
Okay, thank you. I aim to take a look at these evals and hopefully learn something and maybe give some useful feedback.
And one more point which maybe is obvious but just to get it out there.
Sure, but these are hard to account for. I agree it’s better to adjust the model when it’s possible, but you’ll still be left with a model that has a tonne of uncertainty.
I agree that a large amount of uncertainty will persist, but I suppose we should aim to do the modeling and adjustments is mean zero. E.g., we’d put in a large adjustment for ‘potential non-counterfactuality’ for things like “maybe the people who pledged would have pledged later on anyways and the fact that they pledged and donated now means that they’re likely to end their pledges earlier.”
I suspect that the impact evaluations indeed consider things like these, and I am looking forward to going over them when I have a moment. Thanks for engaging.
If it’s average future that still could justify a 1x bar, depending on what we’re averaging over.
I agree with the concerns about uncertainty, displacing less-effective charities, and counterfactuality. But I’d rather see attempts to adjust the estimate for that rather than ~”we’re saying 6x but not really, probably lower after considering this”. This will help avoid temptations towards soldier/promotion mentality, and make it more comparable to other estimates.
(RE “opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc”—if EA people are putting in free labor into these efforts, that should also be factored into the cost estimates, naturally, not just the direct CG investment.)
I don’t think it does. It’s conceptually coherent for an organization to have a very high average cost-effectiveness while also having a marginal cost-effectiveness below 1x. For this reason, I don’t think you should have a “bar” for average cost-effectiveness. (You might be making the point that if the average cost-effectiveness is above 1, then you are better off making the grant than burning the money, and so it clears a bar in that sense, but it’s not clear it’s worth making the grant vs making a potentially much smaller grant, and so it’s not a helpful ‘bar’ in the sense the term is usually used.)
Sure, but these are hard to account for. I agree it’s better to adjust the model when it’s possible, but you’ll still be left with a model that has a tonne of uncertainty.
Yep! I wasn’t trying to suggest you shouldn’t account for that.
I want to make sure we’re talking about the same thing here. I’d be want to know the cost-effectiveness in terms “for each $1 we spend to promote giving” (via starting new orgs or doing more fundraising) “how much do we raise in truly counterfactual donations to the most effective charities” and I’d want this to be net of any donations or effective work that might be crowded out.
E.g., suppose Joe lives in the USA and earns $100k per year. Without our spending Joe, would not give anything to charity and would also not be doing socially-useful work. We spend $1 on ads and this causes Joe (a rich guy) to give $1.50 to The Humane League or The Malaria Consortium, without affecting anyone else’s behavior. From the PoV of ~”the EA community” we have earned our $1 back plus gained an additional 50 cents. Again from the global EA community perspective, wouldn’t we always want to do this?
The example you gave is about marginal cost-effectiveness (we spend “$1 on ads”). I agree that then, in this abstract/idealized case, you should spend the $1 on ads. I think all the uncertainty you would realistically have makes it less obvious, though.
But average cost-effectiveness would be more like, we spent $1,000,000 on an organization that did a bunch of different activities, and we think that led to $1,500,000 counterfactually going to charity. This seems good on average, but there’s a further question of whether we should give another $1 to the organization. And I think that the 6x figure of the orignal post is referring to average cost-effectiveness (“our current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolio”). This is at least conceptually coherent with the bar for the marginal $ being closer to 1x.
I think you might find the GWWC impact evals interesting, they go into an enormous amount of depth on all these issues.
Okay, thank you. I aim to take a look at these evals and hopefully learn something and maybe give some useful feedback.
And one more point which maybe is obvious but just to get it out there.
I agree that a large amount of uncertainty will persist, but I suppose we should aim to do the modeling and adjustments is mean zero. E.g., we’d put in a large adjustment for ‘potential non-counterfactuality’ for things like “maybe the people who pledged would have pledged later on anyways and the fact that they pledged and donated now means that they’re likely to end their pledges earlier.”
I suspect that the impact evaluations indeed consider things like these, and I am looking forward to going over them when I have a moment. Thanks for engaging.