On 2) I think this shouldn’t be a barrier, since the first step in impact evaluation is establishing a theory of change, and any individual organisation of a given maturity should have a clear picture of this, and consensus on their goals. (I’m not advocating for a singular ToC across the whole EA movement, just for individual organisations)
On 3) I agree this is a factor for evaluations for funders. But I think organisations should carry out internal impact evaluation according to their theory of change, so I think should still be able to have clear and established goals. They can say to funders “Our Theory of Change is X, and along those lines our impact has been Y”. Funders might also carry out their own evaluations in terms of the goals they most care about.
On 4), I agree there’s a trade-off between spending on direct activities and impact evaluation. Of course the value of impact evaluation is that it may improve the effectiveness of the direct activities. So I don’t think the existence of the trade-off is in itself a reason that impact evaluation shouldn’t be prevalent.
Thanks a lot Robert!
On 1) great point, I agree.
On 2) I think this shouldn’t be a barrier, since the first step in impact evaluation is establishing a theory of change, and any individual organisation of a given maturity should have a clear picture of this, and consensus on their goals. (I’m not advocating for a singular ToC across the whole EA movement, just for individual organisations)
On 3) I agree this is a factor for evaluations for funders. But I think organisations should carry out internal impact evaluation according to their theory of change, so I think should still be able to have clear and established goals. They can say to funders “Our Theory of Change is X, and along those lines our impact has been Y”. Funders might also carry out their own evaluations in terms of the goals they most care about.
On 4), I agree there’s a trade-off between spending on direct activities and impact evaluation. Of course the value of impact evaluation is that it may improve the effectiveness of the direct activities. So I don’t think the existence of the trade-off is in itself a reason that impact evaluation shouldn’t be prevalent.