I agree independence has advantages. OTOH I think there are also important advantages to internal impact evaluation: the results are more likely to be bought into internally, and important context or nuance is less likely to be missed. For making a theory of change specifically, I think it’s quite important this is done internally, usually. Overall I think the ideal setup would quite often be for organisations to have their own internal impact evaluation function.
And that’s interesting on funder interest. In a few cases, organisations I’ve spoken to have been able to get specific grants for impact evaluation. But also org’s might choose to reallocate their existing budget, without needing additional funding, if they consider impact evaluation an essential function. E.g. for a fixed budget, they might decide that they should be allocating at least e.g. 5 / 10% to impact evaluation. (But I guess this might be harder if it required pulling back on existing activities)
I kind’ve see this as more at the org-level than funder-level tbh, similarly to any other spending decision facing an organisation. Perhaps because I’m thinking most about the benefits to org’s themselves. But I definitely still agree that funder interest is a big driver.
I think there are also important advantages to internal impact evaluation: the results are more likely to be bought into internally, and important context or nuance is less likely to be missed.
I agree there are some advantages to internal evaluation. But I think “the results are more likely to be bought into internally” is, in many cases, only an advantage insofar that orgs are erroneously more likely to trust their own work than external independent work.
That said, I agree that the importance of orgs bringing important context and nuance (and just basic information) to the evaluation can hardly be over-stated. My general take here is that the ideal arrangement is for the org and external evaluators to work very closely on an evaluation, so they can combine the benefits of insider knowledge and external expertise.
I would even say that in those kinds of cases, it’s not extremely important whether the evaluation is primarily lead by the org or primarily lead by the external evaluator (so long as there’s still scope for the external evaluator to offer an independent, and maybe even dissenting, take on the org’s conclusions). I think people can reasonably disagree about how important it is that, in addition, the external evaluator is truly independent (i.e. ideally funded by an external funder, not selected and contracted by the org in question, which obviously potentially risks biasing the evaluator).
I actually think it could be quite reasonable for an org to trust or place more weight on an internal evaluation more than an external one, but apart from that fully agree with all you say!
Thanks David!
I agree independence has advantages. OTOH I think there are also important advantages to internal impact evaluation: the results are more likely to be bought into internally, and important context or nuance is less likely to be missed. For making a theory of change specifically, I think it’s quite important this is done internally, usually. Overall I think the ideal setup would quite often be for organisations to have their own internal impact evaluation function.
And that’s interesting on funder interest. In a few cases, organisations I’ve spoken to have been able to get specific grants for impact evaluation. But also org’s might choose to reallocate their existing budget, without needing additional funding, if they consider impact evaluation an essential function. E.g. for a fixed budget, they might decide that they should be allocating at least e.g. 5 / 10% to impact evaluation. (But I guess this might be harder if it required pulling back on existing activities)
I kind’ve see this as more at the org-level than funder-level tbh, similarly to any other spending decision facing an organisation. Perhaps because I’m thinking most about the benefits to org’s themselves. But I definitely still agree that funder interest is a big driver.
Thanks for the reply!
I agree there are some advantages to internal evaluation. But I think “the results are more likely to be bought into internally” is, in many cases, only an advantage insofar that orgs are erroneously more likely to trust their own work than external independent work.
That said, I agree that the importance of orgs bringing important context and nuance (and just basic information) to the evaluation can hardly be over-stated. My general take here is that the ideal arrangement is for the org and external evaluators to work very closely on an evaluation, so they can combine the benefits of insider knowledge and external expertise.
I would even say that in those kinds of cases, it’s not extremely important whether the evaluation is primarily lead by the org or primarily lead by the external evaluator (so long as there’s still scope for the external evaluator to offer an independent, and maybe even dissenting, take on the org’s conclusions). I think people can reasonably disagree about how important it is that, in addition, the external evaluator is truly independent (i.e. ideally funded by an external funder, not selected and contracted by the org in question, which obviously potentially risks biasing the evaluator).
I actually think it could be quite reasonable for an org to trust or place more weight on an internal evaluation more than an external one, but apart from that fully agree with all you say!