Hi Nick, thanks for the thoughtful response. I think you make a lot of good points and I agree that there are numerous incentives can can lead an M+E provider to bias results positively. That’s why there is a ton of bad M+E out there.
One main reaction: for an employee who works in an M+E org, there is arguably no worse situation than being pressured to skew your results positively, or even worse, taking on projects where you know a certain results is expected by your clients. It makes you feel you work is meaningless, and really sucks. And when you are put in this situations, you sure as hell don’t want to work for the same client again.
Yes, i hear you that for bean-counters in an organization (or those who get dividends in a for-profit org), there are strong incentives to make clients happy and get more contracts. But I think that the job-satisfaction incentive for rank-and-file employees skews the other way. And in the course of my experience, I think it is this latter incentive toward truth-telling that has dominated in most cases.
Perhaps, like the rules for auditors established after accounting scandals, funders should adopt a policy requiring changes in the M&E provider at certain intervals, maybe with some random selection of interval? Knowing that next year’s assessment may be done by a different firm may create a disincentive for gaming the system (and a pathway for easier detection of any gaming). That may only work for projects with longer-term M&E efforts though.
Hi Nick, thanks for the thoughtful response. I think you make a lot of good points and I agree that there are numerous incentives can can lead an M+E provider to bias results positively. That’s why there is a ton of bad M+E out there.
One main reaction: for an employee who works in an M+E org, there is arguably no worse situation than being pressured to skew your results positively, or even worse, taking on projects where you know a certain results is expected by your clients. It makes you feel you work is meaningless, and really sucks. And when you are put in this situations, you sure as hell don’t want to work for the same client again.
Yes, i hear you that for bean-counters in an organization (or those who get dividends in a for-profit org), there are strong incentives to make clients happy and get more contracts. But I think that the job-satisfaction incentive for rank-and-file employees skews the other way. And in the course of my experience, I think it is this latter incentive toward truth-telling that has dominated in most cases.
Perhaps, like the rules for auditors established after accounting scandals, funders should adopt a policy requiring changes in the M&E provider at certain intervals, maybe with some random selection of interval? Knowing that next year’s assessment may be done by a different firm may create a disincentive for gaming the system (and a pathway for easier detection of any gaming). That may only work for projects with longer-term M&E efforts though.