The kind of grifting that most concerns me within EA would result in a lot of fudging and inflation around reported impact in order to gain money & status. I think the hardest thing is fake, and the main thing we care about protecting, is real, verifiable impact. If there is an influx of money into EA, I would want the first thing to be done is to undertake a lot of new, third-party evaluation with depth. This is not happening today and I have not seen great plans to do so right now.
One point where I disagree is that I’m more concerned about well-intentioned people genuinely believing their project is extremely valuable, but who lack clear evidence of impact and end up not achieving much. I’m less worried about people deliberately lying/exaggerating to raise money for things they do not themselves believe are impactful.
But of course it’s hard to say from the outside, you could always argue that people are secretly doing whatever they’re doing just for the money & status, and I’ve seen people making such claims even when there’s tons of evidence to the contrary.
In my view, this one of EA’s biggest problem. Even GiveWell rarely bothers to verify numbers if it’s boring or manual to do so (e.g. number of people vaccinated, number of bednets delivered). Even when past charity evaluations could be calibrated against third party data sources, nobody bothers. We’re allocating billions using tools we’ve never bother to calibrate.
Without anyone establishing anything resembling ground truth, there is no accurate feedback signal for leaders, grantmakers, donors. When the only signal is what LOOKS impactful, and that’s how all money and status is awarded, that’s what everyone will clamour after. We’re going to learn the wrong lessons, fund the wrong projects, hire the wrong people, and then never find out.
Tons of money. Zero verification. Every fraud’s wet dream come true.
Frankly, if you can’t measure your work in a that an independent person could evaluate, I would be very skeptical that you’re having any impact at all.
The kind of grifting that most concerns me within EA would result in a lot of fudging and inflation around reported impact in order to gain money & status. I think the hardest thing is fake, and the main thing we care about protecting, is real, verifiable impact. If there is an influx of money into EA, I would want the first thing to be done is to undertake a lot of new, third-party evaluation with depth. This is not happening today and I have not seen great plans to do so right now.
See also this comment which I also agree with.
One point where I disagree is that I’m more concerned about well-intentioned people genuinely believing their project is extremely valuable, but who lack clear evidence of impact and end up not achieving much. I’m less worried about people deliberately lying/exaggerating to raise money for things they do not themselves believe are impactful.
But of course it’s hard to say from the outside, you could always argue that people are secretly doing whatever they’re doing just for the money & status, and I’ve seen people making such claims even when there’s tons of evidence to the contrary.
In my view, this one of EA’s biggest problem. Even GiveWell rarely bothers to verify numbers if it’s boring or manual to do so (e.g. number of people vaccinated, number of bednets delivered). Even when past charity evaluations could be calibrated against third party data sources, nobody bothers. We’re allocating billions using tools we’ve never bother to calibrate.
Without anyone establishing anything resembling ground truth, there is no accurate feedback signal for leaders, grantmakers, donors. When the only signal is what LOOKS impactful, and that’s how all money and status is awarded, that’s what everyone will clamour after. We’re going to learn the wrong lessons, fund the wrong projects, hire the wrong people, and then never find out.
Tons of money. Zero verification. Every fraud’s wet dream come true.
Sure, if you can directly measure what you actually want (positive impact), that is of course best :)
Frankly, if you can’t measure your work in a that an independent person could evaluate, I would be very skeptical that you’re having any impact at all.