As someone with experience in more traditional humanitarian organisations it is difficult to not be sarcastic about Give Well seemingly just discovering the limitation of their approach, especially when their self reflection still reaches its limit with self congratulatory statements like this one:
“That said, it’s interesting that this “failed” intervention — which reached maybe 1⁄4 as many people as originally thought — was still probably helping people more than 90% of NGO work out there.”
What evidence does this self soothing statement has to support it exactly? Especially since their own proposed solutions are practices that more traditional NGOs follow as opposed to Give Well approach: iterating to improve interventions as they are being delivered, decision making closer to the actual point of delivery and all the “administrative overhead” that is necessary to achieve this?
I think the claim is that while this intervention wasn’t as cost effective as we thought, it’s still more cost effective than most interventions. The belief regarding the cost effectiveness (about a third as good as we thought) is itself supported by evidence, so if you think the best options are 10× as good as the average ones, you would still expect this one to be “above average”. Not sure about 90th percentile but I do think there’s a fair amount of NGO work whose effectiveness is statist indistinguishable from zero, and a minority that’s arguable actively harmful.
As someone with experience in more traditional humanitarian organisations it is difficult to not be sarcastic about Give Well seemingly just discovering the limitation of their approach, especially when their self reflection still reaches its limit with self congratulatory statements like this one:
“That said, it’s interesting that this “failed” intervention — which reached maybe 1⁄4 as many people as originally thought — was still probably helping people more than 90% of NGO work out there.”
What evidence does this self soothing statement has to support it exactly? Especially since their own proposed solutions are practices that more traditional NGOs follow as opposed to Give Well approach: iterating to improve interventions as they are being delivered, decision making closer to the actual point of delivery and all the “administrative overhead” that is necessary to achieve this?
I think the claim is that while this intervention wasn’t as cost effective as we thought, it’s still more cost effective than most interventions. The belief regarding the cost effectiveness (about a third as good as we thought) is itself supported by evidence, so if you think the best options are 10× as good as the average ones, you would still expect this one to be “above average”. Not sure about 90th percentile but I do think there’s a fair amount of NGO work whose effectiveness is statist indistinguishable from zero, and a minority that’s arguable actively harmful.