I really appreciate your insight here. I’ve been thinking lately about lobbying the IRS in the US to require cost effectiveness disclosures in yearly reports. There are a couple concerns I have...
1: if it is even possible to convince the IRS to add cost effectiveness disclosures to the 990
2: if orgs have the expertise/capacity to evaluate their programs
3: I think the disclosure requirement must be VERY vague to allow orgs to disclose what they think is most appropriate
4: lack of oversight means these disclosure could be easily exaggerated
Benefits:
Within a similar outcome, effectiveness could be compared across orgs. Something we have never been able to do without folks like Givewell, so things like the program ratio (ew) are used by many donors.
What gets measured gets managed, so this will likely lead to orgs operating more effectively overall.
IRS and US charity regulators do a lousy job of preventing charitable scams and near-scams. I don’t see any likely universe in which they have bandwidth to decide on a cost-effectiveness methodology that works for 1.5 MM US charities doing a extremely wide range of activities.
Very likely it would be so loose all charities could give themselves high marks.… and if it were somehow not loose, there’s no clear reason to think the resultant methodology would be anything like EA.
After all, it’s a political decision (my guess is that it would require legislation by Congress, not just Treasury regs: medium confidence as a legal matter without wading through the tax code, high confidence that IRS would never do this w/o Congressional directive).
Fair points, and the idea is certainly a massive longshot.
It’s unclear to me exactly how the decision for the big redesign in 2007 was made, and if congress was involved at all. My guess is that a change like this could be made just by the IRS.
I am FAR from an expert on evaluation but even if orgs just reported outputs (X people helped, Y houses built, etc.), that information would be immensely useful. Guidestar is allowing orgs to disclose output metrics to get a platinum seal, but this is a voluntary disclosure. Form 990 could use a similar library of metrics for orgs to select from and make disclosure mandatory. I don’t think the form would be like EA at all—but the data would be extremely useful for donors/orgs trying to actually evaluate charity effectiveness.
At the end of the day, orgs may just completely BS the outputs so the idea certainly needs some work. Maybe encourage audits of outcomes?
I read an org’s 990 before making a non-trivial donation, and my guess from your bio is that you do too. But I wonder how many people (1) would take the time to carefully read the 990, (2) have enough methodological sophistication to see through at least moderate levels of obfuscation, but (3) are not significant enough donors that they feel (or are) empowered to call someone and ask for someone to provide relevant information.
Thanks Kyle, I like the idea of the IRS cost effectiveness thing but agree with Jason that practially it would never work. Even if it came in, it would be so loose as to be meaningless.
Hi Nick,
I really appreciate your insight here. I’ve been thinking lately about lobbying the IRS in the US to require cost effectiveness disclosures in yearly reports. There are a couple concerns I have...
1: if it is even possible to convince the IRS to add cost effectiveness disclosures to the 990
2: if orgs have the expertise/capacity to evaluate their programs
3: I think the disclosure requirement must be VERY vague to allow orgs to disclose what they think is most appropriate
4: lack of oversight means these disclosure could be easily exaggerated
Benefits:
Within a similar outcome, effectiveness could be compared across orgs. Something we have never been able to do without folks like Givewell, so things like the program ratio (ew) are used by many donors.
What gets measured gets managed, so this will likely lead to orgs operating more effectively overall.
IRS and US charity regulators do a lousy job of preventing charitable scams and near-scams. I don’t see any likely universe in which they have bandwidth to decide on a cost-effectiveness methodology that works for 1.5 MM US charities doing a extremely wide range of activities.
Very likely it would be so loose all charities could give themselves high marks.… and if it were somehow not loose, there’s no clear reason to think the resultant methodology would be anything like EA.
After all, it’s a political decision (my guess is that it would require legislation by Congress, not just Treasury regs: medium confidence as a legal matter without wading through the tax code, high confidence that IRS would never do this w/o Congressional directive).
Fair points, and the idea is certainly a massive longshot.
It’s unclear to me exactly how the decision for the big redesign in 2007 was made, and if congress was involved at all. My guess is that a change like this could be made just by the IRS.
I am FAR from an expert on evaluation but even if orgs just reported outputs (X people helped, Y houses built, etc.), that information would be immensely useful. Guidestar is allowing orgs to disclose output metrics to get a platinum seal, but this is a voluntary disclosure. Form 990 could use a similar library of metrics for orgs to select from and make disclosure mandatory. I don’t think the form would be like EA at all—but the data would be extremely useful for donors/orgs trying to actually evaluate charity effectiveness.
At the end of the day, orgs may just completely BS the outputs so the idea certainly needs some work. Maybe encourage audits of outcomes?
I read an org’s 990 before making a non-trivial donation, and my guess from your bio is that you do too. But I wonder how many people (1) would take the time to carefully read the 990, (2) have enough methodological sophistication to see through at least moderate levels of obfuscation, but (3) are not significant enough donors that they feel (or are) empowered to call someone and ask for someone to provide relevant information.
Thanks Kyle, I like the idea of the IRS cost effectiveness thing but agree with Jason that practially it would never work. Even if it came in, it would be so loose as to be meaningless.