Risk 1: Charities could alter, conceal, fabricate and/or destroy evidence to cover their tracks.
While this is possible in the abstract, I don’t think it is generally applicable enough to credit without a case-specific application to the facts at hand.
For example: Your most recent post asserts that Singeria has made “false claims” and that this is an “important issue because Sinergia receives millions of dollars in grants from EA organizations.” You note specifically that it has been a ACE top charity since 2018 and recently received a $3.3MM grant from Open Phil. I’ll allude to that at points in the discussion as an example of what types of reports you might be producing, but emphasize that the discussion is entirely hypothetical.
One problem is that your argument for importance also underscores how difficult and risky for an organization to “alter, fabricate, and/or destroy evidence” in the way you describe. If the charity challenges your work, presumably you’d produce the screenshots. At that point, the charity would have to double down and accuse you of fabricating the screenshots; there’s no plausible way to claim “misunderstanding” at that point in the game.
That would be a serious charge, not least because I would expect (and generally hope) that grantmakers would impose an organizational death penalty on an organization that fraudulently claimed that an evaluator’s screenshots were forged. Stuff like misattributed credit and poor calculations can be written off as non-malicious error, but deceit about something as concrete as what the organization’s website said would be very hard to spin as anything other than career/organization-ending lack of trustworthiness.
In addition to general Internet archives, there are a number of places not under the organization’s control where corroborating evidence could potentially be found. And I think it would be very hard for an organization to reduce the risk of information leakage to a level it deemed an acceptable risk:
Presumably, the alleged false claims would appear in fundraising and fundraising-adjacent materials in the possession of third parties, such as Open Phil, ACE, other grantmakers, recipients of fundraising e-mails, etc.
It’s likely that references to the false claims would also exist in some form in individual memories and in written discussions under the control of disinterested third parties (e.g., discussions on the Forum, which I believe are regularly scraped by various people). Although this might not be conclusive, it would leave the organization under a cloud of suspicion probably worse than admitting the truth of the original allegations!
There’s also the angle that falsely accusing someone of forging screenshots would constitute libel—and even in the US, the organization isn’t likely to have any good defenses other than non-falsity of the accusation. And litigation would bring even more tools to help uncover the truth:
If a preservation demand were filed quickly enough, third-party service providers (e.g., a webhost, an Internet archive that received takedown requests) may have evidence establishing the attempted cover-up. Some of these might even voluntarily release information without a lawsuit even being filed.
The organization might have to turn over its own records in discovery that would establish the coverup. Of course, the organization could try to hide documents, but that comes with risks!
Of course, there might not be litigation . . . but I wouldn’t underestimate the risk to the organization. If there were enough suspicion, I think people would fund it if you were able to conditionally commit some funds to it upfront and/or if there were some third-party recollections of the disputed statements. The litigation would be fairly simple and thus cost-effective as a way to determine whether the organization should receive funds.
Given the difficulties and risk of accusing an evaluator of committing forgery, I’d generally be inclined to give this issue meaningful weight where there was a reasonable basis to believe that:
* Distribution of the information to third parties was limited enough that the organization could predict that evidence tampering had a high degree of success.
* The report was so damning that the organization had little to lose from claiming forgery (cf. the Hail Mary pass situation in American football).
* Organization leadership was irrational enough to claim forgery even when doing so was strongly against the organization’s interests and their personal interests.
Yeah the causal model here is extremely implausible to me. I’m not saying fraud or falsified evidence is rare in charities, on balance I expect it to be slightly higher than numbers I’ve seen of the economy as a whole (3-5%). But the specific causal model of (summarizing)
charity was reviewed-> charity gets sent evidence from the review → charity hides the evidence that was previously on the public internet → they were successful in doing so
Thanks for your reply Jason—great to hear your perspective.
One problem is that your argument for importance also underscores how difficult and risky for an organization to “alter, fabricate, and/or destroy evidence” in the way you describe.
We understand it is risky, but could you clarify why it would be difficult for an organization to “alter, fabricate, and/or destroy evidence?” In many cases, wouldn’t it be as simple as changing values in a spreadsheet?
It’s not difficult to try. I believe it would be difficult to succeed (or even not get easily caught) because the evidence exists in too many third-party hands outside the organization’s custody and control. Thus, absent special circumstances, it is very unlikely an organization would even try; the difficulty of success would deter them.
(I meant “risk” to refer more to the consequences of being caught—that it would pose a grave risk to the organization’s continued existence, and to the careers of those who attempted the tampering.)
While this is possible in the abstract, I don’t think it is generally applicable enough to credit without a case-specific application to the facts at hand.
For example: Your most recent post asserts that Singeria has made “false claims” and that this is an “important issue because Sinergia receives millions of dollars in grants from EA organizations.” You note specifically that it has been a ACE top charity since 2018 and recently received a $3.3MM grant from Open Phil. I’ll allude to that at points in the discussion as an example of what types of reports you might be producing, but emphasize that the discussion is entirely hypothetical.
One problem is that your argument for importance also underscores how difficult and risky for an organization to “alter, fabricate, and/or destroy evidence” in the way you describe. If the charity challenges your work, presumably you’d produce the screenshots. At that point, the charity would have to double down and accuse you of fabricating the screenshots; there’s no plausible way to claim “misunderstanding” at that point in the game.
That would be a serious charge, not least because I would expect (and generally hope) that grantmakers would impose an organizational death penalty on an organization that fraudulently claimed that an evaluator’s screenshots were forged. Stuff like misattributed credit and poor calculations can be written off as non-malicious error, but deceit about something as concrete as what the organization’s website said would be very hard to spin as anything other than career/organization-ending lack of trustworthiness.
In addition to general Internet archives, there are a number of places not under the organization’s control where corroborating evidence could potentially be found. And I think it would be very hard for an organization to reduce the risk of information leakage to a level it deemed an acceptable risk:
Presumably, the alleged false claims would appear in fundraising and fundraising-adjacent materials in the possession of third parties, such as Open Phil, ACE, other grantmakers, recipients of fundraising e-mails, etc.
It’s likely that references to the false claims would also exist in some form in individual memories and in written discussions under the control of disinterested third parties (e.g., discussions on the Forum, which I believe are regularly scraped by various people). Although this might not be conclusive, it would leave the organization under a cloud of suspicion probably worse than admitting the truth of the original allegations!
There’s also the angle that falsely accusing someone of forging screenshots would constitute libel—and even in the US, the organization isn’t likely to have any good defenses other than non-falsity of the accusation. And litigation would bring even more tools to help uncover the truth:
If a preservation demand were filed quickly enough, third-party service providers (e.g., a webhost, an Internet archive that received takedown requests) may have evidence establishing the attempted cover-up. Some of these might even voluntarily release information without a lawsuit even being filed.
The organization might have to turn over its own records in discovery that would establish the coverup. Of course, the organization could try to hide documents, but that comes with risks!
Of course, there might not be litigation . . . but I wouldn’t underestimate the risk to the organization. If there were enough suspicion, I think people would fund it if you were able to conditionally commit some funds to it upfront and/or if there were some third-party recollections of the disputed statements. The litigation would be fairly simple and thus cost-effective as a way to determine whether the organization should receive funds.
Given the difficulties and risk of accusing an evaluator of committing forgery, I’d generally be inclined to give this issue meaningful weight where there was a reasonable basis to believe that:
* Distribution of the information to third parties was limited enough that the organization could predict that evidence tampering had a high degree of success.
* The report was so damning that the organization had little to lose from claiming forgery (cf. the Hail Mary pass situation in American football).
* Organization leadership was irrational enough to claim forgery even when doing so was strongly against the organization’s interests and their personal interests.
Yeah the causal model here is extremely implausible to me. I’m not saying fraud or falsified evidence is rare in charities, on balance I expect it to be slightly higher than numbers I’ve seen of the economy as a whole (3-5%). But the specific causal model of (summarizing)
just seems extremely implausible to me.
Thanks for your reply Jason—great to hear your perspective.
We understand it is risky, but could you clarify why it would be difficult for an organization to “alter, fabricate, and/or destroy evidence?” In many cases, wouldn’t it be as simple as changing values in a spreadsheet?
It’s not difficult to try. I believe it would be difficult to succeed (or even not get easily caught) because the evidence exists in too many third-party hands outside the organization’s custody and control. Thus, absent special circumstances, it is very unlikely an organization would even try; the difficulty of success would deter them.
(I meant “risk” to refer more to the consequences of being caught—that it would pose a grave risk to the organization’s continued existence, and to the careers of those who attempted the tampering.)