It is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claims.
If a charity has done this, they should not be given the benefit of the doubt, because it is their own fault that there is not sufficient publicly stated evidence to justify their important claims; they had the opportunity to state this evidence but did not. Additionally, giving a charity the benefit of the doubt in this situation incentivizes not publicly stating evidence in situations where sufficient evidence does not exist, since the charity will simply be given the benefit of the doubt.
Note that I interpret this standard as “provide sufficient evidence to support their claims in the eyes of any skeptical outside observer”, as that’s the role you’re placing yourself in here
Thanks for clarifying, Neel, and for giving us the opportunity to clarify our statements. Quoting our original post:
It is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claims.
By “sufficient evidence” we did not mean proof eliminating all possible doubt. We meant evidence that would satisfy a reasonable person. For example, a reasonable person thinks there is sufficient evidence that Neil Armstrong landed on the moon, but some people do not.
Additionally, we do not expect charities to provide sufficient evidence for all of their publicly stated claims, just their important ones. For instance, if a charity claims to have provided 10,000 meals to homeless individuals, they should provide sufficient evidence that this actually happened. However, if they announce that their end-of-year party was at Disneyland, no additional evidence is necessary.
This rationale only works if you very carefully calibrate your claims. “Charity X has not provided sufficient, publicly available evidence to substantiate its claim to have provided 10K meals to homeless individuals” is a much different claim than “Charity X didn’t feed 10K meals to homeless individuals.” The former is about transparency; the latter is about whether the claimed results are true or not. An assertion that lack of transparency is not acceptable does not warrant an inference linking a lack of transparency to the falsity of the claim—at least not without giving the charity a chance to correct its alleged transparency error ahead of time.
Thanks for the insights Jason, we completely agree.
We are curious if you would you also agree that it is (1) reasonable to assert “Charity X didn’t feed 10K meals to homeless individuals” and (2) unreasonable to give Charity X the benefit of the doubt, if:
“Charity X has not provided sufficient, publicly available evidence to substantiate its claim to have provided 10K meals to homeless individuals” and
There is publicly available and sufficient evidence indicating Charity X did not provide 10K meals to homeless individuals.
I think your second strand of evidence is doing almost all the work here. There’s a big difference between evidence sufficient to establish a prima facie case, such that a reasonable mind would accept the conclusion if no additional evidence were offered and evidence that so conclusively establishes the truth that it’s a waste of time to ask the charity for a response.
As with other considerations, this is a fact-bound consideration. If you’re relying on it to justify not giving a specific organization notice and an opportunity to prepare a response for concurrent publication, you can always say that—and thereby accept the loss in your own credibility if readers later judge that the charity had a reasonable response that it could have provided if asked.
I would include situations in which the charity could establish the substantial truth of the assertion. E.g., if the charity could show that it made a math error and actually fed 9,823 individuals, that is information a reasonable reader would find highly relevant to evaluating the assertion of falsity. Not giving the charity a chance to concurrently explain that does a disservice to both the charity and the reader.
Yeah thi. In particular anytime you criticise an organisation and they are only able to respond a few weeks later, many readers will see your criticism but will not see the organization’s response. This inherently will give a misleading impression, so you must be incredibly confident that there is no mitigating context that this organisation would give lest you do their reputation undue damage empirically. I think it is obviously the case empirically that when an organisation gets critiqued including your two previous examples that there is valuable additional context they’re able to provide when given notice
If you’re relying on it to justify not giving a specific organization notice and an opportunity to prepare a response for concurrent publication, you can always say that
Could you clarify what you are saying we are relying on?
It’s a hypothetical about feeding homeless people, so it doesn’t track any of your prior decisions not to give advance notice to the target organization.
In the hypothetical, you could assert that advance notice wasn’t warranted because your external evidence was overwhelming enough that providing advance notice and opportunity for comment would have been a waste of time. This would be consistent with my interpretation of the community norm, which is that critics should provide advance notice unless they can (and do) articulate a good cause on a situation-specific basis to dispense with doing so. But if you did provide this justification, and the organization was later able to provide a response that materially changed the conclusions to be drawn, then your organization would suffer a loss in credibility as a result.
Quoting your post:
It is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claims.
If a charity has done this, they should not be given the benefit of the doubt, because it is their own fault that there is not sufficient publicly stated evidence to justify their important claims; they had the opportunity to state this evidence but did not. Additionally, giving a charity the benefit of the doubt in this situation incentivizes not publicly stating evidence in situations where sufficient evidence does not exist, since the charity will simply be given the benefit of the doubt.
Note that I interpret this standard as “provide sufficient evidence to support their claims in the eyes of any skeptical outside observer”, as that’s the role you’re placing yourself in here
Thanks for clarifying, Neel, and for giving us the opportunity to clarify our statements. Quoting our original post:
By “sufficient evidence” we did not mean proof eliminating all possible doubt. We meant evidence that would satisfy a reasonable person. For example, a reasonable person thinks there is sufficient evidence that Neil Armstrong landed on the moon, but some people do not.
Additionally, we do not expect charities to provide sufficient evidence for all of their publicly stated claims, just their important ones. For instance, if a charity claims to have provided 10,000 meals to homeless individuals, they should provide sufficient evidence that this actually happened. However, if they announce that their end-of-year party was at Disneyland, no additional evidence is necessary.
This rationale only works if you very carefully calibrate your claims. “Charity X has not provided sufficient, publicly available evidence to substantiate its claim to have provided 10K meals to homeless individuals” is a much different claim than “Charity X didn’t feed 10K meals to homeless individuals.” The former is about transparency; the latter is about whether the claimed results are true or not. An assertion that lack of transparency is not acceptable does not warrant an inference linking a lack of transparency to the falsity of the claim—at least not without giving the charity a chance to correct its alleged transparency error ahead of time.
Thanks for the insights Jason, we completely agree.
We are curious if you would you also agree that it is (1) reasonable to assert “Charity X didn’t feed 10K meals to homeless individuals” and (2) unreasonable to give Charity X the benefit of the doubt, if:
“Charity X has not provided sufficient, publicly available evidence to substantiate its claim to have provided 10K meals to homeless individuals” and
There is publicly available and sufficient evidence indicating Charity X did not provide 10K meals to homeless individuals.
I think your second strand of evidence is doing almost all the work here. There’s a big difference between evidence sufficient to establish a prima facie case, such that a reasonable mind would accept the conclusion if no additional evidence were offered and evidence that so conclusively establishes the truth that it’s a waste of time to ask the charity for a response.
As with other considerations, this is a fact-bound consideration. If you’re relying on it to justify not giving a specific organization notice and an opportunity to prepare a response for concurrent publication, you can always say that—and thereby accept the loss in your own credibility if readers later judge that the charity had a reasonable response that it could have provided if asked.
I would include situations in which the charity could establish the substantial truth of the assertion. E.g., if the charity could show that it made a math error and actually fed 9,823 individuals, that is information a reasonable reader would find highly relevant to evaluating the assertion of falsity. Not giving the charity a chance to concurrently explain that does a disservice to both the charity and the reader.
Yeah thi. In particular anytime you criticise an organisation and they are only able to respond a few weeks later, many readers will see your criticism but will not see the organization’s response. This inherently will give a misleading impression, so you must be incredibly confident that there is no mitigating context that this organisation would give lest you do their reputation undue damage empirically. I think it is obviously the case empirically that when an organisation gets critiqued including your two previous examples that there is valuable additional context they’re able to provide when given notice
Thanks for the insights, Jason.
Could you clarify what you are saying we are relying on?
It’s a hypothetical about feeding homeless people, so it doesn’t track any of your prior decisions not to give advance notice to the target organization.
In the hypothetical, you could assert that advance notice wasn’t warranted because your external evidence was overwhelming enough that providing advance notice and opportunity for comment would have been a waste of time. This would be consistent with my interpretation of the community norm, which is that critics should provide advance notice unless they can (and do) articulate a good cause on a situation-specific basis to dispense with doing so. But if you did provide this justification, and the organization was later able to provide a response that materially changed the conclusions to be drawn, then your organization would suffer a loss in credibility as a result.
Seems pretty clear that he meant your second statement