I do think there are downsides with sharing draft reviews with organizations ahead of time, but I think theyâre mostly different from the ones listed here. The biggest risk I see is that the organization could use the time to take an adversarial approach:
Trying to keep the review from being published. This could look like accusations of libel and threats to sue, or other kinds of retaliation (âis publishing this really in the best interest of your career...?â).
Preparing people to astroturf the comment section
Preparing a refutation that is seriously flawed but in a way that takes significant effort to investigate. This then risks turning into the opposite of the situation people usually worry about: instead of people seeing a negative review but not the orgâs follow-up with corrections they might see a negative review and a thorough refutation come out at the same time, and then never see the reviewerâs follow-up where they show that the refutation is misleading.
I also think what you list as risk 2, âUnconscious biases from interacting with charity staffâ, is a real risk. If people at an evaluator have been working with people at a charity, especially if they do this over long periods, they will naturally become more sympathetic. [1]
Of the other listed issues, however, I agree with the other commenters that theyâre avoidable:
There are many services for archiving web pages, and falsely claiming that archives have been tampered with is a pretty terrible strategy for a charity to take. If youâre especially concerned about this, however, you could publish your own archives of your evidence in advance (without checking with the org). The analogy to police is not a good one, because police have the ability get search warrants and learn additional things that are not already public.
If the charity says âVettedCausesâ review is about problems we have already addressedâ without acknowledging that they fixed the problems in response to your feedback I think that would look quite bad for them. There is risk of dispute over whether they made changes in response to your review or coincidentally, but if you give them a week to review and they claim they just happened to make the changes in that short time between their receiving the draft and you releasing it I think people would be quite skeptical.
On âIt is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claimsâ, I donât think youâve weighed how difficult this is. When I read through the funding appeals of even pretty careful and thoughtful charities I basically always notice claims that are not fully backed up by publicly stated evidence. While this does sound bad, organizations have a bunch of competing priorities and justifying their work to this level is rarely worth it.
[1] Charities can also trade access (allowing a more comprehensive evaluation) for more favorable coverage, generally not in an explicit way. I think this is related to why GiveWell and ACE have ended up with a policy that they only release reviews if charities are willing to see them released. This is a lot like access journalism. But this isnât related to whether you share drafts for review.
I do think there are downsides with sharing draft reviews with organizations ahead of time, but I think theyâre mostly different from the ones listed here. The biggest risk I see is that the organization could use the time to take an adversarial approach:
Trying to keep the review from being published. This could look like accusations of libel and threats to sue, or other kinds of retaliation (âis publishing this really in the best interest of your career...?â).
Preparing people to astroturf the comment section
Preparing a refutation that is seriously flawed but in a way that takes significant effort to investigate. This then risks turning into the opposite of the situation people usually worry about: instead of people seeing a negative review but not the orgâs follow-up with corrections they might see a negative review and a thorough refutation come out at the same time, and then never see the reviewerâs follow-up where they show that the refutation is misleading.
I also think what you list as risk 2, âUnconscious biases from interacting with charity staffâ, is a real risk. If people at an evaluator have been working with people at a charity, especially if they do this over long periods, they will naturally become more sympathetic. [1]
Of the other listed issues, however, I agree with the other commenters that theyâre avoidable:
There are many services for archiving web pages, and falsely claiming that archives have been tampered with is a pretty terrible strategy for a charity to take. If youâre especially concerned about this, however, you could publish your own archives of your evidence in advance (without checking with the org). The analogy to police is not a good one, because police have the ability get search warrants and learn additional things that are not already public.
If the charity says âVettedCausesâ review is about problems we have already addressedâ without acknowledging that they fixed the problems in response to your feedback I think that would look quite bad for them. There is risk of dispute over whether they made changes in response to your review or coincidentally, but if you give them a week to review and they claim they just happened to make the changes in that short time between their receiving the draft and you releasing it I think people would be quite skeptical.
On âIt is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claimsâ, I donât think youâve weighed how difficult this is. When I read through the funding appeals of even pretty careful and thoughtful charities I basically always notice claims that are not fully backed up by publicly stated evidence. While this does sound bad, organizations have a bunch of competing priorities and justifying their work to this level is rarely worth it.
Overall, I donât think these considerations appreciably change my view that you should run reviews by the orgs theyâre about.
[1] Charities can also trade access (allowing a more comprehensive evaluation) for more favorable coverage, generally not in an explicit way. I think this is related to why GiveWell and ACE have ended up with a policy that they only release reviews if charities are willing to see them released. This is a lot like access journalism. But this isnât related to whether you share drafts for review.