I am somewhat against a norm of reaching out to people before criticizing their work. Dynomight’s Arguing Without Warning has IMO the strongest arguments on this topic.
First: Criticism is difficult; requiring more effort on the part of critics makes criticism less likely to happen. OP did acknowledge this with
The best reason not to involve the criticised person or org is if doing so would in practice stop you from posting your criticism.
But realistically, there is a social penalty to saying “I would not have posted this criticism if I’d been required to reach out to orgs first.” It makes you look lazy. A norm of “reach out to people, unless it would stop you from posting your criticism” is not a viable norm because critics who would’ve been stopped from posting criticism are unable to defend themselves. In some cases, I expects this norm to stop criticisms from getting written. So I think a better norm is “you don’t have to reach out to people”.
Second: A big reason to reach out to people is to resolve misunderstandings. But it’s even better to resolve misunderstandings in public, after publishing the criticism. Readers may have the same misunderstandings, and writing a public back-and-forth is better for readers.
(Dynomight’s post also gives a couple other arguments that I don’t think are quite as important.)
My preferred norms are:
Critics don’t have to do anything.
If they want to be nice, critics can reach out to the subject of criticism to inform them of the criticism after it has already been published.
That’s what I did for my recent critical review of one of Social Change Lab’s reports.
There are some circumstances where it makes sense to reach out to people before publishing, but I don’t think that should be the norm, and I don’t think we should have any expectation that critics do it.
For some reason I find this title delightful. I kind of wish I could have an “argues without warning” flair or something.
I agree with arguments you present above and your conclusion about preferred norms. That said, I think people might have in mind certain types of cases that might justify the need for reaching out beyond the case of general “criticism”. For example, imagine something like this:
Critic: Org X made this super big mistake that makes all their conclusions inaccurate and means all the work of the org is actually net-negative!!!!!
Org X: We didn’t make any mistakes, critic made a mistake on their part because they don’t have all the info/context, but since we do have that it was easy for us to spot their error. If they had just reached out we would have explained the context. We didn’t publicly explain it because its detailed/nuanced and we can’t spend 100% of our time explaining context about things on the off-chance someone is going to criticize us about some random detail.
Now, my view is, even if this is what happens, this is still a positive outcome, because, like you say:
it’s even better to resolve misunderstandings in public
Transparency has costs, but I think they are usually internal costs to the org, while transparency also has external benefits, and thus would be expected to be systematically under-supplied by orgs.
At the same time, I think most cases of criticism are realistically more mixed, with the critic making reasonable points but also some mistakes, and the org having some obvious corrections to the criticism but also some places where the back-and-forth is very enlightening. Requiring people to reach out I think risks losing a lot of the value that comes from such “debatable” cases for the reasons you mention.
Another set of cases that is worth separating out are allegations of intentional misconduct. I think there are particular reasons why it might make sense to have a higher expectation for critics to reach out to an org if they are accusing that org of intentional misconduct. I think this may also vary by whether the critic personally observed misconduct, in which case I think issues like a risk of retaliation or extreme difficulty for the critic may weigh in favor of not expecting the critic to reach out.
That’s what I did for my recent critical review of one of Social Change Lab’s reports.
One of the challenges here is defining what “criticism” is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
For what it’s worth, I think your review was solidly on the “not a criticism of a person or organization” side of the ledger.
Second: A big reason to reach out to people is to resolve misunderstandings. But it’s even better to resolve misunderstandings in public, after publishing the criticism. Readers may have the same misunderstandings, and writing a public back-and-forth is better for readers.
That’s consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
One of the challenges here is defining what “criticism” is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
I think this is much messier than suggested here. Consider a situation where charity evaluator org A performs and publicly publishes a cost-effective analysis of org B. A critic publishes a re-analysis that suggests the cost-effectiveness is much lower than the original analysis, perhaps far below the expected funding bar. Org A may feel the criticism goes to its competence as a charity evaluator, and org B may consider this an existential threat that could result in loss of funding, yet I think a public cost effectiveness analysis simply has to be considered “intellectual work product”.
I don’t think this is hypothetical. A while ago there was a case where a critic posted some criticism of a cost-effectiveness analysis of a mental health charity, I believe the charity being evaluated was called “StrongMinds”. I think that case is similar to what I describe above.
That’s consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
This presents a similar problem to the laziness allegations mentioned above except worse, since the critic may face allegations that failure to change their criticism pre-publication demonstrates how the critic is insufficiently “truthseeking”.
In fact, to the extent that their is no community accepted safe-harbor for what is expected, I think there is likely to be a death-by-a-thousand-cuts problem. Critics can predictably expect that they will need to litigate their conduct regarding these meta-issues when they publish (even if they actually do a lot of the what is suggested!), likely in a way that moves discussion away from the content of their object-level criticism.
Again, not hypothetical. The Nonlinear situation goes to this, I think.
I am somewhat against a norm of reaching out to people before criticizing their work. Dynomight’s Arguing Without Warning has IMO the strongest arguments on this topic.
First: Criticism is difficult; requiring more effort on the part of critics makes criticism less likely to happen. OP did acknowledge this with
But realistically, there is a social penalty to saying “I would not have posted this criticism if I’d been required to reach out to orgs first.” It makes you look lazy. A norm of “reach out to people, unless it would stop you from posting your criticism” is not a viable norm because critics who would’ve been stopped from posting criticism are unable to defend themselves. In some cases, I expects this norm to stop criticisms from getting written. So I think a better norm is “you don’t have to reach out to people”.
Second: A big reason to reach out to people is to resolve misunderstandings. But it’s even better to resolve misunderstandings in public, after publishing the criticism. Readers may have the same misunderstandings, and writing a public back-and-forth is better for readers.
(Dynomight’s post also gives a couple other arguments that I don’t think are quite as important.)
My preferred norms are:
Critics don’t have to do anything.
If they want to be nice, critics can reach out to the subject of criticism to inform them of the criticism after it has already been published.
That’s what I did for my recent critical review of one of Social Change Lab’s reports.
There are some circumstances where it makes sense to reach out to people before publishing, but I don’t think that should be the norm, and I don’t think we should have any expectation that critics do it.
For some reason I find this title delightful. I kind of wish I could have an “argues without warning” flair or something.
I agree with arguments you present above and your conclusion about preferred norms. That said, I think people might have in mind certain types of cases that might justify the need for reaching out beyond the case of general “criticism”. For example, imagine something like this:
Now, my view is, even if this is what happens, this is still a positive outcome, because, like you say:
Transparency has costs, but I think they are usually internal costs to the org, while transparency also has external benefits, and thus would be expected to be systematically under-supplied by orgs.
At the same time, I think most cases of criticism are realistically more mixed, with the critic making reasonable points but also some mistakes, and the org having some obvious corrections to the criticism but also some places where the back-and-forth is very enlightening. Requiring people to reach out I think risks losing a lot of the value that comes from such “debatable” cases for the reasons you mention.
Another set of cases that is worth separating out are allegations of intentional misconduct. I think there are particular reasons why it might make sense to have a higher expectation for critics to reach out to an org if they are accusing that org of intentional misconduct. I think this may also vary by whether the critic personally observed misconduct, in which case I think issues like a risk of retaliation or extreme difficulty for the critic may weigh in favor of not expecting the critic to reach out.
One of the challenges here is defining what “criticism” is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
For what it’s worth, I think your review was solidly on the “not a criticism of a person or organization” side of the ledger.
That’s consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
I think this is much messier than suggested here. Consider a situation where charity evaluator org A performs and publicly publishes a cost-effective analysis of org B. A critic publishes a re-analysis that suggests the cost-effectiveness is much lower than the original analysis, perhaps far below the expected funding bar. Org A may feel the criticism goes to its competence as a charity evaluator, and org B may consider this an existential threat that could result in loss of funding, yet I think a public cost effectiveness analysis simply has to be considered “intellectual work product”.
I don’t think this is hypothetical. A while ago there was a case where a critic posted some criticism of a cost-effectiveness analysis of a mental health charity, I believe the charity being evaluated was called “StrongMinds”. I think that case is similar to what I describe above.
This presents a similar problem to the laziness allegations mentioned above except worse, since the critic may face allegations that failure to change their criticism pre-publication demonstrates how the critic is insufficiently “truthseeking”.
In fact, to the extent that their is no community accepted safe-harbor for what is expected, I think there is likely to be a death-by-a-thousand-cuts problem. Critics can predictably expect that they will need to litigate their conduct regarding these meta-issues when they publish (even if they actually do a lot of the what is suggested!), likely in a way that moves discussion away from the content of their object-level criticism.
Again, not hypothetical. The Nonlinear situation goes to this, I think.
FWIW I do not expect people to run cost-effectiveness analyses by orgs before publishing them.