That’s what I did for my recent critical review of one of Social Change Lab’s reports.
One of the challenges here is defining what “criticism” is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
For what it’s worth, I think your review was solidly on the “not a criticism of a person or organization” side of the ledger.
Second: A big reason to reach out to people is to resolve misunderstandings. But it’s even better to resolve misunderstandings in public, after publishing the criticism. Readers may have the same misunderstandings, and writing a public back-and-forth is better for readers.
That’s consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
One of the challenges here is defining what “criticism” is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
I think this is much messier than suggested here. Consider a situation where charity evaluator org A performs and publicly publishes a cost-effective analysis of org B. A critic publishes a re-analysis that suggests the cost-effectiveness is much lower than the original analysis, perhaps far below the expected funding bar. Org A may feel the criticism goes to its competence as a charity evaluator, and org B may consider this an existential threat that could result in loss of funding, yet I think a public cost effectiveness analysis simply has to be considered “intellectual work product”.
I don’t think this is hypothetical. A while ago there was a case where a critic posted some criticism of a cost-effectiveness analysis of a mental health charity, I believe the charity being evaluated was called “StrongMinds”. I think that case is similar to what I describe above.
That’s consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
This presents a similar problem to the laziness allegations mentioned above except worse, since the critic may face allegations that failure to change their criticism pre-publication demonstrates how the critic is insufficiently “truthseeking”.
In fact, to the extent that their is no community accepted safe-harbor for what is expected, I think there is likely to be a death-by-a-thousand-cuts problem. Critics can predictably expect that they will need to litigate their conduct regarding these meta-issues when they publish (even if they actually do a lot of the what is suggested!), likely in a way that moves discussion away from the content of their object-level criticism.
Again, not hypothetical. The Nonlinear situation goes to this, I think.
One of the challenges here is defining what “criticism” is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
For what it’s worth, I think your review was solidly on the “not a criticism of a person or organization” side of the ledger.
That’s consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
I think this is much messier than suggested here. Consider a situation where charity evaluator org A performs and publicly publishes a cost-effective analysis of org B. A critic publishes a re-analysis that suggests the cost-effectiveness is much lower than the original analysis, perhaps far below the expected funding bar. Org A may feel the criticism goes to its competence as a charity evaluator, and org B may consider this an existential threat that could result in loss of funding, yet I think a public cost effectiveness analysis simply has to be considered “intellectual work product”.
I don’t think this is hypothetical. A while ago there was a case where a critic posted some criticism of a cost-effectiveness analysis of a mental health charity, I believe the charity being evaluated was called “StrongMinds”. I think that case is similar to what I describe above.
This presents a similar problem to the laziness allegations mentioned above except worse, since the critic may face allegations that failure to change their criticism pre-publication demonstrates how the critic is insufficiently “truthseeking”.
In fact, to the extent that their is no community accepted safe-harbor for what is expected, I think there is likely to be a death-by-a-thousand-cuts problem. Critics can predictably expect that they will need to litigate their conduct regarding these meta-issues when they publish (even if they actually do a lot of the what is suggested!), likely in a way that moves discussion away from the content of their object-level criticism.
Again, not hypothetical. The Nonlinear situation goes to this, I think.
FWIW I do not expect people to run cost-effectiveness analyses by orgs before publishing them.