There are almost no examples of criticism clearly mattering (e.g. getting someone to significantly improve their project)
I don’t know what “clearly mattering” means, but I think this characterization unduly tips the scales. People who don’t like being criticized are often going to be open about that fact, which makes it easier to build an anti-criticism case under a “clearly” standard.
Also, “criticism” covers a lot of ground—you may have a somewhat narrower definition in mind, but (even after limiting to EA projects with <10 FTEs) people are understandably reacting to a pretty broad definition.
The most obvious use of criticism is probably to deter and respond to inappropriate conduct. Setting aside whether the allegations were sustained, I think that was a major intended mechanism of action in several critical pieces. I can’t prove that having a somewhat pro-criticism culture furthers this goal, but I think it’s appropriate to give it some weight. It does seem plausible on the margin that (e.g.) orgs will be less likely to exaggerate their claims and cost-effectiveness analyses given the risk of someone posting criticism with receipts.
A softer version of this purpose could be phrased as follows: criticism is a means by which the community expresses how it expects others to act (and hopefully influences future actions by third parties even if not by the criticized organization). In your model, “public critique clearly creates barriers to starting new projects,” so one would expect public critique (or the fear thereof) to influence decisions by existing orgs as well. Then we have to decide whether that critique is on the whole good or not.
Criticism can help direct resources away from certain orgs to more productive uses. The StrongMinds-related criticisms of 2023 come to mind here. The resources could include not only funding but also mindshare (e.g., how much do I want to defer to this org?) and decisions by talent. This kind of criticism doesn’t generally pay in financial terms, so it’s reasonable to be generous in granting social credit to compensate for that. These outcomes could be measured, but doing so will often be resource-intensive and so they may not make the cut under a “clearly” standard either.
Criticism can also serve the function of market research. The usual response to people who aren’t happy about how orgs are doing their work is to go start their own org. That’s a costly response—for both the unhappy person and for the ecosystem! Suppose someone isn’t happy about CEA and EA Funds spinning off together and is thinking about trying to stand up an independent grantmaker. First off, they need to test their ideas against people who have different perspectives. They would also need to know whether a critical mass of people would move their donations over to an independent grantmaker for this or other reasons. (I think it would also be fair for someone not in a position to lead a new org to signal support for the idea, hoping that it might inspire someone else.)
It’s probably better for the market-research function to happen in public rather than in back channels. Among other things, it gives the org a chance to defend its position, and gives it a chance to adjust course if too many relevant stakeholders agree with the critic. The counterargument to this one is that little criticism actually makes it into a new organization. But I’m not sure what success rate we should expect given considerable incumbency advantage in some domains.
“People who don’t like being criticized are often going to be open about that fact”
[Just responding to this narrow point and not the comment as a whole, which contains plenty of things I agree with.]
Fwiw, I don’t think this is true in this community. Disliking criticism is a bad look and seeming responsive to criticism is highly valued. I’ve seen lots of situations up close where it would have been very aversive/costly for someone to say “I totally disagree with this criticism and think it wasn’t useful” and very tempting for someone to express lots of gratitude for criticism and change in response to it whether or not it was right. I think it’s not uncommon for the former to take more bravery than the latter and I personally feel unsure whether I’ve felt more bias towards agreeing with criticism that was wrong or disagreeing with criticism that was right.
I don’t know what “clearly mattering” means, but I think this characterization unduly tips the scales. People who don’t like being criticized are often going to be open about that fact, which makes it easier to build an anti-criticism case under a “clearly” standard.
Also, “criticism” covers a lot of ground—you may have a somewhat narrower definition in mind, but (even after limiting to EA projects with <10 FTEs) people are understandably reacting to a pretty broad definition.
The most obvious use of criticism is probably to deter and respond to inappropriate conduct. Setting aside whether the allegations were sustained, I think that was a major intended mechanism of action in several critical pieces. I can’t prove that having a somewhat pro-criticism culture furthers this goal, but I think it’s appropriate to give it some weight. It does seem plausible on the margin that (e.g.) orgs will be less likely to exaggerate their claims and cost-effectiveness analyses given the risk of someone posting criticism with receipts.
A softer version of this purpose could be phrased as follows: criticism is a means by which the community expresses how it expects others to act (and hopefully influences future actions by third parties even if not by the criticized organization). In your model, “public critique clearly creates barriers to starting new projects,” so one would expect public critique (or the fear thereof) to influence decisions by existing orgs as well. Then we have to decide whether that critique is on the whole good or not.
Criticism can help direct resources away from certain orgs to more productive uses. The StrongMinds-related criticisms of 2023 come to mind here. The resources could include not only funding but also mindshare (e.g., how much do I want to defer to this org?) and decisions by talent. This kind of criticism doesn’t generally pay in financial terms, so it’s reasonable to be generous in granting social credit to compensate for that. These outcomes could be measured, but doing so will often be resource-intensive and so they may not make the cut under a “clearly” standard either.
Criticism can also serve the function of market research. The usual response to people who aren’t happy about how orgs are doing their work is to go start their own org. That’s a costly response—for both the unhappy person and for the ecosystem! Suppose someone isn’t happy about CEA and EA Funds spinning off together and is thinking about trying to stand up an independent grantmaker. First off, they need to test their ideas against people who have different perspectives. They would also need to know whether a critical mass of people would move their donations over to an independent grantmaker for this or other reasons. (I think it would also be fair for someone not in a position to lead a new org to signal support for the idea, hoping that it might inspire someone else.)
It’s probably better for the market-research function to happen in public rather than in back channels. Among other things, it gives the org a chance to defend its position, and gives it a chance to adjust course if too many relevant stakeholders agree with the critic. The counterargument to this one is that little criticism actually makes it into a new organization. But I’m not sure what success rate we should expect given considerable incumbency advantage in some domains.
“People who don’t like being criticized are often going to be open about that fact”
[Just responding to this narrow point and not the comment as a whole, which contains plenty of things I agree with.]
Fwiw, I don’t think this is true in this community. Disliking criticism is a bad look and seeming responsive to criticism is highly valued. I’ve seen lots of situations up close where it would have been very aversive/costly for someone to say “I totally disagree with this criticism and think it wasn’t useful” and very tempting for someone to express lots of gratitude for criticism and change in response to it whether or not it was right. I think it’s not uncommon for the former to take more bravery than the latter and I personally feel unsure whether I’ve felt more bias towards agreeing with criticism that was wrong or disagreeing with criticism that was right.