I think I agree with your overall point but some counterexamples:
EA Criticism and Red Teaming Contest winners. E.g. GiveWell said “We believe HLI’s feedback is likely to change some of our funding recommendations, at least marginally, and perhaps more importantly improve our decision-making across multiple interventions”
GiveWell said of their Change Our Mind contest “To give a general sense of the magnitude of the changes we currently anticipate, our best guess is that Matthew Romer and Paul Romer Present’s entry will change our estimate of the cost-effectiveness of Dispensers for Safe Water by very roughly 5 to 10% and that Noah Haber’s entry may lead to an overall shift in how we account for uncertainty (but it’s too early to say how it would impact any given intervention).”
HLI discussed some meaningful ways they changed as the result of criticism here.
I think I agree with your overall point but some counterexamples:
EA Criticism and Red Teaming Contest winners. E.g. GiveWell said “We believe HLI’s feedback is likely to change some of our funding recommendations, at least marginally, and perhaps more importantly improve our decision-making across multiple interventions”
GiveWell said of their Change Our Mind contest “To give a general sense of the magnitude of the changes we currently anticipate, our best guess is that Matthew Romer and Paul Romer Present’s entry will change our estimate of the cost-effectiveness of Dispensers for Safe Water by very roughly 5 to 10% and that Noah Haber’s entry may lead to an overall shift in how we account for uncertainty (but it’s too early to say how it would impact any given intervention).”
HLI discussed some meaningful ways they changed as the result of criticism here.
Those are great examples.