Here’s a random org/project idea: hire full-time, thoughtful EA/AIS red teamers whose job is to seriously critique parts of the ecosystem — whether that’s the importance of certain interventions, movement culture, or philosophical assumptions. Think engaging with critics or adjacent thinkers (e.g., David Thorstad, Titotal, Tyler Cowen) and translating strong outside critiques into actionableinternal feedback.
The key design feature would be incentives: instead of paying for generic criticism, red teamers receive rolling “finder’s fees” for critiques that are judged to be high-quality, good-faith, and decision-relevant (e.g., identifying strategic blind spots, diagnosing vibe shifts that can be corrected, or clarifying philosophical cruxes that affect priorities).
Part of why I think this is important is because I generally think have the intuition that the marginal thoughtful contrarian is often more valuable than the marginal agreer, yet most movement funding and prestige flows toward builders rather than structured internal critics. If that’s true, a standing red-team org — or at least a permanent prize mechanism — could be unusually cost-effective.
There have been episodic versions of this (e.g., red-teaming contests, some longtermist critiquing stuff), but I’m not sure why this should come in waves rather than exist as ongoing infrastructure (org or just some prize pool that’s always open for sufficiently good criticisms).
While I like the potential incentive alignment, I suspect finder’s fees are unworkable. It’s much easier to promise impartiality and fairness in a single game as opposed to an iterated one, and I suspect participants relying on the fees for income would become very sensitive to the nuances of previous decisions rather than the ultimate value of their critiques.
Ultimately, I don’t think there are many shortcuts in changing the philosophy of a movement. If something is worth challenging, than people strongly believe it and there will have to be a process of contested diffusion from the outside in. You can encourage this in individual cases, but systemizing it seems difficult.
Here’s a random org/project idea: hire full-time, thoughtful EA/AIS red teamers whose job is to seriously critique parts of the ecosystem — whether that’s the importance of certain interventions, movement culture, or philosophical assumptions. Think engaging with critics or adjacent thinkers (e.g., David Thorstad, Titotal, Tyler Cowen) and translating strong outside critiques into actionable internal feedback.
The key design feature would be incentives: instead of paying for generic criticism, red teamers receive rolling “finder’s fees” for critiques that are judged to be high-quality, good-faith, and decision-relevant (e.g., identifying strategic blind spots, diagnosing vibe shifts that can be corrected, or clarifying philosophical cruxes that affect priorities).
Part of why I think this is important is because I generally think have the intuition that the marginal thoughtful contrarian is often more valuable than the marginal agreer, yet most movement funding and prestige flows toward builders rather than structured internal critics. If that’s true, a standing red-team org — or at least a permanent prize mechanism — could be unusually cost-effective.
There have been episodic versions of this (e.g., red-teaming contests, some longtermist critiquing stuff), but I’m not sure why this should come in waves rather than exist as ongoing infrastructure (org or just some prize pool that’s always open for sufficiently good criticisms).
While I like the potential incentive alignment, I suspect finder’s fees are unworkable. It’s much easier to promise impartiality and fairness in a single game as opposed to an iterated one, and I suspect participants relying on the fees for income would become very sensitive to the nuances of previous decisions rather than the ultimate value of their critiques.
Ultimately, I don’t think there are many shortcuts in changing the philosophy of a movement. If something is worth challenging, than people strongly believe it and there will have to be a process of contested diffusion from the outside in. You can encourage this in individual cases, but systemizing it seems difficult.