Pre-announcing a contest for critiques and red teaming

Lizka25 Mar 2022 11:52 UTC
173 points
27 comments2 min readEA link

June 1, 2022 update: the contest is now posted. We’ve also compiled a separate resource for criticisms and red teaming.

We’re writing this post to say that we’re interested in running a contest on the Forum for writing that critically engages with theory or work from the EA community.

We are: Lizka (the Content Specialist at CEA), Joshua, and Fin.

Consider this a pre-announcement: we’re still figuring out the details, but we want to commit right now to supporting high-quality critical work within EA — including critiques, questioning, red teaming, and minimal-trust investigations. We think a contest could make a good start.

Why do we think this matters? In short, we think there are some reasons to expect good criticism to be undersupplied relative to its real value. And that matters: as EA grows, it’s going to become increasingly important that we scrutinize the ideas and assumptions behind key decisions — and that we welcome outside experts to do the same.[1]

So we want to run a contest to incentivize and reward the most thoughtful and most action-relevant critical work (cross-)posted to the Forum.[2] The Creative Writing Contest last year would be a good point of comparison, but we’re hopeful the prize pool will be significantly larger this time — perhaps up to around $100k if enough submissions really meet the bar. The judging panel is taking shape already, and we’re looking forward to sharing it.

We’re excited to read about errors in widely-cited analyses, gaps in EA’s philosophical toolset, missing crucial considerations, independent reassessments of key claims, evaluations of EA organizations, suggestions for how to further improve community norms and institutions, and efforts to communicate and ‘steelman’ existing criticisms for an EA audience.

The judging criteria aren’t finalized, but we’re imagining an ideal submission would be:

  • Critical — the piece takes a critical or questioning stance towards some aspect of EA, theory or practice;

  • Important — the issues discussed really matter for our ability to do the most good as a movement;

  • Novel — the piece presents new arguments, or otherwise presents familiar ideas in a new way;

  • Constructive — we can see how the work is decision-relevant (directly or indirectly). Bonus points for concrete solutions.

We don’t expect that every winning piece needs to do well at every one of these criteria, but we do value them all.

We also want to reward reasoning transparency /​ ‘epistemic legibility’, clarity of writing, ‘punching up’, awareness of context, and a scout mindset. On the other hand, we don’t want to encourage personal attacks or diatribes that are likely to produce much more heat than light. And we hope that subject-matter experts who don’t typically associate with EA find out about this and share insights we haven’t yet heard.

Should you hold off on posting until the contest is properly announced in order to ensure eligibility? You don’t have to: if we go ahead, we’ll accept pieces posted from this week (beginning March 21st, 2022) onwards.[3]

Another reason we’re ‘pre-announcing’ this idea is to hear some initial thoughts from Forum users. We think it’s especially important to get the details right here, so we’d be interested to hear what considerations we should bear in mind, that we might otherwise miss. Thanks!

  1. ^

    Joshua Teperowski Monrad elaborates in his post ‘Rowing and Steering the Effective Altruism Movement’. Fin Moorhouse describes the rationale for a contest in his ‘EA Project Ideas’ post. We also recommend Linch’s shortform post on ‘red teaming papers as an EA training exercise’.

  2. ^

    This should complement, rather than duplicate, the ‘Red Team Challenge’ recently announced by Training for Good.

  3. ^

    Of course, it might make sense to wait to read the full announcement and the finalized judging criteria.

Lizka25 Mar 2022 11:52 UTC
173 points
27 comments2 min readEA link