Idea: Red-teaming fellowships

MaxRa2 Feb 2022 22:13 UTC
101 points
12 comments3 min readEA link

Summary

A red team is an independent group that challenges an organization or movement in order to improve it. Red teaming is the practice of using red teams.

  • Participants get prepared for and work on a red-teaming challenge, focussed on specific arguments, ideas, cause areas or programmes of organizations.

  • This will:

    • i) Allow fellows to dig deeply into a topic of their interest, encourage critical thinking and reduce deferring to an “EA consensus”

    • ii) Increase scrutiny of key EA ideas and lead to clarifying discussions

    • iii) Allow fellows to show useful research skills to potential employers

  • Factors to make this succesful are a good selection process for fellows, well-informed EAs to supervise the fellowship and connection to EA research organizations.

Edit: HT to Linch, who came up with the same idea in a shortform half a year ago and didn’t follow Aaron’s advice of turning it into a top-level post!

Example of a concrete red-teaming fellowship, focussing on LAWS advocacy

  1. Introducing fellows to red-teaming.

  2. 4 weeks of background reading and discussions on AI governance, autonomous lethal weapons and related policy considerations.

  3. 2 full-day red-teaming sprints scrutinizing the hypothetical report “Why EAs should work on bans of autonomous lethal weapons”.

  4. Write-up results of the sprints, receive and incorporate feedback from organizers and volunteer EA researchers, and finally share the write-up with possibly interested parties like FLI, and possibly post it on the EA forum.

Why?

  • Red-teaming fellows might be more involved and engaged compared to reading groups

  • Fellowships would use and reinforce epistemic norms

  • Red-teaming might generate useful insights, generally increase scrutiny of EA ideas, assumptions and cause prioritization

  • Identifying people that could become red-teaming or cause prioritization researchers

Who might organize this?

  • Fellowships can be organized by broadly capable local or national groups or EA orgs.

  • This will requires some ability to

    • choose good red-teaming targets

    • choose appropriate background reading material

    • (if applicable) communicate with organization whose work is scrutinized and ensure that results would be published in a constructive manner

    • give feedback

  • The fellowship would greatly benefit from mentorship and feedback from experienced EA researchers.

More concrete thoughts on implementation

Participants

  • Fellows might ideally be EAs with some solid background knowledge on EA, for example those who have finished an In-Depth Fellowship before.

  • Submission of past writing samples could be part of the application.

Structure

  • Participants are assigned to groups of 3–4 that focus on a particular topic.

  • Groups initially gather and read material on the topic (e.g. 4–8 weeks) before doing the red-teaming exercise during the second part of the fellowship.

  • Results from the exercise are written up by a chosen deadline and, if participants decide so, shared with interested parties and afterwards possibly on the EA forum

The first part of the red-teaming fellowship would be structured like a normal EA introductory fellowship, only focused on specific background reading material for the red-teaming exercise

Desiderata for the red-teaming challenge

  • The scope of the problem should fit the fellowship’s limited duration of ~2–3 workdays.

  • Facilitators should provide a variety of topics so fellows can sort according to interest.

  • Red-teaming targets should ideally be actual problems from EA researchers who would like to have an idea/​approach/​model/​conclusion/​… red-teamed against.

Some off-the-cuff examples for topics

  • “Make the best case why this recommendation of charity X should not convince a potential donor to donate.”

  • “Why might one not believe in the arguments for -

    • living at the hinge of history?”

    • shrimps mattering morally?”

    • infinite ethics being a thing?”

    • GMO regulations being relevant?”

    • fixing adolescence being a new cause area?”

  • “Why might this case against prioritizing climate change be less convincing?”

  • “Scrutinize this career profile on X. Why might it turn out to be misleading/​counterproductive/​unhelpful lecture for a young aspiring EA?”

Some reservations

We have some concerns about potential down-side risks of this idea. They probably can be averted by input from experienced community members, though.

  • Un-constructive critiques could create discontent and decrease the cooperative atmosphere in the community.

  • Similarly, this might make EA look unwelcoming and uncooperative from the outside.

  • If research isn’t supervised properly, fellows could spread hazardous information if working on sensitive subjects.

Other thoughts

  • We would encourage experimenting with paying fellows, as discussed here by Aaron_Scher:

    • the red-teaming version of an EA fellowship might alleviate some concerns about the oddness of paying for the participation because fellows are expected to put in work that can easily be understood as a service to the EA community

  • Facilitators might use Karnovsky’s minimal trust investigations as a concrete example for going deep on the case that long-lasting insecticide-treated nets (LLINs) are a cheap and effective way of preventing malaria.

  • Another good example is the red-team by AppliedDivinityStudies on Ben Todd’s post about the impact of small donations.

MaxRa2 Feb 2022 22:13 UTC
101 points
12 comments3 min readEA link