Chesterton Fences and EA’s X-risks

jehan2 Sep 2022 11:56 UTC
78 points
4 commentsEA link

TL;DR: EA lacks the protective norms nearly universal in mature institutions, leaving it vulnerable to the two leading causes of organizational sudden death: corruption and sex scandals.

A foundational belief in EA is the importance of unintended negative consequences; a famous actionable heuristic around this is Chesterton’s Fence:

Let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.

It’s not to say that you can never take down fences, only that you recognize what you’re losing so that you can weigh it against what you stand to gain. However, I believe EA has carelessly dismantled valuable fences while gaining comparatively little. At the very least there are low-hanging fruit fences that could have outsized returns with few costs.

1) Dismantled Fence: Undervaluing Outside Experience

This has been discussed extensively, so I won’t comment further except to note that outside experience is a rite of passage in a wide range of communities.[1]

It may be that this general lack of outside experience has prevented EA from adopting hard-won norms found elsewhere:

2) Dismantled Fence: Excessive Fraternization

Meme in EA: All your friends, roommates and partners are EAs.

This is not a good thing. Many have commented, including Will MacAskill, that this is personally suboptimal and have tried to correct it. The EA-consistent reason for this seems to either be work-life balance or viewpoint diversity.

These reasons are valid, but not the reason nearly every successful organization has rules around this: because it’s an existential risk for The Mission.

Not in some second-order way either, excessively close and incestuous relationships cause dysfunction directly and in particular leave it open to perhaps the modern era’s leading cause of organizational sudden death: sex scandals.

Successful institutions have rules around fraternization and power relationships, EA seems to largely lack explicit norms here-if anything the norm seems to actually go the other way from the mainstream.

I understand that relationships are what makes a community, but bad relationships will make a bad community while also endangering the mission.[2]

3) Dismantled Fence: Lack of Explicit Anti-Corruption Rules

You may read “corruption” and think “graft”, but that’s typically the last rung on the corruption ladder. It’s not so overt or conscious at first, so more like self-dealing, conflict of interest, and nepotism.

Few explicit rules address the problems above, and there is at least the perception that within EA they exist (but this perception could be mitigated via visible norms).[3]


Mature institutions have rules preventing these sorts of failure modes. EA however is now combining fraternization with few rules around self-dealing and favoritism to create a massive opportunity for organizational death with no comparable upside.

Fences as an Immune system

These fences not only prevent existing members from inadvertently sliding into scandal, but also act as an immune system for bad actors.

EA now has an enormous amount of power, money, and influence. When it was small and unappealing it may have gotten away with security through obscurity, but it’s now far too big of a prize to exist so immunocompromised.




(Cross-posted here: www.atvbt.com/​eacrit)

  1. ^

    Examples: Mormon missionaries, family businesses, New Zealand

  2. ^

    “But our people/​intrinsic beliefs/​other norms prevent this from being an issue!” is what pre-scandal institutions always say.

  3. ^

    Relatedly, I can say from having lived in China for many years that the perception of charities as corrupt is a major reason for comparatively low charitable giving. The potential collateral damage of an EA sex or corruption scandal should not be underestimated.

jehan2 Sep 2022 11:56 UTC
78 points
4 commentsEA link