Deferring

Owen Cotton-Barratt12 May 2022 23:44 UTC
101 points
42 comments11 min readEA link

[meta: my attempt to find a good big-picture framing for a topic that’s clearly important but I am not convinced we’re nailing]

Deferring is when you adopt someone else’s view on a question over your own independent view (or instead of taking the time to form an independent view). You can defer on questions of fact or questions of what to do. You might defer because you think they know better (epistemic deferring), or because there is a formal or social expectation that you should go along with their view (deferring to authority).

Both types of deferring are important — epistemic deferring lets people borrow the fruits of knowledge; deferring to authority enables strong coordination. But they are two-edged. Deferring can mean that you get less chance to test out your own views, so developing mastery is slower. Deferring to the wrong people can be straightforwardly bad. And when someone defers without everyone understanding that’s what’s happening, it can cause issues. Similarly, unacknowledged expectations of deferral from others can cause problems. We should therefore learn when and how to defer, when not to, and how to be explicit about what we’re doing.

Why deferring is useful

Epistemic deferring

Epistemic deferring is giving more weight to someone else’s view than your own because you think they’re in a position to know better. The opposite of epistemic deferring is holding one’s own view.

Examples:

  • “You’ve been to this town before; where’s the best place to get coffee?”

  • “My doctor/​lawyer says this is a common situation, and the right thing to do is …”

  • “A lot of smart folks seem to think AI risk is a big deal; it sounds batshit to me, but I guess I’ll look into it more”

The case for epistemic deferring is simple: for most questions, we can identify someone (or some institution or group of people) whose judgement on the question would — if they were possessed of the facts we knew — be better than our own. So to the extent that

  • (A) We want to optimize for accurate judgements above all else, &

  • (B) We are willing to make the investment to uncover that better judgement,

deferring will be correct.

Partial deferring

The degree to which (A) and (B) hold will vary with circumstance. It will frequently be the case that they partially hold; in this case it may be appropriate to partially defer, e.g.

  • “I’m torn between whether to take job X or job Y. On my view job X seems better. When I talk to my friends and family they overwhelmingly think job Y sounds better; maybe they’re seeing something I’m not. If I thought it was a close call anyway this might be enough to tip me over, but it won’t change my mind if my preference for X was clear.”

Deferring to authority

Deferring to authority is adopting someone else’s view because of a social contract to do so. Often deferring to authority happens on questions of what should be done — e.g. “I’m going to put this fire alarm up because [my boss /​ my client /​ the law] tells me to”, or “I’m helping my friend cook dinner, so I’ll cut the carrots the way they want, even though I think this other way is better”.[1] The opposite of deferring to authority is acting on one’s own conscience.

Deferring to authority — and the reasonable expectation of such deferring — enables groups of people to coordinate more effectively. Militaries rely on it, but so do most projects (large and small, but especially large). It’s unreasonable to expect that everyone working on a large software project will have exactly the same views over the key top-level design choices, but it’s better if there’s some voice that can speak authoritatively, so everyone can work on that basis. If we collectively want to be able to undertake large ambitious projects, we’ll likely need to use deferring to authority as a tool.

Ways deferring goes wrong

  1. Deferring to the wrong people

    • The “obvious” failure mode, applies to both:

      • Epistemic deferring — misidentifying who is an expert

      • Deferring to authority — buying into social contracts it would be better to withdraw from

  2. Deferring with insufficient bandwidth

    • Even if Aditi would make a better decision than Sarah, the process of Sarah deferring to Aditi (for epistemic or authority reasons) can produce a worse decision if either:

      1. There’s too much context for Sarah to communicate to Aditi

      2. The “right” decision includes too much detail for Aditi to communicate to Sarah

    • This is more often a problem with questions of what to do than questions of fact (since high context on the situation is so often important for the answer), but may come up in either case

    • A special case is deferring with zero bandwidth (e.g. Sarah is deferring to she imagines Aditi would say in the situation, based on an article she read)

    • Another cause of deferring with insufficient bandwidth is if someone wants to delegate responsibility but not authority for a project, and not to spend too much time on it; this is asking for deferral to them as an authority without providing much bandwidth

  3. Deferring can be bad for learning

    • Contrast — “letting people make their own mistakes”

      • The basic dynamic is that if you act from your own models, you bring them more directly into contact with the world, and can update faster

    • Note that a certain amount of deferring can be good for learning, especially:

      1. When first trying to get up to speed with an area

      2. When taking advice on what to pay attention to

        • In particular because this can help rescue people from traps where they think some dimension is unimportant, so never pay attention to it to notice that it’s actually important

    • This intersects with #2; deferring is more often good for learning when it’s high-bandwidth (since the person deferring can use it as an opportunity to interrogate the model of the person being deferred to), and more often bad for learning when it’s low-bandwidth

  4. Deferring can interfere with belief formation

    • If people aren’t good at keeping track of why they believe things, it can be hard to notice when one’s body of knowledge has caught up and one should stop deferring on an issue (because the deferred-to-belief may be treated as a primitive belief); cf. independent impressions for discussion of habits for avoiding this

    • Conflation between epistemic deferring and deferring to authority can lead to people accidentally adopting as beliefs things that were only supposed to be operating assumptions

      • This can happen e.g.

        • When deferring to one’s boss

          • Easy to slip between the two since one’s boss is often in a superior epistemic position re. what needs to be done

          • In some cases organizational leadership might exert explicit pressure towards shared beliefs, e.g. saying “if someone doesn’t look like they hold belief X, this could destabilize the team’s ability to orient together as a team”

        • Deferring to someone high status when the true motivation for deferring is to seem like one has cool beliefs /​ get social acceptance

          • Again there’s plausible deniability since the high status person may well be in a superior epistemic position

          • The high-status person may like it when others independently have similar views to them (since this is evidence of good judgement), which can create incentives for the junior people to adopt “as their own view” the relevant positions

Deferring without common knowledge of deferring is a risk factor for these issues (since it’s less likely that anyone is going to spot and correct them).

Social deferring

Often there’s a lot of deferring within a group or community on a particular issue (i.e. both the person deferring and the person being deferred to are within the group, and the people being deferred to often have their own views substantially via deferring). This can lead to issues, for reasons like:

  1. If there are long chains of deferral, this means there’s often little bandwidth to the people originating the views

  2. If you don’t know when others are deferring vs having independent views, it may be unclear how many times a given view has been independently generated, which can make it hard to know how much weight to put on it (“the emperor’s new clothes” gives an extreme example)

  3. If the people with independent takes update their views in response to evidence, it may take some time until the newer views have filtered through to the people who are deferring

  4. If people are deferring to the authority of the social group (where there’s a pressure to have the view as a condition of membership), this may be bad for belief formation

Ultimately we don’t have good alternatives to basing a lot of our beliefs on chains of deferral (there are too many disparate disciplines of expertise in the world to personally be fluent with knowing who are the experts to listen to in each one). But I think it’s helpful to be wary of ways in which it can cause problems, and we should feel relatively better about:

  • A group or community collectively deferring to a single source (e.g. the same expert report, or a prediction market), as it’s much more legible what’s happening

  • People sometimes taking the effort to dive into a topic and shorten the deferral chain (cf. “minimal trust investigations”)

  • Creating spaces which explicitly state their operating assumptions as a condition of entry (“in this workshop we’ll discuss how to prepare for a nuclear war in 2025”) without putting pressure on the beliefs of the participants

When & how to defer

Epistemic deferring

There’s frequently a tension between on the one hand knowing that you can identify someone who knows more than you, and on the other hand not wanting to take the time to get answers from them, or wanting to optimize for your own learning rather than just the best answer for the question at hand.

Here are the situations where I think epistemic deferring is desirable:

  1. Early in the learning process for any paradigm

    • By “paradigm” I mean a body of knowledge with something like agreed-on metrics of progress

      • This might include “learning a new subfield of chemistry” or “learning to ride a unicycle”

      • I’m explicitly not including areas that feel preparadigmatic — among which notably I want to include cause prioritization — where I feel more confused about the correct advice (although it certainly seems helpful to hear existing ideas)

    • Here you ideally want to defer-but-question — perhaps you assume that the thing you’re being told is correct, but are intensely curious about why that could be (and remain open to questioning the assumption later)

    • Taking advice on what to pay attention to is a frequent special case of this — it’s very early in the learning process of “how to pay attention to X”, for some X you previously weren’t giving attention to

  2. When the importance of a good answer is reasonably high compared to the cost of gathering the information about how to defer, and either:

    1. It’s on a topic that you’re not hoping to develop mastery of

      • i.e. you just want the easily-transmissible conclusions, not the underlying generators

    2. There are only weak feedback loops from the process back into your own models

    3. The importance of a good answer is high even compared to the cost of gathering thorough information about how to defer

      • Sometimes thorough information about how to defer is cheap! e.g. if you want to know about a variable that has high quality public expert estimates

      • If you’re making a decision about what to do, however, often gathering thorough information about how to defer means very high bandwidth context-sharing

    4. You intend to defer only a little

Note: even when not deferring, asking for advice is often a very helpful move. You can consider the advice and let it guide your thinking and how to proceed without deferring to any of the advice-givers.[2]

Deferring to authority

Working out when to defer to authority is often simply a case of determining whether you want to participate in the social contract.

It’s often good to communicate when you’re deferring, e.g. tell your boss “I’m doing X because you told me to, but heads up that Y looks better to me”. Sometimes the response will just be “cool”; at other times they might realize that you need to understand why X is good in order to do a good job of X (or that they need to reconsider X). In any case it’s helpful to keep track for yourself of when you’re deferring to authority vs have an independent view.

A dual question of when to defer to authority is when to ask people to defer to you as an authority. I think the right answer is “when you want someone to go on following the plan even if they’re not personally convinced”. If you’re asking others to defer it’s best if you’re explicit about this. Vice-versa if you’re in a position of authority and not asking others to defer it’s good to be explicit that you want them to act on their own conscience. (People take cultural cues from those in positions of authority; if they perceive ambiguity about whether they should defer, it may be ambiguous in their own mind, which seems bad for the reasons discussed above.)

Deferring to authority in the effective altruism community

I think people are often reluctant to ask others to defer to their authority within EA. We celebrate people thinking for themselves, taking a consequentialist perspective, and acting on their own conscience. Deferring to authority looks like it might undermine these values. Or perhaps we’d get people who reluctantly “deferred to authority” while trying to steer their bosses towards things that seemed better to them.

This is a mistake. Deferring to authority is the natural tool for coordinating groups of people to do big things together. If we’re unwilling to use this tool, people will use social pressure towards conformity of beliefs as an alternate tool for the same ends. But this is worse at achieving coordination[3], and is more damaging to the epistemics of the people involved.

We should (I think) instead encourage people to be happy taking jobs where they adopt a stance of “how can I help with the agenda of the people steering this?”, without necessarily being fully bought into that agenda. This might seem a let down for individuals, but I think we should be willing to accept more “people work on agendas they’re not fully bought into” if the alternatives are “there are a bunch of epistemic distortions to get people to buy into agendas” and “nobody can make bets which involve coordinating more than 6 people”. People doing this can keep their eyes open for jobs which better fit their goals, while being able and encouraged to have their own opinions, and still having professional pride in doing a good job at the thing they’re employed to do.

This isn’t to say that all jobs in EA should look like this. I think it is a great virtue of the community that we recognise the power of positions which give people significant space to act on their own conscience. But when we need more coordination, we should use the correct tools to get that.

Meta-practices

My take on the correct cultural meta-practices around deferring:

  1. Choices to defer — or to request deferral — should as far as possible be made deliberately rather than accidentally

    • We should be conscious of whether we’re deferring for epistemic or authority reasons

  2. We should discuss principles of when to defer and when not to defer

  3. Responsibility for encouraging non-deferral (when that’s appropriate) should lie significantly with the people who might be deferred to

  4. We should be explicit about when we’re deferring (in particular striving not to let the people-being-deferred-to remain ignorant of what’s happening)

Closing remarks

A lot of this content, insofar as it is perceptive, is not original to me; a good part of what I’m doing here is just trying to name the synthesis position for what I perceive to be strong pro-deferral and anti-deferral arguments people make from time to time. This draft benefited from thoughts and comments from Adam Bales, Buck Shlegeris, Claire Zabel, Gregory Lewis, Jennifer Lin, Linch Zhang, Max Dalton, Max Daniel, Raymond Douglas, Rose Hadshar, Scott Garrabrant, and especially Anna Salamon and Holden Karnofsky. I might edit later to tighten or clarify language (or if there are one or two substantive points I want to change).

Should anyone defer to me on the topic of deferring?

Epistemically — I’ve spent a while thinking about the dynamics here, so it’s not ridiculous to give my views some weight. But lots of people have spent some time on this; I’m hoping this article is more helpful as a guide to let people understand things they already see than as something that needs people to defer to.

As an authority — not yet. But I’m offering suggestions for community norms around deferring. Norms are a thing which it can make sense to ask people to defer to. If my suggestions are well received in the discussion here, perhaps we’ll want to make asks for deference to them at some point down the line.

  1. ^

    Some less central examples of deferring to authority in my sense:

    • Doing something because you promised to (the “authority” deferred to is your past self)

    • Adopting a belief that the startup you’re joining will succeed as part of the implicit contract of joining (not necessarily a fully adopted belief, but acted upon while at work)

  2. ^

    cf. https://​​www.lesswrong.com/​​posts/​​yeADMcScw8EW9yxpH/​​a-sketch-of-good-communication

  3. ^

    At least “just using ideological conformity” is worse for coordination than “using ideological conformity + deference to authority”. After we’re using deference to authority well I imagine there’s a case that having ideological conformity as well would help further; my guess is that it’s not worth the cost of damage to epistemics.

Owen Cotton-Barratt12 May 2022 23:44 UTC
101 points
42 comments11 min readEA link