if you make a decision with large-scale and irreversible effects on the world (e.g. “who should get this $5M grant?”) I think it would usually be predictably worse for the world to ignore others’ views
Taking into account specific facts or arguments made by other people seems reasonable here. Just writing down e.g. “person X doesn’t like MIRI” in the “cons” column of your spreadsheet seems foolish and wrongheaded.
Framing it as “taking others’ views into account” or “ignoring others’ views” is a big part of the problem, IMO—that language itself directs people towards evaluating the people rather than the arguments, and overall opinions rather than specific facts or claims.
I think we disagree. I’m not sure why you think that even for decisions with large effects one should only or mostly take into account specific facts or arguments, and am curious about your reasoning here.
I do think it will often be even more valuable to understand someone’s specific reasons for having a belief. However, (i) in complex domains achieving a full understanding would be a lot of work, (ii) people usually have incomplete insight into the specific reasons for why they hold a certain belief themselves and instead might appeal to intuition, (iii) in practice you only have so much time and thus can’t fully pursue all disagreements.
So yes, always stopping at “person X thinks that p” and never trying to understand why would be a poor policy. But never stopping at that seems infeasible to me, and I don’t see the benefits from always throwing away the information that X believes p in situations where you don’t fully understand why.
For instance, imagine I pointed a gun to your head and forced you to now choose between two COVID mitigation policies for the US for the next 6 months. I offer you to give you additional information of the type “X thinks that p” with some basic facts on X but no explanation for why they hold this belief. Would you refuse to view that information? If someone else was in that situation, would you pay for me not giving them this information? How much?
There is a somewhat different failure mode where person X’s view isn’t particularly informative compared to the view of other people Y, Z, etc., and so by considerung just X’s view you give it undue weight. But I don’t think you’re talking about that?
I’m partly puzzled by your reaction because the basic phenomenon of deferring to the output of others’ reasoning processes without understanding the underlying facts or arguments strikes me as not unusual at all. For example, I believe that the Earth orbits the Sun rather than the other way around. But I couldn’t give you any very specific argument for this like “on the geocentric hypothesis, the path of this body across the sky would look like this”. Instead, the reason for my belief is that the heliocentric worldview is scientific consensus, i.e. epistemic deference to others without understanding their reasoning.
This also happens when the view in question makes a difference in practice. For instance, as I’m sure you’re aware, hierarchical organizations work (among other things) because managers don’t have to recapitulate every specific argument behind the conclusions of their reports.
To sum up, a very large amount of division of epistemic labor seems like the norm rather than the exception to me, just as for the division of manual labor. The main thing that seems somewhat unusual is making that explicit.
I note that the framing / example case has changed a lot between your original comment / my reply (making a $5m grant and writing “person X is skeptical of MIRI” in the “cons” column) and this parent comment (“imagine I pointed a gun to your head and… offer you to give you additional information;” “never stopping at [person X thinks that p]”). I’m not arguing for entirely refusing to trust other people or dividing labor, as you implied there. I specifically object to giving weight to other people’s top-line views on questions where there’s substantial disagreement, based on your overall assessment of that particular person’s credibility / quality of intuition / whatever, separately from your evaluation of their finer-grained sub-claims.
If you are staking $5m on something, it’s hard for me to imagine a case where it makes sense to end up with an important node in your tree of claims whose justification is “opinions diverge on this but the people I think are smartest tend to believe p.” The reason I think this is usually bad is that (a) it’s actually impossible to know how much weight it’s rational to give someone else’s opinion without inspecting their sub-claims, and (b) it leads to groupthink/herding/information cascades.
As a toy example to illustrate (a): suppose that for MIRI to be the optimal grant recipient, it both needs to be the case that AI risk is high (A) and that MIRI is the Best organization working to mitigate it (B). A and B are independent. The prior is (P(A) = 50, P(B) = 50). Alice and Bob have observed evidence with a 9:1 odds ratio in favor of A, so think (P(A) = 90, P(B) = 50). Carol has observed evidence with a 9:1 odds ratio in favor of B. Alice, Bob and Carol all have the same top-line view of MIRI (P(A and B) = 0.45), but the rational aggregation of Alice and Bob’s “view” is much less positive than the rational aggregation of Bob and Carol’s.
It’s interesting that you mention hierarchical organizations because I think they usually follow a better process for dividing up epistemic labor, which is to assign different sub-problems to different people rather than by averaging a large number of people’s beliefs on a single question. This works better because the sub-problems are more likely to be independent from each other, so they don’t require as much communication / model-sharing to aggregate their results.
In fact, when hierarchical organizations do the other thing—”brute force” aggregate others’ beliefs in situations of disagreement—it usually indicates an organizational failure. My own experience is that I often see people do something a particular way, even though they disagree with it, because they think that’s my preference; but it turns out they had a bad model of my preferences (often because they observed a contextual preference in a different context) and would have been better off using their own judgment.
I think I perceive less of a difference between the examples we’ve been discussing, but after reading your reply I’m also less sure if and where we disagree significantly.
I read your previous claim as essentially saying “it would always be bad to include the information that some person X is skeptical about MIRI when making the decision whether to give MIRI a $5M grant, unless you understand more details about why X has this view”.
I still think this view basically commits you to refusing to see information of that type in the COVID policy thought experiment. This is essentially for the reasons (i)-(iii) I listed above: I think that in practice it will be too costly to understand the views of each such person X in more detail.
(But usually it will be worth it to do this for some people, for instance for the reason spelled out in your toy model. As I said: I do think it will often be even more valuable to understand someone’s specific reasons for having a belief.)
Instead, I suspect you will need to focus on the few highest-priority cases, and in the end you’ll end up with people X1,…,Xl whose views you understand in great detail, people Y1,…,Ym where your understanding stops at other fairly high-level/top-line views (e.g. maybe you know what they think about “will AGI be developed this century?” but not much about why), and people Z1,…,Zn of whom you only know the top-line view of how much funding they’d want to give to MIRI.
At that point, I think you’re basically in a similar situation. There is no gun pointed at your head, but you still want to make a decision right now, and so you can either throw away the information about the views of person Zi or use it without understanding their arguments.
Furthermore, I don’t think your situation with respect to person Yj is that different: if you take their view on “AGI this century?” into account for the decision whether to fund MIRI but have a policy of never using “bare top-level views”, this would commit to to ignoring the same information in a different situation, e.g. the decision whether to place a large bet on whether AGI will be developed this century (purely because what’s a top-level view in one situation will be an argument or “specific” fact in another); this seems odd.
(This is also why I’m not sure I understand the relevance of your point on hierarchical organizations. I agree that usually sub-problems will be assigned to different employees. But e.g. if I assign “AGI this century?” to one employee and “is MIRI well run?” to another employee, why am I justified in believing their conclusions on these fairly high-level questions but not justified in believing anyone’s view on whether MIRI is worth funding?)
Note that thus far I’m mainly arguing against taking into account no-one’s top-level views. Your most recent claim involving “the people I think are smartest” suggests that maybe you mainly object to using a lot of discretion in which particular people’s top-level views to use.
I think my reaction to this is mixed: On one hand, I certainly agree that there is a danger involved here (e.g. in fact I think that many EAs defer too much to others EAs relative to non-EA experts), and that it’s impossible to assess with perfect accuracy how much weight to give to each person. On the other hand, I think it is often possible to assess this with limited but still useful accuracy, both based on subjective and hard-to-justify assessments of how good someone’s judgment seemed in the past (cf. how senior politicians often work with advisors they’ve had a long work relationship with) and on crude objectives proxies (e.g. ‘has a PhD in computer science’).
On the latter, you said that specifically you object to allocating weight to someone’s top-line opinion “separately from your evaluation of their finer-grained sub-claims”. If that means their finer-grained sub-claims on the particular question under consideration, then I disagree for the reasons explained so far. If that means “separately from your evaluation of any finer-grained sub-claim they ever made on anything”, then I agree more with this, though still think this is both common and justified in some cases (e.g. if I learn that I have rare disease A for which specialists universally recommend drug B as treatment, I’ll probably happily take drug B without having ever heard of any specific sub-claim made by any disease-A specialist).
Similarly, I agree that information cascades and groupthink are dangers/downsides, but that they will sometimes be outweighed by the benefits.
If 100 forecasters (who I roughly respect) look at the likelihood of a future event and think it’s ~10% likely, and I look at the same question and think it’s ~33% likely, I think I will be incorrect in my private use of reason for my all-things-considered-view to not update somewhat downwards from 33%.
I think this continues to be true even if we all in theory have access to the same public evidence, etc.
Now, it does depend a bit on the context of what this information is for. For example if I’m asked to give my perspective on a group forecast (and I know that the other 100 forecasters’ predictions will be included anyway), I think it probably makes sense for me to continue to publicly provide ~33% for that question to prevent double-counting and groupthink.
But I think it will be wrong for me to believe 33%, and even more so, wrong to say 33% in a context where somebody else doesn’t have access to the 100 other forecasters.
An additional general concern here to me is computational capacity/kindness—sometimes (often) I just don’t have enough time to evaluate all the object-level arguments! You can maybe argue that until I evaluate all the object-level arguments, I shouldn’t act, yet in practice I feel like I act with lots of uncertainty* all the time! ___
One disagreement I have with Max is whether someone should defer is contingent upon the importance of a decision. I think this begs the question in that it pre-assumes that deference lead to the best outcomes.
Instead, I think you should act such that you all-things-considered-view is that you’re making the best decision. I do think that for many decisions (with the possible exception of creative work), some level of deference leads to better outcomes than zero deference at all, but I don’t think it’s unusually true for important decisions except inasmuch as a) the benefits (and also costs!) of deference are scaled accordingly and b) more people are likely to have thought about important decisions.
__ * Narrow, personal, example that’s basically unrelated to EA: I brush my teeth with fluoride toothpaste. I don’t floss. Why? Cochrane review was fairly equivocal about flossing and fairly certain about toothbrushing. Maybe it’d be more principled if I looked at the data myself and performed my own meta-analysis on the data, or perhaps self-experimented like Gwern, to decide what dental hygiene activities I should take. But in practice I feel like it’s a reasonable decision procedure to just defer to Cochrane review on the empirical facts of the matter, and apply my own value judgments on what activities to take given the facts available.
One disagreement I have with Max is whether someone should defer is contingent upon the importance of a decision. I think this begs the question in that it pre-assumes that deference lead to the best outcomes.
Instead, I think you should act such that you all-things-considered-view is that you’re making the best decision. I do think that for many decisions (with the possible exception of creative work), some level of deference leads to better outcomes than zero deference at all, but I don’t think it’s unusually true for important decisions except inasmuch as a) the benefits (and also costs!) of deference are scaled accordingly and b) more people are likely to have thought about important decisions.
I’m not sure if we have a principled disagreement here, it’s possible that I just described my view badly above.
I agree that one should act such that one’s all-things-considered view is that one is making the best decision (the way I understand that statement it’s basically a tautology).
Then I think there are some heuristics for which features of a decision situation make it more or less likely that deferring more (or at all) leads to decisions with that property. I think on a high level I agree with you that it depends a lot “on the context of what this information is for”, more so than on e.g. importance.
With my example, I was also trying to point less to importance per se but on something like how the costs and benefits are distributed between yourself and others. This is because very loosely speaking I expect not deferring to often be better if the stakes are concentrated on oneself and more deference to be better if one’s own direct stake is small. I used a decision with large effects on others largely because then it’s not plausible that you yourself are affected by a similar amount; but it would also apply to a decision with zero effect on yourself and a small effect on others. Conversely, it would not apply to a decision that is very important to yourself (e.g. something affecting your whole career trajectory).
Apologies for the long delay in response, feel free to not reply if you’re busy.
Hmm I still think we have a substantive rather than framing disagreement (though I think it is likely that our disagreements aren’t large).
This is because very loosely speaking I expect not deferring to often be better if the stakes are concentrated on oneself and more deference to be better if one’s own direct stake is small. I used a decision with large effects on others largely because then it’s not plausible that you yourself are affected by a similar amount; but it would also apply to a decision with zero effect on yourself and a small effect on others. Conversely, it would not apply to a decision that is very important to yourself (e.g. something affecting your whole career trajectory).
Perhaps this heuristic is really useful for a lot of questions you’re considering. I’m reminded of AGB’s great quote:
There are enough individual and practical considerations here (in both directions) that in many situations the actual thing I would advocate for is something like “work out what you would do with both approaches, check against results ‘without fear or favour’, and move towards whatever method is working best for you”.
For me personally and the specific questions I’ve considered, I think considering whether/how much to defer to by dividing into buckets of “how much it affects myself or others” is certainly a pretty useful heuristic in the absence of better heuristics, but it’s mostly superseded by a different decomposition:
Epistemic—In a context-sensitive manner, do we expect greater or lower deference in this particular situation to lead to more accurate beliefs.
Role expectations* -- Whether the explicit and implicit social expectations on the role you’re assuming privilege deference or independence.
So I think a big/main reason it’s bad to defer completely to others (say 80k) on your own career reasons is epistemic: you have so much thought and local knowledge about your own situation that your prior should very strongly be against others having better all-things-considered views on your career choice than you do. I think this is more crux-y for me than how much your career trajectory affects yourself vs others (at any rate hopefully as EAs our career trajectories affect many others anyway!).
On the other hand, I think my Cochrane review example above is a good epistemic example of deference. even though my dental hygiene practices mainly affect myself and not others (perhaps my past and future partners may disagree), I contend it’s better to defer to the meta-analysis over my own independent analysis in this particular facet of my personal life.
The other main (non-epistemic) lens I’d use to privilege greater or lower humility is whether the explicit and implicit social expectations privilege deference or independence. For example, we’d generally** prefer government bureaucrats in most situations to implement policies, rather than making unprincipled exceptions based on private judgements. This will often look superficially similar to “how much this affects myself or others.”
An example of a dissimilarity is when someone filling out a survey. This is a situation where approximately all of the costs and benefits are borne by other people. So if you have a minority opinion on a topic, it may seem like the epistemically humble-and-correct action is to fill out the poll according to what you believe the majority to think (or alternatively, fill it out with the answer that you privately think is on the margin more conducive to advancing your values).
But in all likelihood, such a policy is one-thought-too-many, and in almost all situations it’d be more prudent to fill out public anonymous polls/surveys with what you actually believe.
I agree that one should act such that one’s all-things-considered view is that one is making the best decision (the way I understand that statement it’s basically a tautology).
Agreed, though I mention this because in discussions of epistemic humility-in-practice, it’s very easy to accidentally do double-counting.
*I don’t like this phrase, happy to use a better one.
**I’m aware that there are exceptions, including during the ongoing coronavirus pandemic.
Thanks! I’m not sure if there is a significant difference about how we’d actually make decisions (I mean, on prior there is probably some difference). But I agree that the single heuristics I mentioned above doesn’t by itself do a great job of describing when and how much to defer, and I agree with your “counterexamples”. (Though note that in principle it’s not surprising if there are counterexamples to a “mere heuristics”.)
I particularly appreciate you describing the “Role expectations” point. I agree that something along those lines is important. My guess is that if we would have debated specific decisions I would have implicitly incorporated this consideration, but I don’t think it was clear to me before reading your comment that this is an important property that will often influence my judgment about how much to defer.
Taking into account specific facts or arguments made by other people seems reasonable here. Just writing down e.g. “person X doesn’t like MIRI” in the “cons” column of your spreadsheet seems foolish and wrongheaded.
Framing it as “taking others’ views into account” or “ignoring others’ views” is a big part of the problem, IMO—that language itself directs people towards evaluating the people rather than the arguments, and overall opinions rather than specific facts or claims.
I think we disagree. I’m not sure why you think that even for decisions with large effects one should only or mostly take into account specific facts or arguments, and am curious about your reasoning here.
I do think it will often be even more valuable to understand someone’s specific reasons for having a belief. However, (i) in complex domains achieving a full understanding would be a lot of work, (ii) people usually have incomplete insight into the specific reasons for why they hold a certain belief themselves and instead might appeal to intuition, (iii) in practice you only have so much time and thus can’t fully pursue all disagreements.
So yes, always stopping at “person X thinks that p” and never trying to understand why would be a poor policy. But never stopping at that seems infeasible to me, and I don’t see the benefits from always throwing away the information that X believes p in situations where you don’t fully understand why.
For instance, imagine I pointed a gun to your head and forced you to now choose between two COVID mitigation policies for the US for the next 6 months. I offer you to give you additional information of the type “X thinks that p” with some basic facts on X but no explanation for why they hold this belief. Would you refuse to view that information? If someone else was in that situation, would you pay for me not giving them this information? How much?
There is a somewhat different failure mode where person X’s view isn’t particularly informative compared to the view of other people Y, Z, etc., and so by considerung just X’s view you give it undue weight. But I don’t think you’re talking about that?
I’m partly puzzled by your reaction because the basic phenomenon of deferring to the output of others’ reasoning processes without understanding the underlying facts or arguments strikes me as not unusual at all. For example, I believe that the Earth orbits the Sun rather than the other way around. But I couldn’t give you any very specific argument for this like “on the geocentric hypothesis, the path of this body across the sky would look like this”. Instead, the reason for my belief is that the heliocentric worldview is scientific consensus, i.e. epistemic deference to others without understanding their reasoning.
This also happens when the view in question makes a difference in practice. For instance, as I’m sure you’re aware, hierarchical organizations work (among other things) because managers don’t have to recapitulate every specific argument behind the conclusions of their reports.
To sum up, a very large amount of division of epistemic labor seems like the norm rather than the exception to me, just as for the division of manual labor. The main thing that seems somewhat unusual is making that explicit.
I note that the framing / example case has changed a lot between your original comment / my reply (making a $5m grant and writing “person X is skeptical of MIRI” in the “cons” column) and this parent comment (“imagine I pointed a gun to your head and… offer you to give you additional information;” “never stopping at [person X thinks that p]”). I’m not arguing for entirely refusing to trust other people or dividing labor, as you implied there. I specifically object to giving weight to other people’s top-line views on questions where there’s substantial disagreement, based on your overall assessment of that particular person’s credibility / quality of intuition / whatever, separately from your evaluation of their finer-grained sub-claims.
If you are staking $5m on something, it’s hard for me to imagine a case where it makes sense to end up with an important node in your tree of claims whose justification is “opinions diverge on this but the people I think are smartest tend to believe p.” The reason I think this is usually bad is that (a) it’s actually impossible to know how much weight it’s rational to give someone else’s opinion without inspecting their sub-claims, and (b) it leads to groupthink/herding/information cascades.
As a toy example to illustrate (a): suppose that for MIRI to be the optimal grant recipient, it both needs to be the case that AI risk is high (A) and that MIRI is the Best organization working to mitigate it (B). A and B are independent. The prior is (P(A) = 50, P(B) = 50). Alice and Bob have observed evidence with a 9:1 odds ratio in favor of A, so think (P(A) = 90, P(B) = 50). Carol has observed evidence with a 9:1 odds ratio in favor of B. Alice, Bob and Carol all have the same top-line view of MIRI (P(A and B) = 0.45), but the rational aggregation of Alice and Bob’s “view” is much less positive than the rational aggregation of Bob and Carol’s.
It’s interesting that you mention hierarchical organizations because I think they usually follow a better process for dividing up epistemic labor, which is to assign different sub-problems to different people rather than by averaging a large number of people’s beliefs on a single question. This works better because the sub-problems are more likely to be independent from each other, so they don’t require as much communication / model-sharing to aggregate their results.
In fact, when hierarchical organizations do the other thing—”brute force” aggregate others’ beliefs in situations of disagreement—it usually indicates an organizational failure. My own experience is that I often see people do something a particular way, even though they disagree with it, because they think that’s my preference; but it turns out they had a bad model of my preferences (often because they observed a contextual preference in a different context) and would have been better off using their own judgment.
I think I perceive less of a difference between the examples we’ve been discussing, but after reading your reply I’m also less sure if and where we disagree significantly.
I read your previous claim as essentially saying “it would always be bad to include the information that some person X is skeptical about MIRI when making the decision whether to give MIRI a $5M grant, unless you understand more details about why X has this view”.
I still think this view basically commits you to refusing to see information of that type in the COVID policy thought experiment. This is essentially for the reasons (i)-(iii) I listed above: I think that in practice it will be too costly to understand the views of each such person X in more detail.
(But usually it will be worth it to do this for some people, for instance for the reason spelled out in your toy model. As I said: I do think it will often be even more valuable to understand someone’s specific reasons for having a belief.)
Instead, I suspect you will need to focus on the few highest-priority cases, and in the end you’ll end up with people X1,…,Xl whose views you understand in great detail, people Y1,…,Ym where your understanding stops at other fairly high-level/top-line views (e.g. maybe you know what they think about “will AGI be developed this century?” but not much about why), and people Z1,…,Zn of whom you only know the top-line view of how much funding they’d want to give to MIRI.
(Note that I don’t think this is hypothetical. My impression is that there are in fact long-standing disagreements about MIRI’s work that can’t be fully resolved or even broken down into very precise subclaims/cruxes, despite many people having spent probably hundreds of hours on this. For instance, in the writeups to their first grants to MIRI, Open Phil remark that “We found MIRI’s work especially difficult to evaluate”, and the most recent grant amount was set by a committee that “average[s] individuals’ allocations” . See also this post by Open Phil’s Daniel Dewey and comments.)
At that point, I think you’re basically in a similar situation. There is no gun pointed at your head, but you still want to make a decision right now, and so you can either throw away the information about the views of person Zi or use it without understanding their arguments.
Furthermore, I don’t think your situation with respect to person Yj is that different: if you take their view on “AGI this century?” into account for the decision whether to fund MIRI but have a policy of never using “bare top-level views”, this would commit to to ignoring the same information in a different situation, e.g. the decision whether to place a large bet on whether AGI will be developed this century (purely because what’s a top-level view in one situation will be an argument or “specific” fact in another); this seems odd.
(This is also why I’m not sure I understand the relevance of your point on hierarchical organizations. I agree that usually sub-problems will be assigned to different employees. But e.g. if I assign “AGI this century?” to one employee and “is MIRI well run?” to another employee, why am I justified in believing their conclusions on these fairly high-level questions but not justified in believing anyone’s view on whether MIRI is worth funding?)
Note that thus far I’m mainly arguing against taking into account no-one’s top-level views. Your most recent claim involving “the people I think are smartest” suggests that maybe you mainly object to using a lot of discretion in which particular people’s top-level views to use.
I think my reaction to this is mixed: On one hand, I certainly agree that there is a danger involved here (e.g. in fact I think that many EAs defer too much to others EAs relative to non-EA experts), and that it’s impossible to assess with perfect accuracy how much weight to give to each person. On the other hand, I think it is often possible to assess this with limited but still useful accuracy, both based on subjective and hard-to-justify assessments of how good someone’s judgment seemed in the past (cf. how senior politicians often work with advisors they’ve had a long work relationship with) and on crude objectives proxies (e.g. ‘has a PhD in computer science’).
On the latter, you said that specifically you object to allocating weight to someone’s top-line opinion “separately from your evaluation of their finer-grained sub-claims”. If that means their finer-grained sub-claims on the particular question under consideration, then I disagree for the reasons explained so far. If that means “separately from your evaluation of any finer-grained sub-claim they ever made on anything”, then I agree more with this, though still think this is both common and justified in some cases (e.g. if I learn that I have rare disease A for which specialists universally recommend drug B as treatment, I’ll probably happily take drug B without having ever heard of any specific sub-claim made by any disease-A specialist).
Similarly, I agree that information cascades and groupthink are dangers/downsides, but that they will sometimes be outweighed by the benefits.
If 100 forecasters (who I roughly respect) look at the likelihood of a future event and think it’s ~10% likely, and I look at the same question and think it’s ~33% likely, I think I will be incorrect in my private use of reason for my all-things-considered-view to not update somewhat downwards from 33%.
I think this continues to be true even if we all in theory have access to the same public evidence, etc.
Now, it does depend a bit on the context of what this information is for. For example if I’m asked to give my perspective on a group forecast (and I know that the other 100 forecasters’ predictions will be included anyway), I think it probably makes sense for me to continue to publicly provide ~33% for that question to prevent double-counting and groupthink.
But I think it will be wrong for me to believe 33%, and even more so, wrong to say 33% in a context where somebody else doesn’t have access to the 100 other forecasters.
An additional general concern here to me is computational capacity/kindness—sometimes (often) I just don’t have enough time to evaluate all the object-level arguments! You can maybe argue that until I evaluate all the object-level arguments, I shouldn’t act, yet in practice I feel like I act with lots of uncertainty* all the time!
___
One disagreement I have with Max is whether someone should defer is contingent upon the importance of a decision. I think this begs the question in that it pre-assumes that deference lead to the best outcomes.
Instead, I think you should act such that you all-things-considered-view is that you’re making the best decision. I do think that for many decisions (with the possible exception of creative work), some level of deference leads to better outcomes than zero deference at all, but I don’t think it’s unusually true for important decisions except inasmuch as a) the benefits (and also costs!) of deference are scaled accordingly and b) more people are likely to have thought about important decisions.
__
* Narrow, personal, example that’s basically unrelated to EA: I brush my teeth with fluoride toothpaste. I don’t floss. Why? Cochrane review was fairly equivocal about flossing and fairly certain about toothbrushing. Maybe it’d be more principled if I looked at the data myself and performed my own meta-analysis on the data, or perhaps self-experimented like Gwern, to decide what dental hygiene activities I should take. But in practice I feel like it’s a reasonable decision procedure to just defer to Cochrane review on the empirical facts of the matter, and apply my own value judgments on what activities to take given the facts available.
I’m not sure if we have a principled disagreement here, it’s possible that I just described my view badly above.
I agree that one should act such that one’s all-things-considered view is that one is making the best decision (the way I understand that statement it’s basically a tautology).
Then I think there are some heuristics for which features of a decision situation make it more or less likely that deferring more (or at all) leads to decisions with that property. I think on a high level I agree with you that it depends a lot “on the context of what this information is for”, more so than on e.g. importance.
With my example, I was also trying to point less to importance per se but on something like how the costs and benefits are distributed between yourself and others. This is because very loosely speaking I expect not deferring to often be better if the stakes are concentrated on oneself and more deference to be better if one’s own direct stake is small. I used a decision with large effects on others largely because then it’s not plausible that you yourself are affected by a similar amount; but it would also apply to a decision with zero effect on yourself and a small effect on others. Conversely, it would not apply to a decision that is very important to yourself (e.g. something affecting your whole career trajectory).
Apologies for the long delay in response, feel free to not reply if you’re busy.
Hmm I still think we have a substantive rather than framing disagreement (though I think it is likely that our disagreements aren’t large).
Perhaps this heuristic is really useful for a lot of questions you’re considering. I’m reminded of AGB’s great quote:
For me personally and the specific questions I’ve considered, I think considering whether/how much to defer to by dividing into buckets of “how much it affects myself or others” is certainly a pretty useful heuristic in the absence of better heuristics, but it’s mostly superseded by a different decomposition:
Epistemic—In a context-sensitive manner, do we expect greater or lower deference in this particular situation to lead to more accurate beliefs.
Role expectations* -- Whether the explicit and implicit social expectations on the role you’re assuming privilege deference or independence.
So I think a big/main reason it’s bad to defer completely to others (say 80k) on your own career reasons is epistemic: you have so much thought and local knowledge about your own situation that your prior should very strongly be against others having better all-things-considered views on your career choice than you do. I think this is more crux-y for me than how much your career trajectory affects yourself vs others (at any rate hopefully as EAs our career trajectories affect many others anyway!).
On the other hand, I think my Cochrane review example above is a good epistemic example of deference. even though my dental hygiene practices mainly affect myself and not others (perhaps my past and future partners may disagree), I contend it’s better to defer to the meta-analysis over my own independent analysis in this particular facet of my personal life.
The other main (non-epistemic) lens I’d use to privilege greater or lower humility is whether the explicit and implicit social expectations privilege deference or independence. For example, we’d generally** prefer government bureaucrats in most situations to implement policies, rather than making unprincipled exceptions based on private judgements. This will often look superficially similar to “how much this affects myself or others.”
An example of a dissimilarity is when someone filling out a survey. This is a situation where approximately all of the costs and benefits are borne by other people. So if you have a minority opinion on a topic, it may seem like the epistemically humble-and-correct action is to fill out the poll according to what you believe the majority to think (or alternatively, fill it out with the answer that you privately think is on the margin more conducive to advancing your values).
But in all likelihood, such a policy is one-thought-too-many, and in almost all situations it’d be more prudent to fill out public anonymous polls/surveys with what you actually believe.
Agreed, though I mention this because in discussions of epistemic humility-in-practice, it’s very easy to accidentally do double-counting.
*I don’t like this phrase, happy to use a better one.
**I’m aware that there are exceptions, including during the ongoing coronavirus pandemic.
Thanks! I’m not sure if there is a significant difference about how we’d actually make decisions (I mean, on prior there is probably some difference). But I agree that the single heuristics I mentioned above doesn’t by itself do a great job of describing when and how much to defer, and I agree with your “counterexamples”. (Though note that in principle it’s not surprising if there are counterexamples to a “mere heuristics”.)
I particularly appreciate you describing the “Role expectations” point. I agree that something along those lines is important. My guess is that if we would have debated specific decisions I would have implicitly incorporated this consideration, but I don’t think it was clear to me before reading your comment that this is an important property that will often influence my judgment about how much to defer.