Something that I think is useful to keep in mind as you probe your own position, whether by yourself, or in debate with others, is:
what’s the least surprising piece of evidence, or set of evidence, that would be enough for me to change my mind?
I think sometimes I e.g. have a object-level disagreement with someone about a technology or a meta-disagreement about the direction of EA strategy, and my interlocutor says something like “oh I’ll only change my mind if you demonstrate that my entire understanding of causality is wrong or everything I learned in the last 25 years about human nature is vastly off.” I think I have two responses to that approach:
1. I think the stated bar for changing your mind is inexplicably high. Maybe I should just stop talking then. Like I doubt I’m able (or willing) to put in enough effort to revise 25 years of your priors about the nature of causality or human nature, or whatever. (and by symmetry, if you perceive that to be my crux, you probably should be skeptical that you can convince me that I’m wrong about deeply held notions of causality or human nature).
2. Fortunately I think this is more a failure of metacognition. Like I usually justdon’t believe you that you will literally require enough evidence to change your mind about a ton of very-hard-to-change priors, given the information I have available about how I and other people usually change our minds about questions similar to this.
Something that I think is useful to keep in mind as you probe your own position, whether by yourself, or in debate with others, is:
I think sometimes I e.g. have a object-level disagreement with someone about a technology or a meta-disagreement about the direction of EA strategy, and my interlocutor says something like “oh I’ll only change my mind if you demonstrate that my entire understanding of causality is wrong or everything I learned in the last 25 years about human nature is vastly off.” I think I have two responses to that approach:
1. I think the stated bar for changing your mind is inexplicably high. Maybe I should just stop talking then. Like I doubt I’m able (or willing) to put in enough effort to revise 25 years of your priors about the nature of causality or human nature, or whatever. (and by symmetry, if you perceive that to be my crux, you probably should be skeptical that you can convince me that I’m wrong about deeply held notions of causality or human nature).
2. Fortunately I think this is more a failure of metacognition. Like I usually just don’t believe you that you will literally require enough evidence to change your mind about a ton of very-hard-to-change priors, given the information I have available about how I and other people usually change our minds about questions similar to this.