Context: This post popped into my head because I was having a conversation with Peter Hartree about whether a specific argument by Peter Thiel made sense. And I was claiming that at a glance, a specific Thiel-argument seemed locally invalid to me, in the sense of Local Validity as a Key to Sanity and Civilization.
And Peter’s response was that Thiel has a good enough track record that we should be very reluctant to assume he’s wrong about something like this, and should put in an effort to steel-man him and figure out what alternative, more-valid things he could have meant.
And I’m OK with trying that out as an intellectual exercise. (Though I’ve said before that steelmanning can encourage fuzzy thinking and misunderstandings, and we should usually prioritize fleshmanning / passing people’s Ideological Turing Test, rather than just trying to make up arguments that seem more plausible ex nihilo.)
But I felt an urge to say “this EA thing where we steel-man and defer to impressive people we respect, rather than just blurting out when a thing doesn’t make sense to us until we hear a specific counter-argument, is part of our Bigger Problem”. I think this problem keeps cropping up in EA across a bunch of domains — not just “why didn’t the EA Forum host a good early discussion of at least one SBF red or yellow flag?”, but “why do EAs keep getting into deference cascades that lead them to double- and triple-count evidence for propositions?”, and “why are EAs herding together and regurgitating others’ claims on AI alignment topics rather than going off in dozens of strange directions to test various weird inside-view ideas and convictions and build their own understanding?”.
(Do people not realize that humanity doesn’t know shit about alignment yet? I feel like people keep going into alignment research and being surprised by this fact.)
It all feels like one thing to me — this idea that it’s fine for me to blurt out an objection to a local thing Thiel said, even though I respect him as a thinker and am a big fan of some of his ideas. Because blurting out objections is just the standard way for EAs to respond to anything that seems off to them.
I then want to be open that I may be wrong about Thiel on this specific point, and I want to listen to counter-arguments. But I don’t think it would be good (for my epistemics or for the group’s epistemics) to go through any special mental gymnastics or justification-ritual in order to blurt out my first-order objection in the first place.
I wanted to say all that in the Peter Thiel conversation. But then I worried that I wouldn’t be able to communicate my point because people would think that I’m darkly hinting at Peter Thiel being a bad actor. (Because they’re conflating the problem “EAs aren’t putting a high enough prior on people being bad actors” with this other problem, and not realizing that organizing their mental universe around “bad actors vs. good actors” can make it harder to spot early signs of bad actors, and also harder to do a lot of other things EA ought to try to do.)
Context: This post popped into my head because I was having a conversation with Peter Hartree about whether a specific argument by Peter Thiel made sense. And I was claiming that at a glance, a specific Thiel-argument seemed locally invalid to me, in the sense of Local Validity as a Key to Sanity and Civilization.
And Peter’s response was that Thiel has a good enough track record that we should be very reluctant to assume he’s wrong about something like this, and should put in an effort to steel-man him and figure out what alternative, more-valid things he could have meant.
And I’m OK with trying that out as an intellectual exercise. (Though I’ve said before that steelmanning can encourage fuzzy thinking and misunderstandings, and we should usually prioritize fleshmanning / passing people’s Ideological Turing Test, rather than just trying to make up arguments that seem more plausible ex nihilo.)
But I felt an urge to say “this EA thing where we steel-man and defer to impressive people we respect, rather than just blurting out when a thing doesn’t make sense to us until we hear a specific counter-argument, is part of our Bigger Problem”. I think this problem keeps cropping up in EA across a bunch of domains — not just “why didn’t the EA Forum host a good early discussion of at least one SBF red or yellow flag?”, but “why do EAs keep getting into deference cascades that lead them to double- and triple-count evidence for propositions?”, and “why are EAs herding together and regurgitating others’ claims on AI alignment topics rather than going off in dozens of strange directions to test various weird inside-view ideas and convictions and build their own understanding?”.
(Do people not realize that humanity doesn’t know shit about alignment yet? I feel like people keep going into alignment research and being surprised by this fact.)
It all feels like one thing to me — this idea that it’s fine for me to blurt out an objection to a local thing Thiel said, even though I respect him as a thinker and am a big fan of some of his ideas. Because blurting out objections is just the standard way for EAs to respond to anything that seems off to them.
I then want to be open that I may be wrong about Thiel on this specific point, and I want to listen to counter-arguments. But I don’t think it would be good (for my epistemics or for the group’s epistemics) to go through any special mental gymnastics or justification-ritual in order to blurt out my first-order objection in the first place.
I wanted to say all that in the Peter Thiel conversation. But then I worried that I wouldn’t be able to communicate my point because people would think that I’m darkly hinting at Peter Thiel being a bad actor. (Because they’re conflating the problem “EAs aren’t putting a high enough prior on people being bad actors” with this other problem, and not realizing that organizing their mental universe around “bad actors vs. good actors” can make it harder to spot early signs of bad actors, and also harder to do a lot of other things EA ought to try to do.)
So I wrote this post. :)