Hm, I think you may be reading the comment from a perspective of “what actions do the symbols refer to, and what would happen if readers did that?” as opposed to “what are the symbols going to cause readers to do?”[1]
The kinds of people who are able distinguish adequate vs inadequate good judgment shouldn’t be encouraged to defer to conventional signals of expertise. But those are also disproportionately the people who, instead of feeling like deferring to Eliezer’s comment, will respond “I agree, but...”
For lack of a better term, and because there should be a term for it: Dan Sperber calls this the “cognitive causal chain”, and contrasts it with the confabulated narratives we often have for what we do. I think it summons up the right image.
When you read something, aspire to always infer what people intend based on the causal chains that led them to write that. Well, no. Not quite. Instead, aspire to always entertain the possibility that the author’s consciously intended meaning may be inferred from what the symbols will cause readers to do. Well, I mean something along these lines. The point is that if you do this, you might discover a genuine optimiser in the wild. : )
Hm, I think you may be reading the comment from a perspective of “what actions do the symbols refer to, and what would happen if readers did that?” as opposed to “what are the symbols going to cause readers to do?”[1]
The kinds of people who are able distinguish adequate vs inadequate good judgment shouldn’t be encouraged to defer to conventional signals of expertise. But those are also disproportionately the people who, instead of feeling like deferring to Eliezer’s comment, will respond “I agree, but...”
For lack of a better term, and because there should be a term for it: Dan Sperber calls this the “cognitive causal chain”, and contrasts it with the confabulated narratives we often have for what we do. I think it summons up the right image.
When you read something, aspire to always infer what people intend based on the causal chains that led them to write that. Well, no. Not quite. Instead, aspire to always entertain the possibility that the author’s consciously intended meaning may be inferred from what the symbols will cause readers to do. Well, I mean something along these lines. The point is that if you do this, you might discover a genuine optimiser in the wild. : )