I really appreciated this update. Mostly it checks out to me, but I wanted to push back on this:
Here’s a dumb thought experiment: Suppose that Yudkowsky wrote all of the same things, but never published them. But suppose, also, that a freak magnetic storm ended up implanting all of the same ideas in his would-be-readers’ brains. Would this absence of a casual effect count against deferring to Yudkowsky? I don’t think so. The only thing that ultimately matters, I think, is his track record of beliefs—and the evidence we currently have about how accurate or justified those beliefs were.
It seems to me that a good part of the beliefs I care about assessing are the beliefs about what is important. When someone has a track record of doing things with big positive impact, that’s some real evidence that they have truth-tracking beliefs about what’s important. In the hypothetical where Yudkowsky never published his work, I don’t get the update that he thought these were important things to publish, so he doesn’t get credit for being right about that.
There’s also (imperfect) information in “lots of smart people thought about EY’s opinions and agree with him” that you don’t get from the freak magnetic storm scenario.
I really appreciated this update. Mostly it checks out to me, but I wanted to push back on this:
It seems to me that a good part of the beliefs I care about assessing are the beliefs about what is important. When someone has a track record of doing things with big positive impact, that’s some real evidence that they have truth-tracking beliefs about what’s important. In the hypothetical where Yudkowsky never published his work, I don’t get the update that he thought these were important things to publish, so he doesn’t get credit for being right about that.
There’s also (imperfect) information in “lots of smart people thought about EY’s opinions and agree with him” that you don’t get from the freak magnetic storm scenario.