I suspect that the epistemic health that matters is on the level of the group and not the individual. I.e. I care about the net effect of the groupās actions, and I donāt much care whether individuals are rational, as long as the group acts rationally. With that framing, would it be better if everyone individually ātried hardā or not?
Seems like itād net out to being better if the correct course of action produces much more utility than the aggregate disutility of the rest of the actions we are likely to take. However, if itās very easy to do badly wrong, doing less might be the better strategy.
One other point though: When you take negative responsibility seriously (as we should), you dispense with the idea of a neutral option, or the choice to do nothing. Bracketing the epistemics-distorting point, there isnāt necessarily a difference in the expected (sign-neutral) impact of doing ānothingā and trying hard.
Yeah, I think thereās a lot of interesting things to be said here. Some points:
-Iām not sure why we should expect the group to be well-calibrated when no individual is? That sounds a little magical to me. Itās not a market dynamic where ground-level feedback directly lowers the viability/āstatus of incorrect hypotheses, and so leads to the collective being smarter than any individualāsince we get no direct feedback about our effect on the long-term future whatsoever, and certainly none that would literally force us to stop what weāre doing (like not being in touch with what the market wants being able to cause your business to fail).
It feels more viable to me for every individual to act with epistemic virtue (or explicitly defer to someone who does so).
-For a slightly tangential point, itās interesting to think about what the optimal social structures of deferral would be, but Iāll note that my guess is that the most influential people also tend to be among the people who work/ātry the hardest? Thatās a big part of why people are successful, after all. So if anything, this is a cause for more worry.
-On your āother pointā: Yeah, I certainly donāt endorse doing nothing. The epistemic distortion is just one consideration that happens to push towards doing less (since it lowers the EV of doing anything, if we havenāt considered it before). It needs to be weighed against the many other considerations that exist.
āIām not sure why we should expect the group to be well-calibrated when no individual is?āāSomething something marketplace of ideas? An analogy is a court, where the prosecution and the defence both have their conclusions assigned to them beforehand. They are both epistemically vicious, in opposite ways. Then the idea is that the best arguments win on their merits. Iām not sure quite how this analogy would fit though (Iāve had false starts writing a blog post on this a few times).
āI certainly donāt endorse doing nothingā Yep I get that this argument is only one consideration, but my point extends to ātrying less hardā as well I think.
āYep I get that this argument is only one consideration, but my point extends to ātrying less hardā as well I think.ā Iām not sure I understand what you mean, could you explain it more?
āSomething something marketplace of ideas?ā Yeah, this is an empirical questionātheoretically, we might see better arguments rise in status and influence people more, and of course to some extent we do (e.g. AI risk rising in status over the last 10 years). But there are other factors influencing the status of ideas too, like some kind of general action-bias /ā power-seeking-bias. Concretely, I feel that most of the bullet points in the āuncertaintyā section of the post are underrepresented in the discourseācurious if you disagree.
Also, one more point on the deferral thingāone way in which I think Iām weird is that I would genuinely raise my esteem of someone (on a gut-level) if they said āIām hopelessly biased on this topic, I canāt think about it, donāt listen to meā. Unfortunately, you never see this. I would really like to see that more. E.g. if someone who tries extremely hard said āI canāt really clearly think about what Iām doing because Iām working so hard, so be careful about listening to meā it would be very beautiful to me. Poetically speaking, it would beā¦ accepting that they are a human weapon, forged for a purpose, impaired by that sacrifice, baring it for the world to see. There would be a slight feeling of heartbreak and love for them in me, and I might very well value them more than before. As I argue in the āEpistemic distortionā section, thereās at least to some extent a deep tradeoff between doing and thinkingāso admitting that they are trading off against thinking, crippling their mind on a deep level, could make people respect them more by showing how much they are sacrificing for the ādoingā status hierarchy. It could be heroic. Thatās just a fantasy I have about how things could work.
Interesting post!
I suspect that the epistemic health that matters is on the level of the group and not the individual. I.e. I care about the net effect of the groupās actions, and I donāt much care whether individuals are rational, as long as the group acts rationally. With that framing, would it be better if everyone individually ātried hardā or not?
Seems like itād net out to being better if the correct course of action produces much more utility than the aggregate disutility of the rest of the actions we are likely to take. However, if itās very easy to do badly wrong, doing less might be the better strategy.
One other point though: When you take negative responsibility seriously (as we should), you dispense with the idea of a neutral option, or the choice to do nothing. Bracketing the epistemics-distorting point, there isnāt necessarily a difference in the expected (sign-neutral) impact of doing ānothingā and trying hard.
Thanks Toby!
Yeah, I think thereās a lot of interesting things to be said here. Some points:
-Iām not sure why we should expect the group to be well-calibrated when no individual is? That sounds a little magical to me. Itās not a market dynamic where ground-level feedback directly lowers the viability/āstatus of incorrect hypotheses, and so leads to the collective being smarter than any individualāsince we get no direct feedback about our effect on the long-term future whatsoever, and certainly none that would literally force us to stop what weāre doing (like not being in touch with what the market wants being able to cause your business to fail).
It feels more viable to me for every individual to act with epistemic virtue (or explicitly defer to someone who does so).
-For a slightly tangential point, itās interesting to think about what the optimal social structures of deferral would be, but Iāll note that my guess is that the most influential people also tend to be among the people who work/ātry the hardest? Thatās a big part of why people are successful, after all. So if anything, this is a cause for more worry.
-On your āother pointā: Yeah, I certainly donāt endorse doing nothing. The epistemic distortion is just one consideration that happens to push towards doing less (since it lowers the EV of doing anything, if we havenāt considered it before). It needs to be weighed against the many other considerations that exist.
Curious what you think!
āIām not sure why we should expect the group to be well-calibrated when no individual is?āāSomething something marketplace of ideas? An analogy is a court, where the prosecution and the defence both have their conclusions assigned to them beforehand. They are both epistemically vicious, in opposite ways. Then the idea is that the best arguments win on their merits. Iām not sure quite how this analogy would fit though (Iāve had false starts writing a blog post on this a few times).
āI certainly donāt endorse doing nothingā Yep I get that this argument is only one consideration, but my point extends to ātrying less hardā as well I think.
āYep I get that this argument is only one consideration, but my point extends to ātrying less hardā as well I think.ā Iām not sure I understand what you mean, could you explain it more?
āSomething something marketplace of ideas?ā
Yeah, this is an empirical questionātheoretically, we might see better arguments rise in status and influence people more, and of course to some extent we do (e.g. AI risk rising in status over the last 10 years). But there are other factors influencing the status of ideas too, like some kind of general action-bias /ā power-seeking-bias. Concretely, I feel that most of the bullet points in the āuncertaintyā section of the post are underrepresented in the discourseācurious if you disagree.
Also, one more point on the deferral thingāone way in which I think Iām weird is that I would genuinely raise my esteem of someone (on a gut-level) if they said āIām hopelessly biased on this topic, I canāt think about it, donāt listen to meā. Unfortunately, you never see this. I would really like to see that more.
E.g. if someone who tries extremely hard said āI canāt really clearly think about what Iām doing because Iām working so hard, so be careful about listening to meā it would be very beautiful to me. Poetically speaking, it would beā¦ accepting that they are a human weapon, forged for a purpose, impaired by that sacrifice, baring it for the world to see. There would be a slight feeling of heartbreak and love for them in me, and I might very well value them more than before. As I argue in the āEpistemic distortionā section, thereās at least to some extent a deep tradeoff between doing and thinkingāso admitting that they are trading off against thinking, crippling their mind on a deep level, could make people respect them more by showing how much they are sacrificing for the ādoingā status hierarchy. It could be heroic.
Thatās just a fantasy I have about how things could work.