Yeah, I do think that “I can’t actually realistically represent this scenario in my imagination, and if I try I’ll just deceive myself, so I won’t” has become a pretty deep intuition for me over the years.
I think it’s more thoroughly internalised for scenarios that are unimaginably large (many people, very long stretches of time) than scenarios that are small but weird. Possibly because the intuition for size has been trained by a lot of real-world experiences – I don’t think a human can really imagine even a million people, so there are many real-world cases where the correct response is to back off from visual imagination and shut up and multiply.
Utility monsters (and the Fat Man trolley problem variant) are small but weird, so it’s more difficult for me to accept that my intuitive imagination of the scenario is likely to be misleading. I’ve seen fictional representations of utility monsters, and in general when I try to imagine a single sentient being it’s difficult not to imagine something like a human. So even though I believe that a real utility monster would in fact be a profoundly alien and hard-to-imagine being, when I think about the scenario my brain conjures up a human tyrant and it seems really bad.
Whereas for the RC my brain sees the words “unimaginably vast” and decides not to try and imagine.
Yeah, I do think that “I can’t actually realistically represent this scenario in my imagination, and if I try I’ll just deceive myself, so I won’t” has become a pretty deep intuition for me over the years.
I think it’s more thoroughly internalised for scenarios that are unimaginably large (many people, very long stretches of time) than scenarios that are small but weird. Possibly because the intuition for size has been trained by a lot of real-world experiences – I don’t think a human can really imagine even a million people, so there are many real-world cases where the correct response is to back off from visual imagination and shut up and multiply.
Utility monsters (and the Fat Man trolley problem variant) are small but weird, so it’s more difficult for me to accept that my intuitive imagination of the scenario is likely to be misleading. I’ve seen fictional representations of utility monsters, and in general when I try to imagine a single sentient being it’s difficult not to imagine something like a human. So even though I believe that a real utility monster would in fact be a profoundly alien and hard-to-imagine being, when I think about the scenario my brain conjures up a human tyrant and it seems really bad.
Whereas for the RC my brain sees the words “unimaginably vast” and decides not to try and imagine.