The very ability of considering what one’s position would be in a scenario very different from the one in which one finds oneself is prefaced by controlling the impulse to react to the immediate environment.
The common feature between, say, Nick Bostrom’s PHD, Nick Beckstead’s PHD and Paul Christiano’s blog Rational Altruist, is a capacity to hold even fewer particularities of one’s environment as true come what may.
Empathy is just the opposite of that, empathy is frequently seen as the immediate, system 1, uncontrollable emotion that one experiences when someone else in the local immediacies undergoes distress.
I’ve argued in the past, and would continue to argue, that the moral obligation is higher, not lower, for people with less empathy. I’m much more forgiving of people who give locally and thereby fail to save globally if they do it to avoid feeling the sadness of empathy.