The problem of motivated reasoning is in some ways much deeper than the trolley problem.
The motivation behind motivated reasoning is often to make ourselves look good (in order to gain status/power/prestige). Much of the problem seems to come from not consciously acknowledging this motivation, and therefore not being able to apply system 2 to check for errors in the subconscious optimization.
My approach has been to acknowledge that wanting to make myself look good may be a part of my real or normative values (something like what I would conclude my values are after solving all of philosophy). Since I can’t rule that out for now (and also because it’s instrumentally useful), I think I should treat it as part of my “interim values”, and consciously optimize for it along with my other “interim values”. Then if I’m tempted to do something to look good, at a cost to my other values or perhaps counterproductive on its own terms, I’m more likely to ask myself “Do I really want to do this?”
BTW I’m curious what courses you teach, and whether / how much you tell your students about motivated reasoning or subconscious status motivations when discussing ethics.
The motivation behind motivated reasoning is often to make ourselves look good (in order to gain status/power/prestige). Much of the problem seems to come from not consciously acknowledging this motivation, and therefore not being able to apply system 2 to check for errors in the subconscious optimization.
My approach has been to acknowledge that wanting to make myself look good may be a part of my real or normative values (something like what I would conclude my values are after solving all of philosophy). Since I can’t rule that out for now (and also because it’s instrumentally useful), I think I should treat it as part of my “interim values”, and consciously optimize for it along with my other “interim values”. Then if I’m tempted to do something to look good, at a cost to my other values or perhaps counterproductive on its own terms, I’m more likely to ask myself “Do I really want to do this?”
BTW I’m curious what courses you teach, and whether / how much you tell your students about motivated reasoning or subconscious status motivations when discussing ethics.