Huh, yeah. I wonder whether this isn’t more of an “inadequate equilibria” type of thing where we use all the right tools that our goals incentivize us to use – an so do all the other groups, except their incentives are weird and different. Then there could easily be groups with uncooperative values but incentives that lead them to use the same tools.
A counterargument could be that a lot of these tools require some expertise, and people who have that expertise are probably not usually desperate enough to have to take some evil job, so most of these people will choose a good/neutral job over and evil job even if the salary is a bit lower.
But I suppose some socially skilled narcissist can just exploit any random modern surrogate religion to recruit good people for something evil by appealing to their morality in twisted ways. So I think it’s a pretty neat mechanism but also one that fails frequently.
Yeah, one of many, many benefits! :-) I don’t think the effect is going to be huge (so that we could rely on it) or tiny. But I’m also hoping that someone will use my system to help me clarify my values. ^^
Huh, yeah. I wonder whether this isn’t more of an “inadequate equilibria” type of thing where we use all the right tools that our goals incentivize us to use – an so do all the other groups, except their incentives are weird and different. Then there could easily be groups with uncooperative values but incentives that lead them to use the same tools.
A counterargument could be that a lot of these tools require some expertise, and people who have that expertise are probably not usually desperate enough to have to take some evil job, so most of these people will choose a good/neutral job over and evil job even if the salary is a bit lower.
But I suppose some socially skilled narcissist can just exploit any random modern surrogate religion to recruit good people for something evil by appealing to their morality in twisted ways. So I think it’s a pretty neat mechanism but also one that fails frequently.
Yeah, one of many, many benefits! :-) I don’t think the effect is going to be huge (so that we could rely on it) or tiny. But I’m also hoping that someone will use my system to help me clarify my values. ^^
Deferring to future versions of us: Yep!