I agree that we need to be careful about who we are empowering.
“Value alignment” is one of those terms which has different meanings to different people. For example, the top hit I got on Google for “effective altruism value alignment” was a ConcernedEAs post which may not reflect what you mean by the term. Without knowing exactly what you mean, I’d hazard a guess that some facets of value alignment are pretty relevant to mitigating this kind of risk, and other facets are not so important. Moreover, I think some of the key factors are less cognitive or philosophical than emotional or motivational (e.g., a strong attraction toward money will increase the risk of defecting, a lack of self-awareness increases the risk of motivated reasoning toward goals one has in a sense repressed).
So, I think it would be helpful for orgs to consider what elements of “value alignment” are of particular importance here, as well as what other risk or protective factors might exist outside of value alignment, and focus on those specific things.
I agree that we need to be careful about who we are empowering.
“Value alignment” is one of those terms which has different meanings to different people. For example, the top hit I got on Google for “effective altruism value alignment” was a ConcernedEAs post which may not reflect what you mean by the term. Without knowing exactly what you mean, I’d hazard a guess that some facets of value alignment are pretty relevant to mitigating this kind of risk, and other facets are not so important. Moreover, I think some of the key factors are less cognitive or philosophical than emotional or motivational (e.g., a strong attraction toward money will increase the risk of defecting, a lack of self-awareness increases the risk of motivated reasoning toward goals one has in a sense repressed).
So, I think it would be helpful for orgs to consider what elements of “value alignment” are of particular importance here, as well as what other risk or protective factors might exist outside of value alignment, and focus on those specific things.
Agreed. “Value alignment” is a simplified framing.