Zach—thanks for this comment; I’m working on a reply to it, which I’ll published as an EA Forum post within a couple of days.
A preview: I think there are good theoretical and empirical reasons why alignment research taking the full heterogeneity of human value types into account (including differences between religious values, political values, food preferences, economic ambitions, mate preferences, cultural taboos, aesthetic tastes, etc) would NOT look identical to current alignment research.
Zach—thanks for this comment; I’m working on a reply to it, which I’ll published as an EA Forum post within a couple of days.
A preview: I think there are good theoretical and empirical reasons why alignment research taking the full heterogeneity of human value types into account (including differences between religious values, political values, food preferences, economic ambitions, mate preferences, cultural taboos, aesthetic tastes, etc) would NOT look identical to current alignment research.