If you buy it, there is a neat continuity from the problems with current social media and AI alignment, explained in some detail in What Failure Looks Like.
It’s already much easier to pursue easy-to-measure goals, but machine learning will widen the gap by letting us try a huge number of possible strategies and search over massive spaces of possible actions. That force will combine with and amplify existing institutional and social dynamics that already favor easily-measured goals.
If you buy it, there is a neat continuity from the problems with current social media and AI alignment, explained in some detail in What Failure Looks Like.