I mostly share this sentiment. One concern I have: I think one must be very careful in developing cause prioritization tools that work with almost any value system. Optimizing for naively held moral views can cause net harm; Scott Alexander implied that terrorists might just be taking beliefs too seriously when those beliefs only work in an environment of epistemic learned helplessness.
One possible way to identify views reasonable enough to develop tools for is checking that they’re consistent under some amount of reflection; another way could be checking that they’re consistent with facts e.g. lack of evidence for supernatural entities, or the best knowledge on conscious experience of animals.
I mostly share this sentiment. One concern I have: I think one must be very careful in developing cause prioritization tools that work with almost any value system. Optimizing for naively held moral views can cause net harm; Scott Alexander implied that terrorists might just be taking beliefs too seriously when those beliefs only work in an environment of epistemic learned helplessness.
One possible way to identify views reasonable enough to develop tools for is checking that they’re consistent under some amount of reflection; another way could be checking that they’re consistent with facts e.g. lack of evidence for supernatural entities, or the best knowledge on conscious experience of animals.