My perspective on this (or more generally on the question of whether the future is likely to involve realizing a large fraction of the possible value it could have, whatever form it turns out âvalueâ takes) is perhaps a bit more hopeful. In my view, the question only makes sense if we are moral realists. If there are no objective facts about morality, then I donât see why we should care whether our own preferences or someone elseâs win out. Furthermore, I think worrying about these questions is probably pointless unless two other things are true: that we have some way of discovering moral facts and that those discoveries have some way of influencing our actions. Unless those two are somehow true, we have no reason to think our efforts can in expectation increase the amount of value realized in the world.
So far this is a somewhat pessimistic take, but Iâm optimistic in a world where all three of these conditions are true, which in some sense is (IMO) the only world where this conversation makes any sense to have. In that world, we should expect that increasing the amount of things like intelligence, time to devote to research/âreflection, and focus on studying moral questions in expectation leads toward getting closer to the true morality. Welfareans (or more broadly whatever target will produce the most true value) may indeed get enough advocates just by virtue of society making more progress on these moral questions. As an example, society today includes lots of advocates for groups like women, LGBT people, people of color etc., when historically the only advocates were a âtiny subset of crazy people.â But of course moral progress is at best an extremely messy and incrementalâfactory-farmed animals are the victims of lots of people being either indifferent or wanting to maximize something other than welfare (profit, tasty food for humans etc.), and the impact of animal advocates has not been sufficient to prevent a massive explosion of suffering.
Still, on net I lean towards thinking that given the opportunity for study and reflection (and given the three conditions described above), we can be optimistic that we will drive toward the things that matter. Therefore, focusing on the efforts to prevent existential catastrophe or value lock-in may be among the best things we could do to ensure that weâre not leaving a huge fraction of the possible value of the future on the table. That may be easier said than done, since preventing value lock-in in practice means preventing people with maximizing ideologies from successfully carrying out that maximization at least for some period of time. But that makes me hopeful that existing EA efforts may not be too far off the mark.
My perspective on this (or more generally on the question of whether the future is likely to involve realizing a large fraction of the possible value it could have, whatever form it turns out âvalueâ takes) is perhaps a bit more hopeful. In my view, the question only makes sense if we are moral realists. If there are no objective facts about morality, then I donât see why we should care whether our own preferences or someone elseâs win out. Furthermore, I think worrying about these questions is probably pointless unless two other things are true: that we have some way of discovering moral facts and that those discoveries have some way of influencing our actions. Unless those two are somehow true, we have no reason to think our efforts can in expectation increase the amount of value realized in the world.
So far this is a somewhat pessimistic take, but Iâm optimistic in a world where all three of these conditions are true, which in some sense is (IMO) the only world where this conversation makes any sense to have. In that world, we should expect that increasing the amount of things like intelligence, time to devote to research/âreflection, and focus on studying moral questions in expectation leads toward getting closer to the true morality. Welfareans (or more broadly whatever target will produce the most true value) may indeed get enough advocates just by virtue of society making more progress on these moral questions. As an example, society today includes lots of advocates for groups like women, LGBT people, people of color etc., when historically the only advocates were a âtiny subset of crazy people.â But of course moral progress is at best an extremely messy and incrementalâfactory-farmed animals are the victims of lots of people being either indifferent or wanting to maximize something other than welfare (profit, tasty food for humans etc.), and the impact of animal advocates has not been sufficient to prevent a massive explosion of suffering.
Still, on net I lean towards thinking that given the opportunity for study and reflection (and given the three conditions described above), we can be optimistic that we will drive toward the things that matter. Therefore, focusing on the efforts to prevent existential catastrophe or value lock-in may be among the best things we could do to ensure that weâre not leaving a huge fraction of the possible value of the future on the table. That may be easier said than done, since preventing value lock-in in practice means preventing people with maximizing ideologies from successfully carrying out that maximization at least for some period of time. But that makes me hopeful that existing EA efforts may not be too far off the mark.