And then I guess both of as are in some kind of agreement that this kind of stuff (deliberate structured initatives to inject some democracy into the models) ends up majorly determining outcomes from AGI.
Yeah I think this is plausible and a good point of agreement, plus a promising leverage point. But I do kind of expect normal capitalist incentives will dominate anything like this, and that governments won’t intervene except for issues of safety, as you seem to.
I find this somewhat confusing
Nash is formally equivalent to the sum of log utilities (preferences) in the disagreement set, so it’s a prioritarian transformation of preference utilitarianism over a particular bargain.
I agree that it can come drastically far apart from totalist utilitarianism. What I actually like about it is that it’s a principled way to give everyone’s values equal weight that preserves the cardinality in people’s value functions and is arbitrarily sensitive to changes in individuals’ values, and that it doesn’t require interpersonally comparable utilities, making it very workable. I also like that it maximizes a certain weighted sum of efficiency and equality. As an antirealist who thinks I have basically unique values, I like that it guarantees that my values have some sway over the future.
One thing I don’t like about Nash is that it’s a logistic form of prioritarianism, and over preferences rather than people. That means that my strongest preferences don’t get that much more weight over my weakest preferences. Perhaps for that reason simple quadratic voting does better. It’s in some ways less elegantly grounded, but it’s also more well-understood by the broader world.
I’m seeing the position as a principled way to have a fair compromise across different people’s moral viewpoints, which also happens to do pretty well by the lights of my own values. It’s not attempting to approximate classical utilitarianism directly, but instead to give me some control over the future in the areas that matter the most to me, and thereby allow me to enact classical utilitarianism. There might be better such approaches, but so far this is the one that seems most promising to me at the moment.
Thanks for all this! I agree that something like Nash is appealing for a bunch of reasons. Not least because it’s Pareto efficient, so doesn’t screw people over for the greater good, which feels more politically legitimate. It is also principled, in that it doesn’t require some social planner to decide how to weigh people’s preferences or wellbeing.
My sense, though you know much more about all this, is that Nash bargaining is not well described as a variant of utilitarianism, though I case it’s a grey area.
Maybe I’m realising now that a lot of the action in your argument is not in arguing for the values which guide the future to be democratically chosen, but rather in thikning through which kinds of democratic mechanisms are best. Where plain old majority rule seems very unappealing, but more granular approaches which give more weight to those who care most about a given issue look much better. And (here we agree) this is especially important if you think that the wrong kind of popular future, such as a homogenous majority-determined future, could fall far short of the best future.
Huh, I mean it just is formally equivalent to the sum of log utilities in the bargaining situation! But “utilitarianism” is fuzzy :)
Yes, the idea of finding a preference aggregation mechanism that does much better than modern electoral systems at capturing the cardinality of societal preferences is, I think, really core to what I’m doing here, so I probably should have brought this out a bit more than I did!
Yeah, fair! I guess there’s a broad understanding of utilitarianism, which is “the sum of any monotone or non-decreasing transformation of utilities”, and a narrower understanding, which is “the sum of utilities”. But I want to say that prioritarianism (a version of the former) is an alternative to utilitarianism, not a variant. Not actually sure what prioritarians would say. Also not really an important point to argue about.
Makes sense! There’s some old writers in the utilitarian tradition like James Griffin that define utilitarianism in the broader way, but I do think your articulation is probably more common.
Yeah I think this is plausible and a good point of agreement, plus a promising leverage point. But I do kind of expect normal capitalist incentives will dominate anything like this, and that governments won’t intervene except for issues of safety, as you seem to.
Nash is formally equivalent to the sum of log utilities (preferences) in the disagreement set, so it’s a prioritarian transformation of preference utilitarianism over a particular bargain.
I agree that it can come drastically far apart from totalist utilitarianism. What I actually like about it is that it’s a principled way to give everyone’s values equal weight that preserves the cardinality in people’s value functions and is arbitrarily sensitive to changes in individuals’ values, and that it doesn’t require interpersonally comparable utilities, making it very workable. I also like that it maximizes a certain weighted sum of efficiency and equality. As an antirealist who thinks I have basically unique values, I like that it guarantees that my values have some sway over the future.
One thing I don’t like about Nash is that it’s a logistic form of prioritarianism, and over preferences rather than people. That means that my strongest preferences don’t get that much more weight over my weakest preferences. Perhaps for that reason simple quadratic voting does better. It’s in some ways less elegantly grounded, but it’s also more well-understood by the broader world.
I’m seeing the position as a principled way to have a fair compromise across different people’s moral viewpoints, which also happens to do pretty well by the lights of my own values. It’s not attempting to approximate classical utilitarianism directly, but instead to give me some control over the future in the areas that matter the most to me, and thereby allow me to enact classical utilitarianism. There might be better such approaches, but so far this is the one that seems most promising to me at the moment.
Thanks for all this! I agree that something like Nash is appealing for a bunch of reasons. Not least because it’s Pareto efficient, so doesn’t screw people over for the greater good, which feels more politically legitimate. It is also principled, in that it doesn’t require some social planner to decide how to weigh people’s preferences or wellbeing.
My sense, though you know much more about all this, is that Nash bargaining is not well described as a variant of utilitarianism, though I case it’s a grey area.
Maybe I’m realising now that a lot of the action in your argument is not in arguing for the values which guide the future to be democratically chosen, but rather in thikning through which kinds of democratic mechanisms are best. Where plain old majority rule seems very unappealing, but more granular approaches which give more weight to those who care most about a given issue look much better. And (here we agree) this is especially important if you think that the wrong kind of popular future, such as a homogenous majority-determined future, could fall far short of the best future.
Huh, I mean it just is formally equivalent to the sum of log utilities in the bargaining situation! But “utilitarianism” is fuzzy :)
Yes, the idea of finding a preference aggregation mechanism that does much better than modern electoral systems at capturing the cardinality of societal preferences is, I think, really core to what I’m doing here, so I probably should have brought this out a bit more than I did!
Yeah, fair! I guess there’s a broad understanding of utilitarianism, which is “the sum of any monotone or non-decreasing transformation of utilities”, and a narrower understanding, which is “the sum of utilities”. But I want to say that prioritarianism (a version of the former) is an alternative to utilitarianism, not a variant. Not actually sure what prioritarians would say. Also not really an important point to argue about.
Glad to have highlighted the cardinality point!
Makes sense! There’s some old writers in the utilitarian tradition like James Griffin that define utilitarianism in the broader way, but I do think your articulation is probably more common.