Maintaining a careful and aligned measure of welfare is likely to be extremely difficult. It is hard to capture everything we value as a society (especially on different levels, like cities and states), and it would also be very difficult to avoid manipulations. Hanson notes this issue (in objections 13-15, 22-23), but does not treat it with the seriousness it deserves. Additionally, Hanson occasionally proposes modifying the measure of welfare to fix other issues, and this is an added complication.
A simpler measure of welfare might, for instance, prompt blind maximization of something that is not quite aligned with our values. If we try to compensate by adding everything we value, however, we may encounter issues of corruption in the measurement processes for certain parameters, encode policies in our measure of welfare (an oversimplified example of this is adding miles of roads built to the measure of welfare), or create a more messy system by attempting to solve other problems (e.g. on page 24, Hanson mentions the possibility of agreeing, by treaty, to give welfare weight to other nations’ welfare).
I’m not sure that it would be any harder than in a society without futarchy. In some ways I think it’s quite neat that Hanson acknowledges this would be the problem of the legislator and that people could vote for politicians they thought would derive a good function.
All the bad situations I can think of here apply to current societies so it seems harsh to judge futarchy by those standards.
I agree that we’re not currently good at “maximizing welfare,” but I worry that futarchy would lead to issues stemming from over-optimization of a measure that is misaligned from what we actually want. In other words, my worry is that common sense barriers would be removed under futarchy (or we would lose sight of what we actually care about after outlining an explicit welfare measure), and we would over-optimize whatever is outlined in our measure of welfare, which is never going to be perfectly aligned to our actual needs/desires/values.
This is a version of Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.” (Or possibly Campbell’s law, which is more specific.)
I’m not sure that it would be any harder than in a society without futarchy. In some ways I think it’s quite neat that Hanson acknowledges this would be the problem of the legislator and that people could vote for politicians they thought would derive a good function.
All the bad situations I can think of here apply to current societies so it seems harsh to judge futarchy by those standards.
I agree that we’re not currently good at “maximizing welfare,” but I worry that futarchy would lead to issues stemming from over-optimization of a measure that is misaligned from what we actually want. In other words, my worry is that common sense barriers would be removed under futarchy (or we would lose sight of what we actually care about after outlining an explicit welfare measure), and we would over-optimize whatever is outlined in our measure of welfare, which is never going to be perfectly aligned to our actual needs/desires/values.
This is a version of Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.” (Or possibly Campbell’s law, which is more specific.)