Great post! I’ll think about the ramifications later. One minor note:
As well as the unilateralist’s curse (where the most optimistic decision-maker determines what happens), there’s a risk of falling into what we could call the bureaucrat’s curse,[10] where everyone has a veto over the actions of others; in such a situation, if everyone follows their own best-guesses, then the most pessimistic decision-maker determines what happens.
If I understand Bostrom et.al’s paper correctly, this is just a special case of the unilateralist’s curse, and is mathematically equivalent (bottom of pg.7-8 in pdf, 355-356 in the journal):
Finally, fifth, though we have thus far focused on cases where a number of agents can undertake an initiative and it matters only whether at least one of them does so, a similar problem arises when any one of a group of agents can spoil an initiative—for instance, where universal action is required to bring about an intended outcome. Consider the following example:
[...]
These cases of unilateral spoiling or abstinence are formally equivalent to the original unilateralist curse, with merely the sign reversed.
Since the problem in these cases is the result of unilateral abstinence, it seems appropriate to include them within the scope of the unilateralist’s curse. Thus, in what follows, we assume that the unilateralist’s curse can arise when each member of a group can unilaterally undertake or spoil an initiative (though for ease of exposition we sometimes mention only the former case).
Great post! I’ll think about the ramifications later. One minor note:
If I understand Bostrom et.al’s paper correctly, this is just a special case of the unilateralist’s curse, and is mathematically equivalent (bottom of pg.7-8 in pdf, 355-356 in the journal):