While this is not a risk for “better collective decision-making”, I think that there might be some inherent risks when doing projects that attempt at improving collective decision-making on a global scale.
The tool itself might be flawed in subtle ways and still be used widely. Say, the tool might work very well for situations that are characterized by one of {quantitative, high-probabilities, homogeneous group of users, only for economic questions} but not in their converse.
If the tool is recognizably flawed, that can bias people to make decisions only on where the tool works well. For example:
Some criticisms of EA, and GiveWell in particular, have argued that the focus on quantitatively measurable and verifiable outcomes leads to recommending interventions that are less important than more fuzzy alternatives.
Some criticisms of capitalism argue that it leads to systemic flaws because of the strong measurability inherent in financial trade.
There might be better projects that could not happen if this project succeeds.
If the tool is very successful, it might lead to some form of “epistemic lock-in” where for many years there seems to be only one major way of making decisions.
Perhaps our current understanding of group epistemics is misleading, and what would seem to be an improvement would actually damage decision making.
Damaging collective decision-making
While this is not a risk for “better collective decision-making”, I think that there might be some inherent risks when doing projects that attempt at improving collective decision-making on a global scale.
The tool itself might be flawed in subtle ways and still be used widely. Say, the tool might work very well for situations that are characterized by one of {quantitative, high-probabilities, homogeneous group of users, only for economic questions} but not in their converse.
If the tool is recognizably flawed, that can bias people to make decisions only on where the tool works well. For example:
Some criticisms of EA, and GiveWell in particular, have argued that the focus on quantitatively measurable and verifiable outcomes leads to recommending interventions that are less important than more fuzzy alternatives.
Some criticisms of capitalism argue that it leads to systemic flaws because of the strong measurability inherent in financial trade.
There might be better projects that could not happen if this project succeeds.
If the tool is very successful, it might lead to some form of “epistemic lock-in” where for many years there seems to be only one major way of making decisions.
Perhaps our current understanding of group epistemics is misleading, and what would seem to be an improvement would actually damage decision making.