One further thing that might help you get in my brain a bit is that I really am thinking this as more like “what values should we be aiming at to guide the future” and being fairly agnostic on mechanism rather than something like “let’s put democracy in The AGI’s model spec”. And I really am envisioning the argument as something like: “Wow, it really seems like the future is not a utilitarian one. Maybe sensitivity to minority values like utilitarianism is the best thing we can ask for.” — rather than something like “democracy good!” And that could mean a lot of different things. On Mondays, Wednesdays, and Fridays I think it means avoiding too much centralization and aiming for highly redistributive free market based futures as an approximation of Deep Democracy.
One further thing that might help you get in my brain a bit is that I really am thinking this as more like “what values should we be aiming at to guide the future” and being fairly agnostic on mechanism rather than something like “let’s put democracy in The AGI’s model spec”. And I really am envisioning the argument as something like: “Wow, it really seems like the future is not a utilitarian one. Maybe sensitivity to minority values like utilitarianism is the best thing we can ask for.” — rather than something like “democracy good!” And that could mean a lot of different things. On Mondays, Wednesdays, and Fridays I think it means avoiding too much centralization and aiming for highly redistributive free market based futures as an approximation of Deep Democracy.
Thanks, super useful! And makes me much more clear on the argument / sympathetic to it.