In general, competition of various kinds seems like it has been one of the most positive forces for human development—competition between individuals for excellence, between scientists for innovation, between companies for cost-effectively meeting consumer wants, and between countries. Historically ‘uncoordinated’ competition has often had much better results than coordination!
I agree with the historical claim (with caveats below), but I think how that historical success ports over to future expected success is at best very murky.
A few comments here why:
Taking a firm moral stance, it’s not at all obvious that the non-centralised society today really is that great. It all depends on: (i) how animals are treated (including wild animals), if we’re just looking at the here and now; (ii) how existential risk is handled, if we’re also considering the long term. (Not claiming centralised societies would have been better, but they might have been, and even small chances of changes on how e.g. animals are treated would make a big difference.)
And there are particular reasons why liberal institutions worked that don’t apply to the post-AGI world:
There are radically diminishing returns to any person or group having greater resources. So it makes sense to divide up resources quite generally, and means that there are enormous gains from trade.
There are huge gains to be had from scientific and economic developments. So a society that achieves scientific and economic development basically ends up being the best society. Liberal democracy turns out to be great for that. But that won’t be a differentiator among societies in the future.
Different people have very different aims and values, and there are huge gains to preventing conflict between different groups. But (i) future beings can be designed; (ii) I suspect that conflict can be avoided even without liberal democracy.
I’m especially concerned that we have a lot of risk-averse intuitions that don’t port over to the case of cosmic ethics.
For example, when thinking about whether an autocracy would be good or bad, the natural thought is: “man, that could go really badly wrong, if the wrong person is in charge.” But, if cosmic value is linear in resources, then that’s not a great argument against autocracy-outside-of-our-solar-system; some real chance of a near-best outcome is better than a guarantee of an ok future.
And this argument gets stronger if the cosmic value of the future is the product of how well things go on many dimensions; if so, then you want success or failure on each of those dimensions to be correlated, which you get if there’s a single decision-maker.
(I feel really torn on centralisation vs decentralisation, and to be clear I strongly share the pro-liberalism pro-decentralisation prior.)
I don’t intend to do a separate post on this argument, but I’d love more discussion of the it, as it is a bit of a throwaway argument in the series, but potentially Big if true.
I agree with the historical claim (with caveats below), but I think how that historical success ports over to future expected success is at best very murky.
A few comments here why:
Taking a firm moral stance, it’s not at all obvious that the non-centralised society today really is that great. It all depends on: (i) how animals are treated (including wild animals), if we’re just looking at the here and now; (ii) how existential risk is handled, if we’re also considering the long term. (Not claiming centralised societies would have been better, but they might have been, and even small chances of changes on how e.g. animals are treated would make a big difference.)
And there are particular reasons why liberal institutions worked that don’t apply to the post-AGI world:
There are radically diminishing returns to any person or group having greater resources. So it makes sense to divide up resources quite generally, and means that there are enormous gains from trade.
There are huge gains to be had from scientific and economic developments. So a society that achieves scientific and economic development basically ends up being the best society. Liberal democracy turns out to be great for that. But that won’t be a differentiator among societies in the future.
Different people have very different aims and values, and there are huge gains to preventing conflict between different groups. But (i) future beings can be designed; (ii) I suspect that conflict can be avoided even without liberal democracy.
I’m especially concerned that we have a lot of risk-averse intuitions that don’t port over to the case of cosmic ethics.
For example, when thinking about whether an autocracy would be good or bad, the natural thought is: “man, that could go really badly wrong, if the wrong person is in charge.” But, if cosmic value is linear in resources, then that’s not a great argument against autocracy-outside-of-our-solar-system; some real chance of a near-best outcome is better than a guarantee of an ok future.
And this argument gets stronger if the cosmic value of the future is the product of how well things go on many dimensions; if so, then you want success or failure on each of those dimensions to be correlated, which you get if there’s a single decision-maker.
(I feel really torn on centralisation vs decentralisation, and to be clear I strongly share the pro-liberalism pro-decentralisation prior.)
I don’t intend to do a separate post on this argument, but I’d love more discussion of the it, as it is a bit of a throwaway argument in the series, but potentially Big if true.
Thanks for the response!