This isn’t exactly the same as self-interest, though. I think a better analogy for this might be human domestication of animals for agriculture. It’s not in the self-interest of a factory farmed chicken to be on a factory farm, but humans have power over which animals exist so we’ll make sure there are lots of animals who serve our interests. AI systems will be selected for to the extent they serve the interests of people making and buying them.
RE international development: competition between states undercut arguments for domestic safety regulations/practices. These are exacerbated by beliefs that international rivals will behave less safely/responsibly, but you don’t actually need to believe that to justify cutting corners domestically. If China or Russia built an AGI that was totally safe in the sense that it is aligned with its creators interests, that would be seen as a big threat by the US govt.
If you think that building AGI is extremely dangerous no matter who does it, then having more well-resourced players in the space increases the overall risk.
I think building AI systems with some level of autonomy/agency would make them much more useful, provided they are still aligned with the interests of their users/creators. There’s already evidence that companies are moving in this direction based on the business case: https://jacobin.com/2024/01/can-humanity-survive-ai#:~:text=Further%2C%20academics%20and,is%20pretty%20good.%E2%80%9D
This isn’t exactly the same as self-interest, though. I think a better analogy for this might be human domestication of animals for agriculture. It’s not in the self-interest of a factory farmed chicken to be on a factory farm, but humans have power over which animals exist so we’ll make sure there are lots of animals who serve our interests. AI systems will be selected for to the extent they serve the interests of people making and buying them.
RE international development: competition between states undercut arguments for domestic safety regulations/practices. These are exacerbated by beliefs that international rivals will behave less safely/responsibly, but you don’t actually need to believe that to justify cutting corners domestically. If China or Russia built an AGI that was totally safe in the sense that it is aligned with its creators interests, that would be seen as a big threat by the US govt.
If you think that building AGI is extremely dangerous no matter who does it, then having more well-resourced players in the space increases the overall risk.