I can add four more to the list of possible dangers:
Effective altruists tend to be more utilitarian, more “maximizing” with regards to ethics, and more willing to bite philosophical bullets, compared to the general population. EA’s involvement in AI safety could end up spreading these beliefs to future AI. It’s well established how this sort of utilitarian reasoning can go wrong (utility monsters, etc), so this could increase the chance of disaster.
Effective altruists also have an implicit epistemology that differs from typical subject matter experts. If there are flaws in parts of the epistemology of EA or other groups involved in AI safety (which I believe to be true), then these flaws could be absorbed by future AI systems, and give rise to negative outcomes due to poor decisonmaking, etc.
People that are informed by AI safety arguments could decide that extreme actions are necessary to avoid AI doom, resulting in violence or rash actions that cause extreme harm.
The focus on AI safety in particular could result in less attention being paid to other threats to humanity that turn out to be more pressing, but are neglected due to the overrepresentation of AI safety people in the existential threat field.
I can add four more to the list of possible dangers:
Effective altruists tend to be more utilitarian, more “maximizing” with regards to ethics, and more willing to bite philosophical bullets, compared to the general population. EA’s involvement in AI safety could end up spreading these beliefs to future AI. It’s well established how this sort of utilitarian reasoning can go wrong (utility monsters, etc), so this could increase the chance of disaster.
Effective altruists also have an implicit epistemology that differs from typical subject matter experts. If there are flaws in parts of the epistemology of EA or other groups involved in AI safety (which I believe to be true), then these flaws could be absorbed by future AI systems, and give rise to negative outcomes due to poor decisonmaking, etc.
People that are informed by AI safety arguments could decide that extreme actions are necessary to avoid AI doom, resulting in violence or rash actions that cause extreme harm.
The focus on AI safety in particular could result in less attention being paid to other threats to humanity that turn out to be more pressing, but are neglected due to the overrepresentation of AI safety people in the existential threat field.
Interesting stuff! I probably don’t buy these as major worries but yeah, interesting.