Yes, perhaps I’m just injecting some of my broader concerns about who gets to use the word “safety” here.
I’m thinking of scenario 4 here:
4. A researcher looked into AI risk, and is convinced that AI could be highly dangerous if misused, and that “misalignment” is a serious problem that could lead to some very bad outcomes, from increased sexism to significant amount of deaths from say misaligned weapon systems or healthcare diagnosis. However, they think the arguments for existential risk are very flimsy, that any x-risk threat is very far away, and that it’s legitimately a waste of time to work on x-risk. So they focus their team on preventing near and medium term danger from existing AI systems.
In case it wasn’t obvious, the opinion of the researcher is the one I hold, and I know a significant amount of others do as well (some perhaps secretly). I don’t think it’s wrong for them to claim they are working on “AI safety”, when they are literally working on making AI safer, but it seems like they would be open to accusations of safewashing if they made that claim.
I like your suggestion of using the phrase “existential safety” instead, I think it would clear up a lot of confusion.
Yes, perhaps I’m just injecting some of my broader concerns about who gets to use the word “safety” here.
I’m thinking of scenario 4 here:
4. A researcher looked into AI risk, and is convinced that AI could be highly dangerous if misused, and that “misalignment” is a serious problem that could lead to some very bad outcomes, from increased sexism to significant amount of deaths from say misaligned weapon systems or healthcare diagnosis. However, they think the arguments for existential risk are very flimsy, that any x-risk threat is very far away, and that it’s legitimately a waste of time to work on x-risk. So they focus their team on preventing near and medium term danger from existing AI systems.
In case it wasn’t obvious, the opinion of the researcher is the one I hold, and I know a significant amount of others do as well (some perhaps secretly). I don’t think it’s wrong for them to claim they are working on “AI safety”, when they are literally working on making AI safer, but it seems like they would be open to accusations of safewashing if they made that claim.
I like your suggestion of using the phrase “existential safety” instead, I think it would clear up a lot of confusion.