Wow that’s super interesting, and not what I would have expected. I appreciate you making this explicit.
My first reaction is that you and I are talking to different groups of people. I frequently encounter the claim that AI will “solve cultivated meat” and so we shouldn’t worry about farmed animals as long as AI doesn’t kill everyone. That’s what I was mainly reacting to here. I don’t work in AI safety though, so I wouldn’t be surprised if the group of people you talk to has thought about it more than the group of people I talk to.
My second reaction is that I don’t understand how biological humans disappearing is different than the disastrous x-risk scenarios that I associate EAs as trying to avoid. Like it seems like EAs are worried about AI scheming in ways that would cause them to make a power grab against humanity’s interest. If instead humans disappear for other (perhaps less violent) reasons, how is that then consistent with AI safety?
Maybe the answer is that humans would be succeeded by some other form of life that are sufficiently similar to humans that it would be better for that form of life to exist than squiggles?
If that’s the claim (which I actually find plausible), then I agree animals would no longer be farmed. However, it seems like this would fall into the bucket of “not actionable unless you work directly on AI,” so it seems like it might be practically useful to act as if this wasn’t going to be the case?
(Also, this claim feels at odds with what I understood your perspective to be from the shallow review you did a while ago. I haven’t had a chance to go back and more carefully read that piece, and maybe on a closer reason it will all look consistent. But I’m just flagging that I still can’t fully model your perspective)
However, it seems like this would fall into the bucket of “not actionable unless you work directly on AI,” so it seems like it might be practically useful to act as if this wasn’t going to be the case?
I think there’s a decent chance that AI won’t actually be very transformative (or at least won’t be transformative soon) and therefore it’s reasonable to bet your career plans on that assumption. But, to the extent you think AI is actually going to be a big deal, I would suggest considering working on it!
During the century we’re in right now, we will develop technologies that cause us to transition to a state in which humans as we know them are no longer the main force in world events. This is our last chance to shape how that transition happens.
Whatever the main force in world events is (perhaps digital people, misaligned AI, or something else) will create highly stable civilizations that populate our entire galaxy for billions of years to come. The transition taking place this century could shape all of that.
I think it’s very unclear whether this would be a good or bad thing.
There are some people (e.g. Evitable) who are trying to stop this transition, but I think more AI safety people would self-describe their work as “shap[ing] how that transition happens.”
Wow that’s super interesting, and not what I would have expected. I appreciate you making this explicit.
My first reaction is that you and I are talking to different groups of people. I frequently encounter the claim that AI will “solve cultivated meat” and so we shouldn’t worry about farmed animals as long as AI doesn’t kill everyone. That’s what I was mainly reacting to here. I don’t work in AI safety though, so I wouldn’t be surprised if the group of people you talk to has thought about it more than the group of people I talk to.
My second reaction is that I don’t understand how biological humans disappearing is different than the disastrous x-risk scenarios that I associate EAs as trying to avoid. Like it seems like EAs are worried about AI scheming in ways that would cause them to make a power grab against humanity’s interest. If instead humans disappear for other (perhaps less violent) reasons, how is that then consistent with AI safety?
Maybe the answer is that humans would be succeeded by some other form of life that are sufficiently similar to humans that it would be better for that form of life to exist than squiggles?
If that’s the claim (which I actually find plausible), then I agree animals would no longer be farmed. However, it seems like this would fall into the bucket of “not actionable unless you work directly on AI,” so it seems like it might be practically useful to act as if this wasn’t going to be the case?
(Also, this claim feels at odds with what I understood your perspective to be from the shallow review you did a while ago. I haven’t had a chance to go back and more carefully read that piece, and maybe on a closer reason it will all look consistent. But I’m just flagging that I still can’t fully model your perspective)
Good question and I claim the answer is “no” because you can work on AI! E.g. The Midas Project (founded by a former THL campaigner) is bringing corporate campaign tactics to AI safety. See also e.g. AI Safety’s Biggest Talent Gap Isn’t Researchers. It’s Generalists.
I think there’s a decent chance that AI won’t actually be very transformative (or at least won’t be transformative soon) and therefore it’s reasonable to bet your career plans on that assumption. But, to the extent you think AI is actually going to be a big deal, I would suggest considering working on it!
What I said was unclear; I probably should have just quoted Holden Karnofsky:
There are some people (e.g. Evitable) who are trying to stop this transition, but I think more AI safety people would self-describe their work as “shap[ing] how that transition happens.”
Yeah, I think I did a bad job explaining my views there. Could you say more about what you thought I believed? I should maybe update the post.