What’s the expected value of working in AI safety?
I’m not certain about longtermism and the value of reducing x-risks, I’m not optimistic that we can really affect the long future, and I guess the future of humanity may be bad. Many EA people are like me, that’s why only 15% people think AI safety is top cause area(survey by Rethink Priority).
However, In a “near-termist” view, AI safety research is still valuable because researching it can may avoid catastrophe(not only extinction), which causes the suffering of 8 billion people and maybe animals. But, things like researching on global health, preventing pandemic seems to have a more certain “expected value”(Maybe 100 QALY/extra person or so). Because we have our history experiences and a feedback loop. AI safety is the most difficult problem on earth, I feel like the expected value is like”???” It may be very high, may be 0. We don’t know how serious suffering it would make(it may cause extinction in a minute when we’re sleeping, or torture us for years?) We don’t know if we are on the way finding the soultion, or we are all doing the wrong predictions of AGI’s thoughts? Will the government control the power of AGI? All of the work on AI safety is kind of “guessing”, so I’m confused why 80000 hours estimates the tracability to be 1%. I know AI safety is highly neglected, and it may cause unpredictable huge suffering for human and animals. But if I work in AI safety, I’d feel a little lost becuase I don’t know if I really did something meaningful, if I don’t work in AI safety, I’d feel guilty. Could some give me(and the people who hestitates to work in AI safety) some recommendations?
Work related to AI trajectories can still be important even if you think the expected value of the far future is net negative (as I do, relative to my roughly negative-utilitarian values). In addition to alignment, we can also work on reducing s-risks that would result from superintelligence. This work tends to be somewhat different from ordinary AI alignment, although some types of alignment work may reduce s-risks also. (Some alignment work might increase s-risks.)
If you’re not a longtermist or think we’re too clueless about the long-run future, then this work would be less worthwhile. That said, AI will still be hugely disruptive even in the next few years, so we should pay some attention to it regardless of what else we’re doing.
It’s probably hard to evaluate the expected value of AI safety because the field is evolving extremely fast in the last year. A year ago we didn’t have DALL-E-2 or GPT-4 and if you would have asked me the same question a year ago I would have told you that:
It’s maybe comparable with Covid, before the pandemic people were advocating for measures to take to prevent or limit the impact of pandemics, but the expected value was very uncertain. Now that Covid happened you have concrete data showing how many people died because of it and can with more certainty say, preventing something similar will have this expected value.
I hope it won’t be necessary for an “AI Covid” to happen for people to start to take things seriously, but I think many very smart people think that there are substantial risks with AI and currently a lot of money is being spent to further the advancement of AI. Chat GPT is the fastest growing product in history!
In comparison the amount of money being spent on AI safety is still from my understanding limited, so if we draw the comparison to pandemic risks, imagine before covid and crisper is open source and the fastest growing product on the planet. Everyone is racing to find ways to make it more accessible, more powerful while, at least funding wise, neglecting safety.
In that timeline people have access to create powerful biological viruses, in our timeline people might have access to powerful computer viruses.
To close, I think it’s hard to evaluate expected value if you haven’t seen the damage yet, but I would hope we don’t need to see the damage and it’s up to each person to make a judgement call on where to spend their time and resources. I wish it was as simple as looking at QALY and then just sort by highest QALY and working on that, but especially in the high risk areas there seems to often be very high uncertainty. Maybe people that have a higher tolerance for uncertainty should focus on those areas because personal fit matters, if you have a low tolerance for uncertainty you might not pursue the field for long.
I apologise for being a bit glib here but: I find it obvious that it would be bad (in itself, ignoring effects on animals or the chance we do space genocide a million years from now etc.) if every human on Earth was suddenly murdered, even if it happened in our sleep and involved zero pain and suffering. And I think this is the normal view outside EA.
(I think your questions are excellent overall though.)
What’s the expected value of working in AI safety?
I’m not certain about longtermism and the value of reducing x-risks, I’m not optimistic that we can really affect the long future, and I guess the future of humanity may be bad. Many EA people are like me, that’s why only 15% people think AI safety is top cause area(survey by Rethink Priority).
However, In a “near-termist” view, AI safety research is still valuable because researching it can may avoid catastrophe(not only extinction), which causes the suffering of 8 billion people and maybe animals. But, things like researching on global health, preventing pandemic seems to have a more certain “expected value”(Maybe 100 QALY/extra person or so). Because we have our history experiences and a feedback loop. AI safety is the most difficult problem on earth, I feel like the expected value is like”???” It may be very high, may be 0. We don’t know how serious suffering it would make(it may cause extinction in a minute when we’re sleeping, or torture us for years?) We don’t know if we are on the way finding the soultion, or we are all doing the wrong predictions of AGI’s thoughts? Will the government control the power of AGI? All of the work on AI safety is kind of “guessing”, so I’m confused why 80000 hours estimates the tracability to be 1%. I know AI safety is highly neglected, and it may cause unpredictable huge suffering for human and animals. But if I work in AI safety, I’d feel a little lost becuase I don’t know if I really did something meaningful, if I don’t work in AI safety, I’d feel guilty. Could some give me(and the people who hestitates to work in AI safety) some recommendations?
Work related to AI trajectories can still be important even if you think the expected value of the far future is net negative (as I do, relative to my roughly negative-utilitarian values). In addition to alignment, we can also work on reducing s-risks that would result from superintelligence. This work tends to be somewhat different from ordinary AI alignment, although some types of alignment work may reduce s-risks also. (Some alignment work might increase s-risks.)
If you’re not a longtermist or think we’re too clueless about the long-run future, then this work would be less worthwhile. That said, AI will still be hugely disruptive even in the next few years, so we should pay some attention to it regardless of what else we’re doing.
It’s probably hard to evaluate the expected value of AI safety because the field is evolving extremely fast in the last year. A year ago we didn’t have DALL-E-2 or GPT-4 and if you would have asked me the same question a year ago I would have told you that:
“AI safety will solve itself because of backwards compatibility”
But I was wrong / see it differently now.
It’s maybe comparable with Covid, before the pandemic people were advocating for measures to take to prevent or limit the impact of pandemics, but the expected value was very uncertain. Now that Covid happened you have concrete data showing how many people died because of it and can with more certainty say, preventing something similar will have this expected value.
I hope it won’t be necessary for an “AI Covid” to happen for people to start to take things seriously, but I think many very smart people think that there are substantial risks with AI and currently a lot of money is being spent to further the advancement of AI. Chat GPT is the fastest growing product in history!
In comparison the amount of money being spent on AI safety is still from my understanding limited, so if we draw the comparison to pandemic risks, imagine before covid and crisper is open source and the fastest growing product on the planet. Everyone is racing to find ways to make it more accessible, more powerful while, at least funding wise, neglecting safety.
In that timeline people have access to create powerful biological viruses, in our timeline people might have access to powerful computer viruses.
To close, I think it’s hard to evaluate expected value if you haven’t seen the damage yet, but I would hope we don’t need to see the damage and it’s up to each person to make a judgement call on where to spend their time and resources. I wish it was as simple as looking at QALY and then just sort by highest QALY and working on that, but especially in the high risk areas there seems to often be very high uncertainty. Maybe people that have a higher tolerance for uncertainty should focus on those areas because personal fit matters, if you have a low tolerance for uncertainty you might not pursue the field for long.
I apologise for being a bit glib here but: I find it obvious that it would be bad (in itself, ignoring effects on animals or the chance we do space genocide a million years from now etc.) if every human on Earth was suddenly murdered, even if it happened in our sleep and involved zero pain and suffering. And I think this is the normal view outside EA.
(I think your questions are excellent overall though.)
Thanks for answering, I respect your value about x-risks (I’d consider if I was wrong)