I actually think the most useful things to do to reduce s-risks can be conceptualized as part of the red box.
For one thing, solving global coordination seems really hard and the best way to solve it may include aligned AI, anyway. ”...and everyone actually follows that manual!” is the hard one, but I’d imagine the EA community will come up with some kind of serious attempt, and people interested in reducing s-risks may not have a comparative advantage at making that happen.
So we’re back to the red box.
I think people interested in reducing s-risks should mostly study alignment schemes and their goal architectures and pick ones that implement hyperexistential separation as much as possible. This produces not-terrible futures even if you fail to address the problem in the top-right blue box.
You might reply “AI alignment is too difficult to be picky, and we don’t have any promising approaches anyway.” In that case, you’d anyway have a large probability of an existential catastrophe, so you can just make sure people don’t try some Hail Mary thing that is unusually bad for s-risks.
By contrast, if you think AI alignment isn’t too difficult, there might be multiple approaches with a shot at working, and those predictably differ with respect to hyperexistential separation.
That’s a cool chart!
I actually think the most useful things to do to reduce s-risks can be conceptualized as part of the red box.
For one thing, solving global coordination seems really hard and the best way to solve it may include aligned AI, anyway. ”...and everyone actually follows that manual!” is the hard one, but I’d imagine the EA community will come up with some kind of serious attempt, and people interested in reducing s-risks may not have a comparative advantage at making that happen.
So we’re back to the red box.
I think people interested in reducing s-risks should mostly study alignment schemes and their goal architectures and pick ones that implement hyperexistential separation as much as possible. This produces not-terrible futures even if you fail to address the problem in the top-right blue box.
You might reply “AI alignment is too difficult to be picky, and we don’t have any promising approaches anyway.” In that case, you’d anyway have a large probability of an existential catastrophe, so you can just make sure people don’t try some Hail Mary thing that is unusually bad for s-risks.
By contrast, if you think AI alignment isn’t too difficult, there might be multiple approaches with a shot at working, and those predictably differ with respect to hyperexistential separation.