CLR’s agenda is aimed at reducing conflict among TAI-enabled actors, and includes credibility and peaceful bargaining, which both seem technical:
https://longtermrisk.org/research-agenda
I’d guess they’re the only (10+ team member) group with multi-agent systems and multipolar scenarios among their main focuses within their technical research.
I think the problem is that when I said “technical AGI safety”, I was thinking the red box, whereas you were thinking “any technical topic in either the red or blue boxes”. I agree that there are technical topics in the top-right blue box in particular, and that’s where “conflict scenarios” would mainly be. My understanding is that working on those topics does not have much of a direct connection to AGI, in the sense that technologies for reducing human-human conflict tend to overlap with technologies for reducing AGI-AGI conflict. (At least, according to this comment thread, I haven’t personally thought about it much.)
Anyway, I guess you would say “in a more s-risk-focused world, we would be working more on the top-right blue box and less on the red box”. But really, in a more s-risk-focused world, we would be working more on all three colored boxes. :-P I’m not an expert on the ITN of particular projects within the blue boxes, and therefore don’t have a strong opinion about how to weigh them against particular projects within the red box. I am concerned / pessimistic about prospects for success in the red box. But maybe if I knew more about the state of the blue boxes, I would be equally concerned / pessimistic about those too!! ¯\_(ツ)_/¯
I actually think the most useful things to do to reduce s-risks can be conceptualized as part of the red box.
For one thing, solving global coordination seems really hard and the best way to solve it may include aligned AI, anyway. ”...and everyone actually follows that manual!” is the hard one, but I’d imagine the EA community will come up with some kind of serious attempt, and people interested in reducing s-risks may not have a comparative advantage at making that happen.
So we’re back to the red box.
I think people interested in reducing s-risks should mostly study alignment schemes and their goal architectures and pick ones that implement hyperexistential separation as much as possible. This produces not-terrible futures even if you fail to address the problem in the top-right blue box.
You might reply “AI alignment is too difficult to be picky, and we don’t have any promising approaches anyway.” In that case, you’d anyway have a large probability of an existential catastrophe, so you can just make sure people don’t try some Hail Mary thing that is unusually bad for s-risks.
By contrast, if you think AI alignment isn’t too difficult, there might be multiple approaches with a shot at working, and those predictably differ with respect to hyperexistential separation.
CLR’s agenda is aimed at reducing conflict among TAI-enabled actors, and includes credibility and peaceful bargaining, which both seem technical: https://longtermrisk.org/research-agenda
Their technical research is mostly on multi-agent systems and decision theory: https://longtermrisk.org/publications/
I’d guess they’re the only (10+ team member) group with multi-agent systems and multipolar scenarios among their main focuses within their technical research.
OK, thanks. Here’s a chart I made:
I think the problem is that when I said “technical AGI safety”, I was thinking the red box, whereas you were thinking “any technical topic in either the red or blue boxes”. I agree that there are technical topics in the top-right blue box in particular, and that’s where “conflict scenarios” would mainly be. My understanding is that working on those topics does not have much of a direct connection to AGI, in the sense that technologies for reducing human-human conflict tend to overlap with technologies for reducing AGI-AGI conflict. (At least, according to this comment thread, I haven’t personally thought about it much.)
Anyway, I guess you would say “in a more s-risk-focused world, we would be working more on the top-right blue box and less on the red box”. But really, in a more s-risk-focused world, we would be working more on all three colored boxes. :-P I’m not an expert on the ITN of particular projects within the blue boxes, and therefore don’t have a strong opinion about how to weigh them against particular projects within the red box. I am concerned / pessimistic about prospects for success in the red box. But maybe if I knew more about the state of the blue boxes, I would be equally concerned / pessimistic about those too!! ¯\_(ツ)_/¯
That’s a cool chart!
I actually think the most useful things to do to reduce s-risks can be conceptualized as part of the red box.
For one thing, solving global coordination seems really hard and the best way to solve it may include aligned AI, anyway. ”...and everyone actually follows that manual!” is the hard one, but I’d imagine the EA community will come up with some kind of serious attempt, and people interested in reducing s-risks may not have a comparative advantage at making that happen.
So we’re back to the red box.
I think people interested in reducing s-risks should mostly study alignment schemes and their goal architectures and pick ones that implement hyperexistential separation as much as possible. This produces not-terrible futures even if you fail to address the problem in the top-right blue box.
You might reply “AI alignment is too difficult to be picky, and we don’t have any promising approaches anyway.” In that case, you’d anyway have a large probability of an existential catastrophe, so you can just make sure people don’t try some Hail Mary thing that is unusually bad for s-risks.
By contrast, if you think AI alignment isn’t too difficult, there might be multiple approaches with a shot at working, and those predictably differ with respect to hyperexistential separation.