Human extinction in particular is plausibly good or not very important relative to other things on asymmetric person-affecting views, especially animal-inclusive ones, so I think we would see extinction risk reduction relatively deemphasized. Of course, extinction is also plausibly very bad on these views, but the case for this is weaker without the astronomical waste argument.
AI safety’s focus would probably shift significantly, too, and some of it may already be of questionable value on person-affecting views today. I’m not an expert here, though.
Broad longtermist interventions don’t seem so robustly positive to me, in case the additional future capacity is used to do things that are in expectation bad or of deeply uncertain value according to person-affecting views, which is plausible if these views have relatively low representation in the future.
Broad longtermist interventions don’t seem so robustly positive to me, in case the additional future capacity is used to do things that are in expectation bad or of deeply uncertain value according to person-affecting views, which is plausible if these views have relatively low representation in the future.
Fair enough. I shouldn’t really have said these broad interventions are robust to person-affecting views because that is admittedly very unclear. I do find these broad interventions to be robustly positive overall though as I think we will get closer to the ‘correct’ population axiology over time.
I’m admittedly unsure if a “correct” axiology even exists, but I do think that continued research can uncover potential objections to different axiologies allowing us to make a more ‘informed’ decision.
AI safety’s focus would probably shift significantly, too, and some of it may already be of questionable value on person-affecting views today. I’m not an expert here, though.
I’ve heard the claim that optimal approaches to AI safety may depend on one’s ethical views, but I’ve never really seen a clear explanation how or why. I’d like to see a write-up of this.
Granted I’m not as read up on AI safety as many, but I’ve always got the impression that the AI safety problem really is “how can we make sure AI is aligned to human interests?”, which seems pretty robust to any ethical view. The only argument against this that I can think of is that human interests themselves could be flawed. If humans don’t care about say animals or artificial sentience, then it wouldn’t be good enough to have AI aligned to human interests—we would also need to expand humanity’s moral circle or ensure that those who create AGI have an expanded moral circle.
I think a person-affecting approach like the following is promising, and it and the others you’ve cited have received little attention in the EA community, parhaps in part because of their technical nature: https://globalprioritiesinstitute.org/teruji-thomas-the-asymmetry-uncertainty-and-the-long-term/
I wrote a short summary here: https://www.lesswrong.com/posts/Btqex9wYZmtPMnq9H/debating-myself-on-whether-extra-lives-lived-are-as-good-as?commentId=yidnhcNqLmSGCsoG9
Human extinction in particular is plausibly good or not very important relative to other things on asymmetric person-affecting views, especially animal-inclusive ones, so I think we would see extinction risk reduction relatively deemphasized. Of course, extinction is also plausibly very bad on these views, but the case for this is weaker without the astronomical waste argument.
AI safety’s focus would probably shift significantly, too, and some of it may already be of questionable value on person-affecting views today. I’m not an expert here, though.
Broad longtermist interventions don’t seem so robustly positive to me, in case the additional future capacity is used to do things that are in expectation bad or of deeply uncertain value according to person-affecting views, which is plausible if these views have relatively low representation in the future.
Fair enough. I shouldn’t really have said these broad interventions are robust to person-affecting views because that is admittedly very unclear. I do find these broad interventions to be robustly positive overall though as I think we will get closer to the ‘correct’ population axiology over time.
I’m admittedly unsure if a “correct” axiology even exists, but I do think that continued research can uncover potential objections to different axiologies allowing us to make a more ‘informed’ decision.
I’ve heard the claim that optimal approaches to AI safety may depend on one’s ethical views, but I’ve never really seen a clear explanation how or why. I’d like to see a write-up of this.
Granted I’m not as read up on AI safety as many, but I’ve always got the impression that the AI safety problem really is “how can we make sure AI is aligned to human interests?”, which seems pretty robust to any ethical view. The only argument against this that I can think of is that human interests themselves could be flawed. If humans don’t care about say animals or artificial sentience, then it wouldn’t be good enough to have AI aligned to human interests—we would also need to expand humanity’s moral circle or ensure that those who create AGI have an expanded moral circle.
I would recommend CLR’s and CRS’s writeups for what more s-risk-focused work looks like:
https://longtermrisk.org/research-agenda
https://www.alignmentforum.org/posts/EzoCZjTdWTMgacKGS/clr-s-recent-work-on-multi-agent-systems
https://centerforreducingsuffering.org/open-research-questions/ (especially the section Agential s-risks)