Also, following up on 2, depending on how we make interpersonal utility comparisons, rather than focusing on those with low personal time discount rates, those with the largest preference-based stakes could be utilitarians, especially those with the widest moral circles, or people with fanatical views or absolutist deontological views.
Thanks for this, Michael. You’re right that if people could be kept alive a lot longer (and, perhaps, made to suffer more intensely than they once could as well), this could change the stakes. It will then come down to the probability you assign to a malicious AI’s inflicting this situation on people. If you thought it was likely enough (and I’m unsure what that threshold is), it could just straightforwardly follow that s-risk work beats all else. And perhaps there are folks in the community who think the likelihood is sufficiently high. If so, then what we’ve drafted here certainly shouldn’t sway them away from focusing on s-risk.
Oh, sorry if I was unclear. I didn’t have in mind torture scenarios here (although that’s a possibility), just people living very long voluntarily and to their own benefit. So rather than AMF saving like 50 years of valuable life in expectation per life saved, it could save thousands or millions or more. And other work may increase some individual’s life expectancy even more.
I think it’s not too unlikely that we’ll cure aging or solve mind uploading in our lifetimes, especially if we get superintelligence.
Also, following up on 2, depending on how we make interpersonal utility comparisons, rather than focusing on those with low personal time discount rates, those with the largest preference-based stakes could be utilitarians, especially those with the widest moral circles, or people with fanatical views or absolutist deontological views.
Thanks for this, Michael. You’re right that if people could be kept alive a lot longer (and, perhaps, made to suffer more intensely than they once could as well), this could change the stakes. It will then come down to the probability you assign to a malicious AI’s inflicting this situation on people. If you thought it was likely enough (and I’m unsure what that threshold is), it could just straightforwardly follow that s-risk work beats all else. And perhaps there are folks in the community who think the likelihood is sufficiently high. If so, then what we’ve drafted here certainly shouldn’t sway them away from focusing on s-risk.
Oh, sorry if I was unclear. I didn’t have in mind torture scenarios here (although that’s a possibility), just people living very long voluntarily and to their own benefit. So rather than AMF saving like 50 years of valuable life in expectation per life saved, it could save thousands or millions or more. And other work may increase some individual’s life expectancy even more.
I think it’s not too unlikely that we’ll cure aging or solve mind uploading in our lifetimes, especially if we get superintelligence.