You’re sort of right on the first point, and I’ve definitely counted that work in my views on the area. I generally prefer to refer to it as ‘making sure the future goes well for non-humans’ - but I’ve had that misinterpreted as just focused on animals. I
I think for me the fact that the minds will be non-human, and probably digital, matter a lot. Firstly, I think arguments for longtermism probably don’t work if the future is mostly just humans. Secondly, the fact that these beings are digital minds, and maybe digital minds very different to us, means a lot of common responses that are given for how to make the future go well (eg make sure they’re preferred government ‘wins’ the ASI race) definitely looks less promising me. Plus you run into trickier problems like what Carlsmith discusses in his Otherness and Control series, and on the other end, if conscious AIs are ‘small minds’ ala insects (lots of small conscious digital minds that are maybe not individually very smart) you run into a bunch of the same issues of how to adequately treat them. So this is sort of why I call it ‘digital minds’, but I guess thats fairly semantic.
On you’re second point, I basically think it could go either way. I think this depends on a bunch of things, including if, how strong and what type (ie what values are encoded) of ‘lock in’ we get, how ‘adaptive’ consciousness is etc. At least to me, I could see it going either way (not saying 50-50 credence towards both, but my guess is I’m at least less skeptical than you). Also, its possible that these are the more likely scenarios to have abundant suffering (although this also isn’t obvious to me given potential motivations for causing deliberate suffering).
You’re sort of right on the first point, and I’ve definitely counted that work in my views on the area. I generally prefer to refer to it as ‘making sure the future goes well for non-humans’ - but I’ve had that misinterpreted as just focused on animals. I
I think for me the fact that the minds will be non-human, and probably digital, matter a lot. Firstly, I think arguments for longtermism probably don’t work if the future is mostly just humans. Secondly, the fact that these beings are digital minds, and maybe digital minds very different to us, means a lot of common responses that are given for how to make the future go well (eg make sure they’re preferred government ‘wins’ the ASI race) definitely looks less promising me. Plus you run into trickier problems like what Carlsmith discusses in his Otherness and Control series, and on the other end, if conscious AIs are ‘small minds’ ala insects (lots of small conscious digital minds that are maybe not individually very smart) you run into a bunch of the same issues of how to adequately treat them. So this is sort of why I call it ‘digital minds’, but I guess thats fairly semantic.
On you’re second point, I basically think it could go either way. I think this depends on a bunch of things, including if, how strong and what type (ie what values are encoded) of ‘lock in’ we get, how ‘adaptive’ consciousness is etc. At least to me, I could see it going either way (not saying 50-50 credence towards both, but my guess is I’m at least less skeptical than you). Also, its possible that these are the more likely scenarios to have abundant suffering (although this also isn’t obvious to me given potential motivations for causing deliberate suffering).