Sure, itâs easy to dismiss the value of unaligned AIs if you compare against some idealistic baseline; but Iâm asking you to compare against a realistic baseline, i.e. actual human nature.
I havenât read your entire post about this, but I understand you believe that if we created aligned AI, it would get essentially âcurrentâ human values, rather than e.g. some improved /â more enlightened iteration of human values. If instead you believed the latter, that would set a significantly higher bar for unaligned AI, right?
If instead you believed the latter, that would set a significantly higher bar for unaligned AI, right?
Thatâs right, if I thought human values would improve greatly in the face of enormous wealth and advanced technology, Iâd definitely be open to seeing humans as special and extra valuable from a total utilitarian perspective. Note that many routes through which values could improve in the future could apply to unaligned AIs too. So, for example, Iâd need to believe that humans would be more likely to reflect, and be more likely to do the right type of reflection, relative to the unaligned baseline. In other words itâs not sufficient to argue that humans would reflect a little bit; that wouldnât really persuade me at all.
I havenât read your entire post about this, but I understand you believe that if we created aligned AI, it would get essentially âcurrentâ human values, rather than e.g. some improved /â more enlightened iteration of human values. If instead you believed the latter, that would set a significantly higher bar for unaligned AI, right?
Thatâs right, if I thought human values would improve greatly in the face of enormous wealth and advanced technology, Iâd definitely be open to seeing humans as special and extra valuable from a total utilitarian perspective. Note that many routes through which values could improve in the future could apply to unaligned AIs too. So, for example, Iâd need to believe that humans would be more likely to reflect, and be more likely to do the right type of reflection, relative to the unaligned baseline. In other words itâs not sufficient to argue that humans would reflect a little bit; that wouldnât really persuade me at all.