I take 5%-60% as an estimate of how much of human civilization’s future value will depend on what AI systems do, but it does not necessarily exclude human autonomy. If humans determine what AI systems do with the resources they acquire and the actions they take, then AI could be extremely important, and humans would still retain autonomy.
I don’t think this really left me more or less concerned about losing autonomy over resources. It does feel like this exercise made it starker that there’s a large chance of AI reshaping the world beyond human extinction. It’s not clear how much of that means the loss of human autonomy. I’m inclined to think in rough, nebulous terms that AI will erode human autonomy over 10% of our future, taking 10% as a sort of midpoint between the extinction likelihood and the degree of AI influence over our future. I think my previous views would have been in that ballpark.
The exercise did lead me to think the importance of AI is higher than I previously did and the likelihood of extinction per se is lower (though my final beliefs place all these probabilities higher than the priors in the report).
I take 5%-60% as an estimate of how much of human civilization’s future value will depend on what AI systems do, but it does not necessarily exclude human autonomy. If humans determine what AI systems do with the resources they acquire and the actions they take, then AI could be extremely important, and humans would still retain autonomy.
I don’t think this really left me more or less concerned about losing autonomy over resources. It does feel like this exercise made it starker that there’s a large chance of AI reshaping the world beyond human extinction. It’s not clear how much of that means the loss of human autonomy. I’m inclined to think in rough, nebulous terms that AI will erode human autonomy over 10% of our future, taking 10% as a sort of midpoint between the extinction likelihood and the degree of AI influence over our future. I think my previous views would have been in that ballpark.
The exercise did lead me to think the importance of AI is higher than I previously did and the likelihood of extinction per se is lower (though my final beliefs place all these probabilities higher than the priors in the report).