Hi Michael. Are you assuming the deployment of ASI will be analogous to an omnipotent civilisation with values completely disconnected from humans suddenly showing up on Earth? I agree human extinction would be a real possibility in this case. However, that would be very much at odds with historical gradual technological development shapedby human values.
Are you assuming the deployment of ASI will be analogous to an omnipotent civilisation with values completely disconnected from humans suddenly showing up on Earth?
Something like that, yeah.
However, that would be very much at odds with historical gradual technological development shaped by human values.
ASI would have a level of autonomy and goal-directedness that’s unlike any previous technology. The case for caring about AI risk doesn’t work if you take too much of an outside view, you have to reason about what properties ASI would have.
I see. I guess the probability over the next 10 years of something like what I described is lower than 0.001 %. I remain open tobetsagainst short AI timelines, or what they supposedly imply, up to 10 k$. Do you see any that we could make that is good for both of us considering we could invest our money, and that you could take loans?
I spent a few days thinking about this but I struggled to come up with a bet structure that I was confident was good for both of us. The financial implications of this sort of bet are complicated. I didn’t want to spend more time on it so I’ll punt on this for now but I will keep it in the back of my mind in case I come up with anything.
Hi Michael. Are you assuming the deployment of ASI will be analogous to an omnipotent civilisation with values completely disconnected from humans suddenly showing up on Earth? I agree human extinction would be a real possibility in this case. However, that would be very much at odds with historical gradual technological development shaped by human values.
Something like that, yeah.
ASI would have a level of autonomy and goal-directedness that’s unlike any previous technology. The case for caring about AI risk doesn’t work if you take too much of an outside view, you have to reason about what properties ASI would have.
I see. I guess the probability over the next 10 years of something like what I described is lower than 0.001 %. I remain open to bets against short AI timelines, or what they supposedly imply, up to 10 k$. Do you see any that we could make that is good for both of us considering we could invest our money, and that you could take loans?
I spent a few days thinking about this but I struggled to come up with a bet structure that I was confident was good for both of us. The financial implications of this sort of bet are complicated. I didn’t want to spend more time on it so I’ll punt on this for now but I will keep it in the back of my mind in case I come up with anything.
Thanks, Michael.