I qualitatively agree with the points you make, but I do not support a global pause. I think tail risk is very low (I guess the risk of human extinction over the next 10 years is 10^-7), the upside very large, and I expect people to overreact to AI risk. For example, it seems that people dislike deaths caused by autonomous driving much more than deaths caused by human driving, and I expect the adoption of autonomous cars to be slower than what would be ideal to prevent deaths from road accidents. I would be very surprised if a global pause passed a standard cost-benefit analysis in the sense of having a benefit-to-cost-ratio higher than 1.
Some complain there should be much more spending on AI safety because it is currently much smaller than that on AI capabilities, but these categories are vague, and I have not seen any detailed quantitative modelling showing that increasing spending on AI safety is very cost-effective. I do not think one can assume the spending on each category should ideally be the same.
Great post, Holly! Strongly upvoted.
I qualitatively agree with the points you make, but I do not support a global pause. I think tail risk is very low (I guess the risk of human extinction over the next 10 years is 10^-7), the upside very large, and I expect people to overreact to AI risk. For example, it seems that people dislike deaths caused by autonomous driving much more than deaths caused by human driving, and I expect the adoption of autonomous cars to be slower than what would be ideal to prevent deaths from road accidents. I would be very surprised if a global pause passed a standard cost-benefit analysis in the sense of having a benefit-to-cost-ratio higher than 1.
Some complain there should be much more spending on AI safety because it is currently much smaller than that on AI capabilities, but these categories are vague, and I have not seen any detailed quantitative modelling showing that increasing spending on AI safety is very cost-effective. I do not think one can assume the spending on each category should ideally be the same.