Great point, Gregor! Tom Adamczewski has done an analysis which combines the answers to the questions about tasks and occupations. Here is the mainline graph.
Tom aggregates the results from the different questions in the most agnostic way possible, which I think is the best one can do.
I achieve this by simply including answers to both questions prior to aggregation, i.e. no special form of aggregation is used for aggregating tasks (HLMI) and occupations (FAOL). Since more respondents were asked about tasks than occupations, I achieve equal weight by resampling from the occupations (FAOL) responses.
Here is how Tom suggests people describe the results.
Experts were asked when it will be feasible to automate all tasks or occupations. The median expert thinks this is 20% likely by 2048, and 80% likely by 2103. There was substantial disagreement among experts. For automation by 2048, the middle half of experts assigned it a probability between 1% and a 60% (meaning ¼ assigned it a chance lower than 1%, and ¼ gave a chance higher than 60%). For automation by 2103, the central half of experts forecasts ranged from a 25% chance to a 100% chance.2
I find this completely inscrutable. I’m not saying there’s anything wrong with it in terms of accuracy, it’s just way too in the weeds of the statistics for me to decipher what’s going on.
For example, I don’t know what a “middle half” or a “central half” is. I looked them up and now I know they are statistics terms, but it would be a lot of work for me to try to figure out what that quoted paragraph is trying to say.
Is AI Impacts going to run this survey again soon? Maybe they can phrase the questions differently in a new survey to avoid this level of confusion between different levels of AI capabilities.
Assuming there were only 4 experts predicting full automation by 2048 with 10 %, 20 %, 30 %, and 40 % chance, the middle half of experts would be predicting full automation by 2048 with 20 % to 30 % chance. If there were 1 k experts each predicting the probability of full automation by multiple dates, one could infer 1 k predictions for the probability of full automation for any given date, order such preductions by ascending order, and then report the 25th and 75th percentile predictions as the lower and upper bound of the middle half of predictions.
Great point, Gregor! Tom Adamczewski has done an analysis which combines the answers to the questions about tasks and occupations. Here is the mainline graph.
Tom aggregates the results from the different questions in the most agnostic way possible, which I think is the best one can do.
Here is how Tom suggests people describe the results.
I find this completely inscrutable. I’m not saying there’s anything wrong with it in terms of accuracy, it’s just way too in the weeds of the statistics for me to decipher what’s going on.
For example, I don’t know what a “middle half” or a “central half” is. I looked them up and now I know they are statistics terms, but it would be a lot of work for me to try to figure out what that quoted paragraph is trying to say.
Is AI Impacts going to run this survey again soon? Maybe they can phrase the questions differently in a new survey to avoid this level of confusion between different levels of AI capabilities.
Hi Yarrow,
Assuming there were only 4 experts predicting full automation by 2048 with 10 %, 20 %, 30 %, and 40 % chance, the middle half of experts would be predicting full automation by 2048 with 20 % to 30 % chance. If there were 1 k experts each predicting the probability of full automation by multiple dates, one could infer 1 k predictions for the probability of full automation for any given date, order such preductions by ascending order, and then report the 25th and 75th percentile predictions as the lower and upper bound of the middle half of predictions.