I have a rough (i.e. considered for <15 minutes) take: if top labs one year ago had attempted these particular milestones, and had the same policies on disclosing capabilities as they currently seem to, then there’s a 40-50% chance they would have achieved 2 of Angry Birds,Atari fifty ,Laundry and Go low by now. But I don’t put much weight on my prediction, whereas I put a lot more weight on my analysis of what has happened (though this is also somewhat subjective!).
I agree though that checking what has actually happened ends up being more conservative than the original “Ignore the question of whether they would choose to” , which makes the expert forecasts worse than they seem to be here. This is a weakness of this analysis! And of the resolvability of the original survey.
Do you have an estimate of how many of the tasks would have been achieved by now if labs tried a year ago?
Thanks Adam :)
I have a rough (i.e. considered for <15 minutes) take: if top labs one year ago had attempted these particular milestones, and had the same policies on disclosing capabilities as they currently seem to, then there’s a 40-50% chance they would have achieved 2 of Angry Birds, Atari fifty , Laundry and Go low by now. But I don’t put much weight on my prediction, whereas I put a lot more weight on my analysis of what has happened (though this is also somewhat subjective!).
I agree though that checking what has actually happened ends up being more conservative than the original “Ignore the question of whether they would choose to” , which makes the expert forecasts worse than they seem to be here. This is a weakness of this analysis! And of the resolvability of the original survey.
Do you have an estimate of how many of the tasks would have been achieved by now if labs tried a year ago?