Thanks for putting this together! I think more scrutiny on these ideas is incredibly important so I’m delighted to see you approach it.
So meta to red team a red team, but some things I want to comment on:
Your median estimate for the conservative and aggressive bioanchor reports in your table are accidentally flipped (2090 is the conservative median, not the aggressive one—and vice versa for 2040).
Looking literally at Cotra’s sheet the median year occurs is 2053. Though in Cotra’s report, you’re right that she rounds this to 2050 and reports this as her official median year. So I think the only differences between your interpretation and Holden’s interpretation is just different rounding.
I do agree more precise definitions would be helpful.
I don’t think it makes sense to deviate from Cotra’s best guess and create a mean out of aggregating between the conservative and aggressive estimates. We shouldn’t assume these estimates are symmetric where the mean lies in the middle using some aggregation method, instead I think we should take Cotra’s report literally where the mean of the distribution is where she says it is (it is her distribution to define how she wants), which would be the “best guess”. In particular, her aggressive vs. conservative range does not represent any sort of formal confidence interval so we can’t interpret it that way. I have some unpublished work where I re-run a version of Cotra’s model where the variables are defined by formal confidence intervals—I think that would be the next step for this analysis.
Your median estimate for the conservative and aggressive bioanchor reports in your table are accidentally flipped (2090 is the conservative median, not the aggressive one—and vice versa for 2040).
Corrected, thanks!
I don’t think it makes sense to deviate from Cotra’s best guess and create a mean out of aggregating between the conservative and aggressive estimates.
I agree. (Note the distribution we fitted to “Bio anchors” (row 4 of the 1st table of this section) only relies on Cotra’s “best guesses” for the probability of TAI by 2036 (18 %) and 2100 (80 %).)
The “Representativeness” section is very interesting and I’d love to see more timelines analyzed concretely and included in aggregations.
Thanks for the sources! Regarding the aggregation of forecasts, I thought this article to be quite interesting.
Thanks for putting this together! I think more scrutiny on these ideas is incredibly important so I’m delighted to see you approach it.
So meta to red team a red team, but some things I want to comment on:
Your median estimate for the conservative and aggressive bioanchor reports in your table are accidentally flipped (2090 is the conservative median, not the aggressive one—and vice versa for 2040).
Looking literally at Cotra’s sheet the median year occurs is 2053. Though in Cotra’s report, you’re right that she rounds this to 2050 and reports this as her official median year. So I think the only differences between your interpretation and Holden’s interpretation is just different rounding.
I do agree more precise definitions would be helpful.
I don’t think it makes sense to deviate from Cotra’s best guess and create a mean out of aggregating between the conservative and aggressive estimates. We shouldn’t assume these estimates are symmetric where the mean lies in the middle using some aggregation method, instead I think we should take Cotra’s report literally where the mean of the distribution is where she says it is (it is her distribution to define how she wants), which would be the “best guess”. In particular, her aggressive vs. conservative range does not represent any sort of formal confidence interval so we can’t interpret it that way. I have some unpublished work where I re-run a version of Cotra’s model where the variables are defined by formal confidence intervals—I think that would be the next step for this analysis.
The “Representativeness” section is very interesting and I’d love to see more timelines analyzed concretely and included in aggregations. For more reviews and analysis that include AI timelines, you should also look to “Reviews of “Is power-seeking AI an existential risk?””. I also liked this LessWrong thread where multiple people stated their timelines.
Thanks for commenting, Peter!
Corrected, thanks!
I agree. (Note the distribution we fitted to “Bio anchors” (row 4 of the 1st table of this section) only relies on Cotra’s “best guesses” for the probability of TAI by 2036 (18 %) and 2100 (80 %).)
Thanks for the sources! Regarding the aggregation of forecasts, I thought this article to be quite interesting.