I was very surprised by the paragraph: ‘However, I also have an intuitive preference (which is related to the “burden of proof” analyses given previously) to err on the conservative side when making estimates like this. Overall, my best guesses about transformative AI timelines are similar to those of Bio Anchors.’ especially in context and especially because of the use of the term ‘conservative’. I would have thought that the conservative assumption to make would be shorter timelines (since less time to prepare). If I remember correctly, Toby Ord discusses something similar in the chapter on AI risk from ‘The Precipice’: how at one of the AI safety conferences (FLI Puerto Rico 2015?) some AI researchers used the term ‘conservative’ to mean ‘we shouldn’t make wild predictions about AI’ and others to mean ‘we should be really risk-averse, so we should assume that it could happen soon’. I would have expected to see the second use here.
Bogdan Ionut Cirstea
Karma: 106
I think aligning narrow superhuman models could be one very valuable megaproject and this seems scalable to >= $100 million, especially if also training large models (not just fine-tuning them for safety). Training their own large models for alignment research seems to be what Anthropic plans to do. This is also touched upon in Chris Olah’s recent 80k interview.
‘CSET-Foretell forecasts were quoted by Quanta Magazine (a) on on whether VC funding for tech startups will dry up’ - the linked article seems to come from Quartz, not Quanta Magazine