I’d prefer to see you pick ‘people who have made AI predictions who are not famous for those predictions’ in some random-ish way. I could just say ‘Gary Marcus’ and be done, but I’d only be saying that because he disagrees with me on AI progress and I think he’d look bad if his track record was examined.
You’re probably not trying to be super scientific, but I definitely wouldn’t cite anything to a policymaker that was cherry-picked. I also wouldn’t cite your tool if you only found an effect because you mainly compared people who are famous for scaling predictions, like Gary Marcus and Gwern.
Thanks for all the work you guys do! And love the new website.
1. Reports on a specific thinker (e.g. Gwern) or body of work’s predictions. These would probably be published individually or showing interesting comparisons, similar to the Futurists track record in Cold Takes (based on Arb’s Big Three research) 2. A dashboard ranking the track records of lots of thinkers
For (2), I agree that cherry picking would be bad, and we’d want it to cover a good range.
For our initial outputs from (1) though, I’m excited about specifically picking thinkers who people would find it especially useful to understand their track record (or to have a good-quality assessment of it that they can cite). Curious if you have thoughts of specific people who fit the bill for you?
Awesome, thanks Adam, this makes a lot of sense. I’d be excited to see reports on specific thinkers like Gwern and Yuval Noah Harrari. I’d be especially excited to look at the track records of institutions, like frontier developers or governments (e.g. the UK Government or its AISI).
I’d prefer to see you pick ‘people who have made AI predictions who are not famous for those predictions’ in some random-ish way. I could just say ‘Gary Marcus’ and be done, but I’d only be saying that because he disagrees with me on AI progress and I think he’d look bad if his track record was examined.
You’re probably not trying to be super scientific, but I definitely wouldn’t cite anything to a policymaker that was cherry-picked. I also wouldn’t cite your tool if you only found an effect because you mainly compared people who are famous for scaling predictions, like Gary Marcus and Gwern.
Thanks for all the work you guys do! And love the new website.
Great thanks!
We have two outputs in mind with this project:
1. Reports on a specific thinker (e.g. Gwern) or body of work’s predictions. These would probably be published individually or showing interesting comparisons, similar to the Futurists track record in Cold Takes (based on Arb’s Big Three research)
2. A dashboard ranking the track records of lots of thinkers
For (2), I agree that cherry picking would be bad, and we’d want it to cover a good range.
For our initial outputs from (1) though, I’m excited about specifically picking thinkers who people would find it especially useful to understand their track record (or to have a good-quality assessment of it that they can cite). Curious if you have thoughts of specific people who fit the bill for you?
Awesome, thanks Adam, this makes a lot of sense. I’d be excited to see reports on specific thinkers like Gwern and Yuval Noah Harrari. I’d be especially excited to look at the track records of institutions, like frontier developers or governments (e.g. the UK Government or its AISI).