Yes, we were particularly concerned with the fact that earlier camps were in-person and likely had a stronger selection bias for people interested in AIS (due to AI/AIS being more niche at the time) as well as a geographic selection bias. That’s why I have more trust in the participant tracking data for camps 4-6 which were more recent, virtual and had a more consistent format.
Since AISC 8 is so big, it will be interesting to re-do this analysis with a single group under the same format and degree of selection.
Thank you for the pointer! I hadn’t seen this before and it looks like there’s a lot of interesting thinking on how to study AI safety field building. I appreciate having more cost-effectiveness estimates to compare to.
I haven’t given it a full read, but it seems like the quality-adjusted researcher year is very similar to the metric I’m proposing here.
To do a comparison between our estimates, lets assume a new AIS researcher does 10 years of quality adjusted, time-discounted AIS research (note that timelines become pretty important here) then we get:
(10 QARY’s/researcher) / ($30K/researcher) = 3.33E-4 QURY’s per dollar = 333 QURY’s per $1M
That seems similar to the CAIS estimates for MLSS, so it seems like these approaches have pretty comparable results!
In the future I’m also interested in modelling how to distribute funding optimally in talent pipelines like these.