Some counterarguments to the “AI safety field is maturing” claim—and that it is still very much in its infancy—which might also (at least to some extent) explain the talent issue:
Self-centrism – strong preference for within-the-field hires [Obs. 1, 4 and Meta-Obs] exhibits the typical tribalism culture of early-stage communities and shows lack of interest in outside views.
Lack of awareness of own environment – an extension of point 1, really, but separate to underscore that no standard operating practices exist within the field [Obs. 6], and that no visible efforts are made to leverage experience of R&D teams in established industries. (This post and the survey results that it summarizes being the case in point. An inward-only perspective as opposed to interviewing actually mature fields on how, or if, they solved their hiring problem.)
Inexperience – particularly visible in approach to “adjacent” roles and manifests itself through the community simply not being able to define what their organizations’ needs are, which further translates to misspecified job requirements and candidate screenings [Obs. 5].
While I appreciate your attempt to systematize the discourse around hiring in AIS, before collecting effort-intense panel data, it sometimes helps to turn the tables and ask yourself a fundamental question if the field is at all attractive to talent (esp. “top talent”)? I’d say there’s reasons abound to be sceptical. Starting with framing all things non-technical as generalist (while in fact each role is a specialization in its own right), through demanding a full mission-alignment, all the way to roll-the-dice type bet on a career in en emerging field that is funded by discretionary donations. Not to mention the reputational risk, by association, from some of the more volatile characters in the community.
In terms of next steps, a look at aggregate spend on roles within AIS should be taken. Might reveal much more than any self-reported “what pains me today” survey.
My understanding of the original post’s intent is that it calls for evidence of the field’s impact, given the funding it receives. I don’t believe it critiques judgmental forecasting as an analytical method and neither do I think that I signal this in my comment.
I stand by my opinion, however, that the community is correct to ask for tactile proof, burden of which rests on organizations that receive the funding.
I regret if this doesn’t satisfy the questions in your comment.