Non-EA interests include chess and TikTok (@benthamite). Formerly @ CEA, METR + a couple now-acquired startups.
Ben_Westđ¸
Karma: 16,192
At least, people should say that the field is bottlenecked on highly skilled generalists.
Thoughts on how to do this as a hiring manager? Things Iâve considered:
Title the role âSenior [whatever].â I think this is ok, but in many fields âseniorâ is a synonym for âoldâ, so this title causes talented young people to not apply (and untalented old people to apply).
Say âif you can do X, then you should applyâ. This is ideal, but itâs often hard to give an objective enough test that it doesnât end up just effectively being a test of the candidateâs self-confidence.
Say âif you have done X, then you should applyâ. Easier to evaluate objectively than (2), but artificially limits the candidate pool to only people who have done something very similar before.
Literally say âhighly skilled generalist.â Seems kind of pretentious, and it also seems like itâs effectively a self-confidence test for the candidate.
Ask for referrals from people who I know well enough that I can effectively say âhighly skilled generalistâ and they will apply that criterion in a way that I would endorse. This is good but means I donât hire from outside my circle.
Iâm curious what you think economic orthodoxy is if Hansonâs claim â[AI will cause the world economy to] double every few months or fasterâ is more of a disagreement with EA orthodoxy than economic orthodoxy?
(Iâm also interested in what you think the EA orthodoxy is; âmonths-long doubling timesâ feels like a pretty mainstream view even amongst people working full time in AI safety, though I agree that it differs from, e.g. Eliezer Yudkowsky c. 2008.)
Thanks for writing this! Iâm on the other side (wanting to hire generalists) and I share your skepticism about the âwrite publiclyâ advice. I think the actual advice should be more like âclearly demonstrate success in the thing you are trying to get hired for,â which usually doesnât involve writing.
(Although I do think that generalist roles can be particularly hard to demonstrate success in. For example, I expect that writing about recruiting might actually be one of the better ways to clearly demonstrate your skill in recruiting, even though recruiting per se doesnât usually involve much writing.)
Thanks for writing this up! I hadnât realized that the feb protest was so big.
This is super cool, thanks for doing it.
What does the average scholar think about this situation? âŚThey think that the work falls beneath standards because it has not been through the system of review which nearly all scholars think is required for work to reliably meet minimal scholarly standards.
This is not my experience. The arxiv version of my paper has been cited 97 times; the peer-reviewed version 7 times. The only time I can remember someone saying the paper shouldnât be trusted because of not having been peer-reviewed yet was, ironically, on the EA Forum.
You can compare the peer review comments with those on LessWrong. Neither is a pareto improvement over the other, but what in hindsight has proven to be the strongest critique (that growth is actually superexponential, not exponential, because of e.g. post-training) was only mentioned on LW (and twitter).
Peer review has many things to recommend it but, as might be guessed from a post whose âCriticism of peer reviewâ section consists solely of claims that all criticism of peer review is invalid, this post is overstating the case.
This is a cool idea!
Whatâs your explanation for why they attack EAs rather than, say, the AI ethics crowd?
I think the AI ethics crowd is the subject of attacks (though arguably this is because they tried to seek power and influence).
I also consider this to a lesser extent around animal sentience arguments
+1, âit would be very easy for me to ignore the possibility that nematodes might be consciousâ is a major impediment to thinking clearly about animal sentience (including for me).
I donât disagree, itâs more that this feels a bit like privileging the hypothesis? I think the modal reason Iâve heard from people who did capabilities work and now regret it is something like âI knew I was misaligned with leadership but I thought leaving would be even worse.â
If, for some reason, Anthropic asked me how to prevent people from regretting working for them, I would focus much more on âhave a thing for people to do once they realize their colleague is corruptâ instead of âhave a more nuanced way of telling if their colleague is corrupt.â
Downvoted; I think this comment was unnecessarily rude.
Thanks! I only know a handful of people in this category, but for what itâs worth, it again feels like people who were predisposed to thinking that working on pretraining would be okay rather than them being âcorrupted.â
E.g., I recently talked to someone who told me that their main takeaway from a safety fellowship was realizing that they didnât fit in because they actually werenât worried about existential risk in the same way that the other attendees were.
People seem surprised and bewildered when AI folks defect away from AI safety towards capabilities. People trust that as AI companies grow, those gaining power and money from shares will not be adversely influenced by that power and money.
fwiw I donât actually know many examples of this, and the ones I hear cited often seem uncompelling to me. E.g.:
Greg Brockman doesnât seem like a true believer in OpenAIâs nonprofit mission who got corrupted but rather someone who went into it wanting to make a profit
Mechanizeâs founders donât seem like EAs who got corrupted by AI money but rather EAs with unusual moral and empirical views which result in them thinking that the best course of action is the exact opposite of what most EAs think
(Counterexamples appreciated, though!)
And credit to the AI skeptics that they seem to mostly have updated in light of the new evidence (or at least claimed that they never actually believed in long timelines, which is maybe less noble, but ends up in the same place).
Yeah I agree that if you only have one bit of detail that you can store, then saying it is âhardâ rather than âeasyâ is probably the correct bit. However I would suggest that for something as important as your career you should investigate in substantially more detail. If you do so I expect you will come up with a range of needed skills/âattributes for these jobs, some of which you might find easy, others of which you might find hard.
I no longer work at METR. I would guess that theyâd be excited about applicants who have done this, but donât want to speak for them.
Many people said they wanted to work for METR. I made what I thought was a good offer: take one of the benchmarks we give AIs; if you get a good score then I guarantee that I will fly you out for an interview, even if you have no work history, have no money to pay for the trip, or any other barrier one might have to employment.
Exactly zero people took me up on this.[1]
How is it possible for there to be sky-high rejection rates yet also zero people sending me applications?
I think the answer is that raw rejection rates arenât a very useful metric. After all, an 80% rejection rate means that the AI safety jobs are 1/â10th as selective as Walmart!
I would suggest ignoring raw rejection rates in favor of just looking at the criteria for the jobs you want. Particularly for something like s-risks the criteria are going to be unusual and specific, meaning that even generically qualified people will often have to dedicate substantial time to skilling up, but if youâre able to do so, then your odds are pretty good.[2]
- ^
I wouldnât be surprised to learn that some people tried this, failed, and then were too embarrassed about failing to tell me. But, to the best of my recollection, literally zero people have told me that they even attempted this task.
- ^
I say this even with the knowledge that you are 19. I donât want to pretend that the deck isnât stacked against younger peopleâit totally isâbut we employ some 19 year olds, as do other AI safety orgs. If a 19 year old had sent me a good solution to that METR challenge, for example, I would have been happy to hire them.
- ^
Cool! Impressive numbers.
Table 1 shows the techniques used; the teams which were allowed to use SAEs (an interpretability technique) used them; the one which was prohibited from using them searched the data.
Also note that âtraining dataâ does not mean âinstructionsâ. Section 3 describes their training process.
Thanks for the questions!
No, Iâm saying that they would interpret it to mean âhaving more years of formal experience (rather than e.g. having had a wider variety of experiences, or having had more useful experiences)â and I instead want a word which means âmore skilledâ.
No reluctance! I check the â20+ years of experienceâ box on eag applications myself. I just am bemoaning the fact that the word âseniorâ indicates both age and skill, and I want a word which only applies to the latter.