We as a movement do a terrible job of communicating just how hard it might be to get a job in AI Safety and honestly cause people to anchor/rely too much on EA resources which sets unrealistic expectations and also not being fair
So, some high-level suggestions based on my interactions with other people I have are:
Being more explicit about this in 80K hours calls or talking about the funding bar (potentially somehow with grantmakers/ intro’ing to successful candidates who do independent stuff). Maybe organisations could explicitly state this in their fellowship/intern/job applications: “Only 10 out of 300 last year got selected” so that people don’t over-rely on some applications.
There is a very obvious point that Community Builders can only do so much because their general job is to point resources out and set initial things rolling. I think that as community builders, being vocal about this from an early point is important. This could look like, “Hey, I only know as much as you do now that you have read AGI SF and Superintelligence.” Community builders could also try connecting with slightly more senior people and doing intros on a selective basis(e.g., I know a few good community builders who try to go out of their way to an EAGx to score convos with such people).
I think metrics for 80K, and CBs need to be more heavily weighted towards(if not already) “X went on to do an internship and publish a paper” and away from “this guy read superintelligence and did a fun hackathon”. The latter also creates weird sub-incentives for community members to score brownie points with CBs and make a lot of movement with little impactful progress.
Talking about creating your own opportunities seems really untalked about in EA circles- there is a lot of talk about finding opportunities and overwhelming newcomers with EA org webpages, which, coupled with neglectedness, causes them to overestimate the opportunities. Maybe there could be a guide for this, some sort of a group/support for this?
For early career folks, maybe there could be some sort of a peer buddy system where people who are a little bit further down the road can get matched and collaborate/talk. A lot of these conversations involve safe spaces, building trust and talking about really sensitive issues(like finances, runway planning and critical feedback on applications). I have been lucky to build such a circle within EA, but I recognize that’s only because of certain opportunities I got early on, along with being comfortable with reaching out to people, something which not necessarily everyone is.
We need to identify more proactive people who already have a track record of social impact/being driven by certain kinds of research instead of just high-potential people- these are probably the only people who will actually convert to returns for the movement(very crudely speaking). This is even more true in non-EA hubs where good connections aren’t just one local meetup away as with NYC or Oxford. I think there is a higher attrition rate of high-potential people in LMICs, at least partly due to this.
I agree, and I think part of the problem is giving a false impression of what kind of training or experience is most useful. You mention this in the over-reliance on EA resources which I think is a major problem. This is especially an issue when someone applies for AI Safety related jobs outside of the EA sphere—because they’re competing with people with a much wider range of experience.
I’ve always felt there should be a ‘useful non-EA courses/resources’ guide.
I think I’m in some ways confused about this. I think it’s true that the hiring situation is hard, but my priors say that this is likely to change fast[1] and that the downside risk for many people is probably low: especially in the technical side, time upskilling for AI Safety is probably not time completely wasted for the industry at large[2].
Are there any particular things you think we could do better? I think one could be just in general being less quick to suggest AIS as a career path for people who might be in risk for financial hardship as a result. Career guides in general do seem very oriented to people who can often take the risk of spending months unemployed, and doing upskilling or job hunting.[3]
Especially ML-wise. But this is probably less true (if at all) for people upskilling in AI Safety policy and governance, strategy and fieldbuilding, etc.
I predict scaling up organizations will be slow and painful rather than easy and fast. I predict organizations will have a hard time productively scaling fast, though they may be either productive or scale fast.
This seems mostly independent of funding.
(I don’t particularly disagree with the rest of the comment.)
I think this would be more the result of new orgs rather than bigger orgs? Like I would argue that we currently don’t have anything near the optimal amount of orgs dedicated to training programs, and as funding increases, we will probably get a lot of them.
We as a movement do a terrible job of communicating just how hard it might be to get a job in AI Safety and honestly cause people to anchor/rely too much on EA resources which sets unrealistic expectations and also not being fair
Any suggestions?
So, some high-level suggestions based on my interactions with other people I have are:
Being more explicit about this in 80K hours calls or talking about the funding bar (potentially somehow with grantmakers/ intro’ing to successful candidates who do independent stuff). Maybe organisations could explicitly state this in their fellowship/intern/job applications: “Only 10 out of 300 last year got selected” so that people don’t over-rely on some applications.
There is a very obvious point that Community Builders can only do so much because their general job is to point resources out and set initial things rolling. I think that as community builders, being vocal about this from an early point is important. This could look like, “Hey, I only know as much as you do now that you have read AGI SF and Superintelligence.” Community builders could also try connecting with slightly more senior people and doing intros on a selective basis(e.g., I know a few good community builders who try to go out of their way to an EAGx to score convos with such people).
I think metrics for 80K, and CBs need to be more heavily weighted towards(if not already) “X went on to do an internship and publish a paper” and away from “this guy read superintelligence and did a fun hackathon”. The latter also creates weird sub-incentives for community members to score brownie points with CBs and make a lot of movement with little impactful progress.
Talking about creating your own opportunities seems really untalked about in EA circles- there is a lot of talk about finding opportunities and overwhelming newcomers with EA org webpages, which, coupled with neglectedness, causes them to overestimate the opportunities. Maybe there could be a guide for this, some sort of a group/support for this?
For early career folks, maybe there could be some sort of a peer buddy system where people who are a little bit further down the road can get matched and collaborate/talk. A lot of these conversations involve safe spaces, building trust and talking about really sensitive issues(like finances, runway planning and critical feedback on applications). I have been lucky to build such a circle within EA, but I recognize that’s only because of certain opportunities I got early on, along with being comfortable with reaching out to people, something which not necessarily everyone is.
We need to identify more proactive people who already have a track record of social impact/being driven by certain kinds of research instead of just high-potential people- these are probably the only people who will actually convert to returns for the movement(very crudely speaking). This is even more true in non-EA hubs where good connections aren’t just one local meetup away as with NYC or Oxford. I think there is a higher attrition rate of high-potential people in LMICs, at least partly due to this.
I agree, and I think part of the problem is giving a false impression of what kind of training or experience is most useful. You mention this in the over-reliance on EA resources which I think is a major problem. This is especially an issue when someone applies for AI Safety related jobs outside of the EA sphere—because they’re competing with people with a much wider range of experience.
I’ve always felt there should be a ‘useful non-EA courses/resources’ guide.
I think I’m in some ways confused about this. I think it’s true that the hiring situation is hard, but my priors say that this is likely to change fast[1] and that the downside risk for many people is probably low: especially in the technical side, time upskilling for AI Safety is probably not time completely wasted for the industry at large[2].
Are there any particular things you think we could do better? I think one could be just in general being less quick to suggest AIS as a career path for people who might be in risk for financial hardship as a result. Career guides in general do seem very oriented to people who can often take the risk of spending months unemployed, and doing upskilling or job hunting.[3]
Both as a result of higher funding and people funding a lot of orgs whenever there is both excess talent and funding overhang.
Especially ML-wise. But this is probably less true (if at all) for people upskilling in AI Safety policy and governance, strategy and fieldbuilding, etc.
Something something rich western countries
I predict scaling up organizations will be slow and painful rather than easy and fast. I predict organizations will have a hard time productively scaling fast, though they may be either productive or scale fast.
This seems mostly independent of funding.
(I don’t particularly disagree with the rest of the comment.)
I think this would be more the result of new orgs rather than bigger orgs? Like I would argue that we currently don’t have anything near the optimal amount of orgs dedicated to training programs, and as funding increases, we will probably get a lot of them.
I’d also predict something similar about well functioning founding orgs.
Doing things is hard.