A few thoughts on this as someone who is even more of a generalist (I work in comms/public affairs at the moment and have no technical skills), and who would like to work on safety if the opportunity came up:
I think people should stop saying that AI safety is “bottlenecked on generalists” or just needs more generalists. If there was truly huge demand for “generalists”, presumably you wouldn’t have to do months of unpaid work to try and get a job! This sounds more like famously competitive fields like publishing. At least, people should say that the field is bottlenecked on highly skilled generalists.
It’s not totally clear to me why generalists should be expected to signal their alignment to this extent. I understand why you’d want this for applicants to technical fellowships. But the risks of a web designer/HR manager/chief of staff going to work on capabilities seems low. In most fields you are expected to give some sort of reason for why you’re interested in the role, but the level of signalling here is extreme and rules out 99% of applicants for generalist roles.
On your second point, I hadn’t thought that the need for alignment was related to the risk of someone going to work on capabilities. I think it’s more about understanding the landscape, the terminology, knowing how to communicate to people in the space, etc., i.e. things that might affect your ability to do good work. Maybe alignment wasn’t the right term for this.
And I think it’s not necessarily about people’s ability to do work, but people’s perceived commitment to ‘the cause’ and their perceived ‘moral fit’ with the organisation.
I think there’s a higher importance placed on this by some organisations, and for some roles. But it seems that some degree of ‘alignment’ is required to be recruited to almost all roles in the EA eco-system.
I’m not taking a stance on whether this is good or bad—people can rightly have opinions both ways on this I think. Just making an observation that it’s a factor.
I’m also going to refer to this post that is based on ‘context’ being a missing criteria for external candidates, but I think this concept includes alignment.
I think context and alignment are quite different. I would say context is basically role-specific knowledge. For some roles, like grantmaker, it’s very useful to have an in-depth understanding of the technical issues of AI safety, the history of the field, the people and organisations involved, etc. For other roles, like comms, you might need less or different knowledge. For example, if you do comms for an alignment research org, you probably need the technical knowledge to roughly understand what your org is working on so that you can explain it to other people. But you don’t need sufficient technical knowledge to be able to predict if it’s a viable approach, as an ideal grantmaker might.
Whereas I would say alignment is not just understanding but actually believing. A long history in EA/LW/various AI safety spaces predicts that you’ll be committed to AI safety and won’t just jump to a better paying job if it comes along.
In my current role (which has nothing whatsoever to do with AI) I think I have pretty good context—I understand the product, I know why we’re building it, I know all the key internal and external stakeholders, etc. But I actually think the product is not that great, and if someone came along and offered me a job that was better, I would probably leave.
I suppose I was just thinking about things like MATS, where I’ve seen people become very worried that it’s basically a programme to take really smart people and give them the skills they need to go and work for xAI once their fellowship is done, so it’s important to make sure they care a lot about AI safety.
Absolutely agree that everything you’ve said is really important to succeed. I’d be curious to know how much it is considered a prerequisite for AI safety roles vs generalist roles elsewhere in tech, as opposed to learning these skills on the job.
At least, people should say that the field is bottlenecked on highly skilled generalists.
Thoughts on how to do this as a hiring manager? Things I’ve considered:
Title the role “Senior [whatever].” I think this is ok, but in many fields “senior” is a synonym for “old”, so this title causes talented young people to not apply (and untalented old people to apply).
Say “if you can do X, then you should apply”. This is ideal, but it’s often hard to give an objective enough test that it doesn’t end up just effectively being a test of the candidate’s self-confidence.
Say “if you have done X, then you should apply”. Easier to evaluate objectively than (2), but artificially limits the candidate pool to only people who have done something very similar before.
Literally say “highly skilled generalist.” Seems kind of pretentious, and it also seems like it’s effectively a self-confidence test for the candidate.
Ask for referrals from people who I know well enough that I can effectively say “highly skilled generalist” and they will apply that criterion in a way that I would endorse. This is good but means I don’t hire from outside my circle.
Title the role “Senior [whatever].” I think this is ok, but in many fields “senior” is a synonym for “old”, so this title causes talented young people to not apply (and untalented old people to apply).
Hi Ben_West!
Are you saying that people would read “senior” in a job description as meaning “older” rather than “more experienced”? That strikes me as an interpretation more likely to come from someone who hasn’t yet had much exposure to the professional workplace.
Why would using the word “senior” cause an untalented old person to apply?
Unfortunately, the use of “old people” in this sentence seems to hint at ageism. What age do you consider old? Should old people not be allowed to apply to jobs? Can you elaborate on your reluctance to hire an “old” person? I went to an online session that BlueDot hosted not too long ago about generalists getting into AI safety and many people asking questions were convinced they were experiencing ageism. I’m not saying they were correct but this sort of comment seems like they weren’t mistaken.
Are you saying that people would read “senior” in a job description as meaning “older” rather than “more experienced”?
No, I’m saying that they would interpret it to mean “having more years of formal experience (rather than e.g. having had a wider variety of experiences, or having had more useful experiences)” and I instead want a word which means “more skilled”.
Can you elaborate on your reluctance to hire an “old” person?
No reluctance! I check the “20+ years of experience” box on eag applications myself. I just am bemoaning the fact that the word “senior” indicates both age and skill, and I want a word which only applies to the latter.
Title the role “Senior [whatever].” I think this is ok, but in many fields “senior” is a synonym for “old”, so this title causes talented young people to not apply (and untalented old people to apply).
I think ‘specialist’ works as a title in this situation, without invoking issues of age—eg Finance specialist, Comms specialist etc.
For me TOR are important, and I wonder sometimes if enough time is spent by hiring organisations (including commercial organisations) in defining what they want out of roles. I see Chief of Staff roles in some organisations that are quite strategic, and in other organisations that are essentially an admin assistant. I appreciate defining responsibilities and tasks is especially tricky in startups, where the roles are harder to define, and you might want a true generalist who can do a bit of everything.
If an organisation advertises for a HR manager, and talks about how they are scaling up and rapidly recruiting, I might apply—I’ve done end to end recruitment roles, performance management, HR policy etc. If the organisation advertises for a ‘HR specialist’, and talks about recruitment, setting comp, HR compliance across multiple jurisdictions, I’ve got a better idea what they’re looking for and I’m going to screen myself out by not applying because I dont have all that expertise.
I dont think I’m an ‘untalented old person’ in the first scenario—the organisational asked for something that I’ve done before, so I apply based on the information that was available. But if the organisation asks for A but really intends B, then everyones’ time has been wasted.
5. Ask for referrals from people who I know well enough that I can effectively say “highly skilled generalist” and they will apply that criterion in a way that I would endorse. This is good but means I don’t hire from outside my circle.
If everyone hires from within their existing circles then the community doesnt grow—it’s just musical chairs with the same people swapping between same organisations. If the thesis is that the community needs to grow to be able to (1) effectively respond to urgent AI safety challenges and (2) effectively absorb and use anticipated increased levels of funding, then to my mind looking at recruitment practices is an important part of the solution.
A few thoughts on this as someone who is even more of a generalist (I work in comms/public affairs at the moment and have no technical skills), and who would like to work on safety if the opportunity came up:
I think people should stop saying that AI safety is “bottlenecked on generalists” or just needs more generalists. If there was truly huge demand for “generalists”, presumably you wouldn’t have to do months of unpaid work to try and get a job! This sounds more like famously competitive fields like publishing. At least, people should say that the field is bottlenecked on highly skilled generalists.
It’s not totally clear to me why generalists should be expected to signal their alignment to this extent. I understand why you’d want this for applicants to technical fellowships. But the risks of a web designer/HR manager/chief of staff going to work on capabilities seems low. In most fields you are expected to give some sort of reason for why you’re interested in the role, but the level of signalling here is extreme and rules out 99% of applicants for generalist roles.
Interesting thoughts.
On your second point, I hadn’t thought that the need for alignment was related to the risk of someone going to work on capabilities. I think it’s more about understanding the landscape, the terminology, knowing how to communicate to people in the space, etc., i.e. things that might affect your ability to do good work. Maybe alignment wasn’t the right term for this.
I think alignment -is- the right word.
And I think it’s not necessarily about people’s ability to do work, but people’s perceived commitment to ‘the cause’ and their perceived ‘moral fit’ with the organisation.
I think there’s a higher importance placed on this by some organisations, and for some roles. But it seems that some degree of ‘alignment’ is required to be recruited to almost all roles in the EA eco-system.
I’m not taking a stance on whether this is good or bad—people can rightly have opinions both ways on this I think. Just making an observation that it’s a factor.
I’m also going to refer to this post that is based on ‘context’ being a missing criteria for external candidates, but I think this concept includes alignment.
https://forum.effectivealtruism.org/posts/b82SLXwEHRCs3TFJA/why-experienced-professionals-fail-to-land-high-impact-roles
I think context and alignment are quite different. I would say context is basically role-specific knowledge. For some roles, like grantmaker, it’s very useful to have an in-depth understanding of the technical issues of AI safety, the history of the field, the people and organisations involved, etc. For other roles, like comms, you might need less or different knowledge. For example, if you do comms for an alignment research org, you probably need the technical knowledge to roughly understand what your org is working on so that you can explain it to other people. But you don’t need sufficient technical knowledge to be able to predict if it’s a viable approach, as an ideal grantmaker might.
Whereas I would say alignment is not just understanding but actually believing. A long history in EA/LW/various AI safety spaces predicts that you’ll be committed to AI safety and won’t just jump to a better paying job if it comes along.
In my current role (which has nothing whatsoever to do with AI) I think I have pretty good context—I understand the product, I know why we’re building it, I know all the key internal and external stakeholders, etc. But I actually think the product is not that great, and if someone came along and offered me a job that was better, I would probably leave.
I suppose I was just thinking about things like MATS, where I’ve seen people become very worried that it’s basically a programme to take really smart people and give them the skills they need to go and work for xAI once their fellowship is done, so it’s important to make sure they care a lot about AI safety.
Absolutely agree that everything you’ve said is really important to succeed. I’d be curious to know how much it is considered a prerequisite for AI safety roles vs generalist roles elsewhere in tech, as opposed to learning these skills on the job.
Thoughts on how to do this as a hiring manager? Things I’ve considered:
Title the role “Senior [whatever].” I think this is ok, but in many fields “senior” is a synonym for “old”, so this title causes talented young people to not apply (and untalented old people to apply).
Say “if you can do X, then you should apply”. This is ideal, but it’s often hard to give an objective enough test that it doesn’t end up just effectively being a test of the candidate’s self-confidence.
Say “if you have done X, then you should apply”. Easier to evaluate objectively than (2), but artificially limits the candidate pool to only people who have done something very similar before.
Literally say “highly skilled generalist.” Seems kind of pretentious, and it also seems like it’s effectively a self-confidence test for the candidate.
Ask for referrals from people who I know well enough that I can effectively say “highly skilled generalist” and they will apply that criterion in a way that I would endorse. This is good but means I don’t hire from outside my circle.
Hi Ben_West!
Are you saying that people would read “senior” in a job description as meaning “older” rather than “more experienced”? That strikes me as an interpretation more likely to come from someone who hasn’t yet had much exposure to the professional workplace.
Why would using the word “senior” cause an untalented old person to apply?
Unfortunately, the use of “old people” in this sentence seems to hint at ageism. What age do you consider old? Should old people not be allowed to apply to jobs? Can you elaborate on your reluctance to hire an “old” person? I went to an online session that BlueDot hosted not too long ago about generalists getting into AI safety and many people asking questions were convinced they were experiencing ageism. I’m not saying they were correct but this sort of comment seems like they weren’t mistaken.
Thanks for the questions!
No, I’m saying that they would interpret it to mean “having more years of formal experience (rather than e.g. having had a wider variety of experiences, or having had more useful experiences)” and I instead want a word which means “more skilled”.
No reluctance! I check the “20+ years of experience” box on eag applications myself. I just am bemoaning the fact that the word “senior” indicates both age and skill, and I want a word which only applies to the latter.
Title the role “Senior [whatever].” I think this is ok, but in many fields “senior” is a synonym for “old”, so this title causes talented young people to not apply (and untalented old people to apply).
I think ‘specialist’ works as a title in this situation, without invoking issues of age—eg Finance specialist, Comms specialist etc.
For me TOR are important, and I wonder sometimes if enough time is spent by hiring organisations (including commercial organisations) in defining what they want out of roles. I see Chief of Staff roles in some organisations that are quite strategic, and in other organisations that are essentially an admin assistant. I appreciate defining responsibilities and tasks is especially tricky in startups, where the roles are harder to define, and you might want a true generalist who can do a bit of everything.
If an organisation advertises for a HR manager, and talks about how they are scaling up and rapidly recruiting, I might apply—I’ve done end to end recruitment roles, performance management, HR policy etc. If the organisation advertises for a ‘HR specialist’, and talks about recruitment, setting comp, HR compliance across multiple jurisdictions, I’ve got a better idea what they’re looking for and I’m going to screen myself out by not applying because I dont have all that expertise.
I dont think I’m an ‘untalented old person’ in the first scenario—the organisational asked for something that I’ve done before, so I apply based on the information that was available. But if the organisation asks for A but really intends B, then everyones’ time has been wasted.
5. Ask for referrals from people who I know well enough that I can effectively say “highly skilled generalist” and they will apply that criterion in a way that I would endorse. This is good but means I don’t hire from outside my circle.
If everyone hires from within their existing circles then the community doesnt grow—it’s just musical chairs with the same people swapping between same organisations. If the thesis is that the community needs to grow to be able to (1) effectively respond to urgent AI safety challenges and (2) effectively absorb and use anticipated increased levels of funding, then to my mind looking at recruitment practices is an important part of the solution.