Since this looks like it was written partially in response to me, I’d like to reply. First, I appreciate the clarification. It is very helpful and I definitely agree with most of it.
It strikes me that the actual problem here is one of messaging. By getting EA orgs to list very large figures for their hires and talk about talent gaps writ large, you risk misleading EAs into thinking that they should be focusing on applying or upskilling for these jobs in particular, when the actual value of doing so may be less than it appears (though still potentially large). It seems like it would be far more informative to ask EAs to place figures on future hires or discuss more detail about how exactly they feel constrained or bottlenecked.
Also, if EA orgs are better informed that there actually are a lot of talented applicants out there, perhaps these orgs may invest more in figuring out how to productively bring on more people.
I appreciate this follow up as well. Peter and I seem to be of similar thinking here and in his comment on the other post, but just to add:
I don’t think the misunderstanding stems as much from the recent-hire valuations, as from terms like “talent bottleneck” and “talent constrained”. Especially “talent constrained” used alongside “funding constrained”. I could be mistaken, but it would seem odd to say you’re “funding constrained” but can’t use more funding at the moment. Whereas we are saying orgs are “talent constrained” but can’t make use of available talent. They evidently don’t function quite the same, so phrasing them in this matched sort of way invites erroneous comparison.
Similarly, I feel a “talent bottleneck” implies an insufficient supply of talent/applicants, which doesn’t seem to be the case. I guess it’s more that there’s insufficient talent actually working on the problems, but it’s not a matter of supply, so it’s more of a “hiring bottleneck” or an “organizational capacity bottleneck”.
EA orgs aren’t so much constrained by a lack of available talent as they are constrained by their capacity to deploy additional talent
It seems like it would be far more informative to ask EAs to place figures on future hires
I had the same thought that it might be more informative to know EA leaders answers to something like, “I’d rather have $X in additional donations than my next ideal hire.” Agree that it’s difficult to know how a future hire will work out, but maybe there’s still something to be learned from the value they’d place on an additional ideal hire, as it wouldn’t necessarily be the same as holding on to a recent successful hire. I’m admittedly out of my depth here.
We have a forthcoming post on whether the expressions ‘talent gap/bottleneck/constraint’ are generating more heat than light and should be phased out in favour of more specific terms.
Hey Peter—it’s partly for you, but also many other people who have the same questions.
I can comment on the choice of question in the survey as I’m the one who wrote it.
The reason we went for an ex post assessment rather than an ex ante one was that we thought people would be able to more reliably assess how they feel about a previous hire today, than remember how they felt about a previous hire in the past.
Asking people to remember what they thought before runs the risk that they will substitute a hard question (what they thought months ago before they knew how someone would work out) for an easy question (what do they think given what they know now). Then we’d get a similar answer, but it would look ex ante when it actually isn’t.
It also seemed quite difficult for organisations to forecast how much they’ll value a typical hire in the future because, among other reasons, it’s difficult to anticipate how successful future searches will be.
In retrospect I think the framing we chose was probably a mistake, because the two assessments are much more different than most readers understand them to be. I agree with your suggestions for improvements and, indeed, we concluded our blog post with our plans to ask new questions next year or else interview a smaller number of people in more depth.
Hopefully a different approach next year will help us avoid this confusion going forward.
As for the article being misleading, we’ve:
i) Commented that these roles are hard to fill at the point when these figures are first mentioned.
ii) Explained the ex post, ex ante distinction in the relevant section, and now added a link to this post.
iii) Noted we don’t have much confidence in the answers to that question and would not recommend that people update very much based on it.
Since this looks like it was written partially in response to me, I’d like to reply. First, I appreciate the clarification. It is very helpful and I definitely agree with most of it.
It strikes me that the actual problem here is one of messaging. By getting EA orgs to list very large figures for their hires and talk about talent gaps writ large, you risk misleading EAs into thinking that they should be focusing on applying or upskilling for these jobs in particular, when the actual value of doing so may be less than it appears (though still potentially large). It seems like it would be far more informative to ask EAs to place figures on future hires or discuss more detail about how exactly they feel constrained or bottlenecked.
Also, if EA orgs are better informed that there actually are a lot of talented applicants out there, perhaps these orgs may invest more in figuring out how to productively bring on more people.
I appreciate this follow up as well. Peter and I seem to be of similar thinking here and in his comment on the other post, but just to add:
I don’t think the misunderstanding stems as much from the recent-hire valuations, as from terms like “talent bottleneck” and “talent constrained”. Especially “talent constrained” used alongside “funding constrained”. I could be mistaken, but it would seem odd to say you’re “funding constrained” but can’t use more funding at the moment. Whereas we are saying orgs are “talent constrained” but can’t make use of available talent. They evidently don’t function quite the same, so phrasing them in this matched sort of way invites erroneous comparison.
Similarly, I feel a “talent bottleneck” implies an insufficient supply of talent/applicants, which doesn’t seem to be the case. I guess it’s more that there’s insufficient talent actually working on the problems, but it’s not a matter of supply, so it’s more of a “hiring bottleneck” or an “organizational capacity bottleneck”.
EA orgs aren’t so much constrained by a lack of available talent as they are constrained by their capacity to deploy additional talent
I had the same thought that it might be more informative to know EA leaders answers to something like, “I’d rather have $X in additional donations than my next ideal hire.” Agree that it’s difficult to know how a future hire will work out, but maybe there’s still something to be learned from the value they’d place on an additional ideal hire, as it wouldn’t necessarily be the same as holding on to a recent successful hire. I’m admittedly out of my depth here.
We have a forthcoming post on whether the expressions ‘talent gap/bottleneck/constraint’ are generating more heat than light and should be phased out in favour of more specific terms.
Hey Peter—it’s partly for you, but also many other people who have the same questions.
I can comment on the choice of question in the survey as I’m the one who wrote it.
The reason we went for an ex post assessment rather than an ex ante one was that we thought people would be able to more reliably assess how they feel about a previous hire today, than remember how they felt about a previous hire in the past.
Asking people to remember what they thought before runs the risk that they will substitute a hard question (what they thought months ago before they knew how someone would work out) for an easy question (what do they think given what they know now). Then we’d get a similar answer, but it would look ex ante when it actually isn’t.
It also seemed quite difficult for organisations to forecast how much they’ll value a typical hire in the future because, among other reasons, it’s difficult to anticipate how successful future searches will be.
In retrospect I think the framing we chose was probably a mistake, because the two assessments are much more different than most readers understand them to be. I agree with your suggestions for improvements and, indeed, we concluded our blog post with our plans to ask new questions next year or else interview a smaller number of people in more depth.
Hopefully a different approach next year will help us avoid this confusion going forward.
As for the article being misleading, we’ve:
i) Commented that these roles are hard to fill at the point when these figures are first mentioned.
ii) Explained the ex post, ex ante distinction in the relevant section, and now added a link to this post.
iii) Noted we don’t have much confidence in the answers to that question and would not recommend that people update very much based on it.