I agree this is a very helpful comment. I would add: these roles in my view are not *lesser* in any sense, for a range of reasons and I would encourage people not to think of them in those terms.
You might have a bigger impact on the margins being the only—or one of the first few—people thinking in EA terms in a philanthropic foundation than by adding to the pool of excellence at OpenPhil. This goes for any role that involves influencing how resources are allocated—which is a LOT, in charity, government, industry, academic foundations etc.
You may not be in the presidential cabinet, or a spad to the UK prime minister, but those people are supported and enabled by people building up the resources, capacity, overton window expansion elsewhere in government and civil service. The ‘senior person’ on their own may not be able to achieve purchase with key policy ideas and influence.
A lot of xrisk research, from biosecurity to climate change, draws on and depends on a huge body of work on biology, public policy, climate science, renewable energy, insulation in homes, and much more. Often there are gaps in research on extreme scenarios due to lack of incentives for this kind of work, and other reasons—and this may make it particularly impactful at times. But that specific work can’t be done well without drawing on all the underlying work. E.g., biorisk mitigation needs not just the people figuring out how to defend against the extreme scenarios, but also everything from people testing birds in vietnam for H5N1 and seals in the north sea for H7, to people planning for overflow capacity in regional hospitals, to people pushing for the value of preparedness funds in the reinsurance industry to much more. Same for climate+environment, same will be true for AI policy etc.
I think there’s probably a good case to be made that in many or perhaps most instances the most useful place for the next generally capable EA to be is *not* an EA org. And for all 80k’s great work, they can’t survey and review everything, nor tailor to personal fit for the thousands, or hundreds of thousands of different-skillset people who can play a role in making the future better.
For EA to really make the future better to the extent that it has the potential, it’s going to need a *much* bigger global team. And that team’s going to need to be interspersed everywhere, sometimes doing glamorous stuff, sometimes doing more standard stuff that is just as important in that it makes the glamorous stuff possible. To annoy everyone with a sports analogy, the defense and midfield positions are every bit as important as the glamorous striker positions, and if you’ve got a team made up primarily of star strikers and wannabe star strikers, that team’s going to underperform.
To annoy everyone with a sports analogy, the defense and midfield positions are every bit as important as the glamorous striker positions, and if you’ve got a team made up primarily of star strikers and wannabe star strikers, that team’s going to underperform.
But the marginal impact of becoming a star striker is so high!
(Just kidding – this is a great analogy & highlights a big problem with reasoning on the margin + focusing on maximizing individual impact.)
I also like the analogy, let’s run with it. Suppose I’m reasoning from the point of view of the movement as a whole, and we’re trying to put together a soccer team. Suppose also that there are two types of positions, midfield and striker. I’m not sure if this is true for strikers in what I would call soccer, but suppose the striker has a higher skillcap than midfield.[1] I’ll define skillcap as the amount of skill with the position before the returns begin to diminish.
Where skill is some product of standard deviation of innate skill and hours practiced.
Back to the problem of putting together a soccer team, if you’re starting with a bunch of players of unknown innate skill, you would get a higher expected value to tell 80% of your players to train to be strikers, and 20% to be midfielders. Because you have a smaller pool, your midfielders will have less innate talent for the position. You can afford to lose this however, as the effect will be small compared to the gain in the increased performance of the strikers.
That’s not to say that you should fill your entire team with wannabe strikers. When you select your team you’ll undoubtedly leave out some very dedicated strikers in favor of someone who trained for midfield. Still, compared to the percentage that end up playing on the team, the people you’d want training for the role leans more towards the high-skillcap positions.
There are all sorts of ways this analogy doesn’t apply directly to the real world, but it might help pump intuitions.
[1] For American football, the quarterback position definitely exhibits this effect. The effect can be seen clearly in this list of highest-paid players.
There are all sorts of ways this analogy doesn’t apply directly to the real world, but it might help pump intuitions.
Yeah, I think this model misses that people who are aiming to be strikers tend to have pretty different dispositions than people aiming to be midfielders. (And so filling a team mostly with intending-to-be-strikers could have weird effects on team cohesion & function.)
Interesting to think about how Delta Force, SEAL Team Six, etc. manage this, as they select for very high-performing recruits (all strikers) then meld them into cohesive teams. I believe they do it via:
1. having a very large recruitment pool
2. intense filtering out of people who don’t meet their criteria
3. breaking people down psychologically + cultivating conformity during training
I found it interesting to cash this out more… thanks!
Ah, so like, in the “real world”, you don’t have a set of people, you end up recruiting a training class of 80% would-be-strikers, which influences the culture compared to if you recruited for the same breakdown as the eventually-selected-team?
I agree this is a very helpful comment. I would add: these roles in my view are not *lesser* in any sense, for a range of reasons and I would encourage people not to think of them in those terms.
You might have a bigger impact on the margins being the only—or one of the first few—people thinking in EA terms in a philanthropic foundation than by adding to the pool of excellence at OpenPhil. This goes for any role that involves influencing how resources are allocated—which is a LOT, in charity, government, industry, academic foundations etc.
You may not be in the presidential cabinet, or a spad to the UK prime minister, but those people are supported and enabled by people building up the resources, capacity, overton window expansion elsewhere in government and civil service. The ‘senior person’ on their own may not be able to achieve purchase with key policy ideas and influence.
A lot of xrisk research, from biosecurity to climate change, draws on and depends on a huge body of work on biology, public policy, climate science, renewable energy, insulation in homes, and much more. Often there are gaps in research on extreme scenarios due to lack of incentives for this kind of work, and other reasons—and this may make it particularly impactful at times. But that specific work can’t be done well without drawing on all the underlying work. E.g., biorisk mitigation needs not just the people figuring out how to defend against the extreme scenarios, but also everything from people testing birds in vietnam for H5N1 and seals in the north sea for H7, to people planning for overflow capacity in regional hospitals, to people pushing for the value of preparedness funds in the reinsurance industry to much more. Same for climate+environment, same will be true for AI policy etc.
I think there’s probably a good case to be made that in many or perhaps most instances the most useful place for the next generally capable EA to be is *not* an EA org. And for all 80k’s great work, they can’t survey and review everything, nor tailor to personal fit for the thousands, or hundreds of thousands of different-skillset people who can play a role in making the future better.
For EA to really make the future better to the extent that it has the potential, it’s going to need a *much* bigger global team. And that team’s going to need to be interspersed everywhere, sometimes doing glamorous stuff, sometimes doing more standard stuff that is just as important in that it makes the glamorous stuff possible. To annoy everyone with a sports analogy, the defense and midfield positions are every bit as important as the glamorous striker positions, and if you’ve got a team made up primarily of star strikers and wannabe star strikers, that team’s going to underperform.
But the marginal impact of becoming a star striker is so high!
(Just kidding – this is a great analogy & highlights a big problem with reasoning on the margin + focusing on maximizing individual impact.)
I also like the analogy, let’s run with it. Suppose I’m reasoning from the point of view of the movement as a whole, and we’re trying to put together a soccer team. Suppose also that there are two types of positions, midfield and striker. I’m not sure if this is true for strikers in what I would call soccer, but suppose the striker has a higher skillcap than midfield.[1] I’ll define skillcap as the amount of skill with the position before the returns begin to diminish.
Where skill is some product of standard deviation of innate skill and hours practiced.
Back to the problem of putting together a soccer team, if you’re starting with a bunch of players of unknown innate skill, you would get a higher expected value to tell 80% of your players to train to be strikers, and 20% to be midfielders. Because you have a smaller pool, your midfielders will have less innate talent for the position. You can afford to lose this however, as the effect will be small compared to the gain in the increased performance of the strikers.
That’s not to say that you should fill your entire team with wannabe strikers. When you select your team you’ll undoubtedly leave out some very dedicated strikers in favor of someone who trained for midfield. Still, compared to the percentage that end up playing on the team, the people you’d want training for the role leans more towards the high-skillcap positions.
There are all sorts of ways this analogy doesn’t apply directly to the real world, but it might help pump intuitions.
[1] For American football, the quarterback position definitely exhibits this effect. The effect can be seen clearly in this list of highest-paid players.
Yeah, I think this model misses that people who are aiming to be strikers tend to have pretty different dispositions than people aiming to be midfielders. (And so filling a team mostly with intending-to-be-strikers could have weird effects on team cohesion & function.)
Interesting to think about how Delta Force, SEAL Team Six, etc. manage this, as they select for very high-performing recruits (all strikers) then meld them into cohesive teams. I believe they do it via:
1. having a very large recruitment pool
2. intense filtering out of people who don’t meet their criteria
3. breaking people down psychologically + cultivating conformity during training
I found it interesting to cash this out more… thanks!
Ah, so like, in the “real world”, you don’t have a set of people, you end up recruiting a training class of 80% would-be-strikers, which influences the culture compared to if you recruited for the same breakdown as the eventually-selected-team?
I really enjoy the extent to which you’ve both taken the ball and run with it ;)