My current model is that actually very few people who went to DC and did “AI Policy work” chose a career that was well-suited to proposing policies that help with existential risk from AI. In-general people tried to choose more of a path of “try to be helpful to the US government” and “become influential in the AI-adjacent parts of the US government”, but there are almost no people working in DC whose actual job it is to think about the intersection of AI policy and existential risk. Mostly just people whose job it is to “become influential in the US government so that later they can steer the AI existential risk conversation in a better way”.
I find this very sad and consider it one of our worst mistakes, though I am also not confident in that model, and am curious whether people have alternative models.
but there are almost no people working in DC whose actual job it is to think about the intersection of AI policy and existential risk.
That’s probably true because it’s not like jobs like that just happen to exist within government (unfortunately), and it’s hard to create your own role descriptions (especially with something so unusual) if you’re not already at the top.
That said, I think the strategy you describe EAs to have been doing can be impactful? For instance, now that AI risk has gone mainstream, some groups in government are starting to work on AI policy more directly, and if you’re already working on something kind of related and have a bunch of contacts and so on, you’re well-positioned to get into these groups and even get a leading role.
What’s challenging is that you need to make career decisions very autonomously and have a detailed understanding of AI risk and related levers to carve out your own valuable policy work at some point down the line (and not be complacent with “down the line never comes until it’s too late”). I could imagine that there are many EA-minded individuals who went into DC jobs or UK policy jobs with the intent to have an impact on AI later, but they’re unlikely to do much with that because they’re not proactive enough and not “in the weeds” enough with thinking about “what needs to happen, concretely, to avert an AI catastrophe?.”
Even so, I think I know several DC EAs who are exceptionally competent and super tuned in and who’ll likely do impactful work down the line, or are already about to do such things. (And I’m not even particularly connected to that sphere, DC/policy, so there are probably many more really cool EAs/EA-minded folks there that I’ve never talked to or read about.)
My current model is that actually very few people who went to DC and did “AI Policy work” chose a career that was well-suited to proposing policies that help with existential risk from AI. In-general people tried to choose more of a path of “try to be helpful to the US government” and “become influential in the AI-adjacent parts of the US government”, but there are almost no people working in DC whose actual job it is to think about the intersection of AI policy and existential risk. Mostly just people whose job it is to “become influential in the US government so that later they can steer the AI existential risk conversation in a better way”.
I find this very sad and consider it one of our worst mistakes, though I am also not confident in that model, and am curious whether people have alternative models.
That’s probably true because it’s not like jobs like that just happen to exist within government (unfortunately), and it’s hard to create your own role descriptions (especially with something so unusual) if you’re not already at the top.
That said, I think the strategy you describe EAs to have been doing can be impactful? For instance, now that AI risk has gone mainstream, some groups in government are starting to work on AI policy more directly, and if you’re already working on something kind of related and have a bunch of contacts and so on, you’re well-positioned to get into these groups and even get a leading role.
What’s challenging is that you need to make career decisions very autonomously and have a detailed understanding of AI risk and related levers to carve out your own valuable policy work at some point down the line (and not be complacent with “down the line never comes until it’s too late”). I could imagine that there are many EA-minded individuals who went into DC jobs or UK policy jobs with the intent to have an impact on AI later, but they’re unlikely to do much with that because they’re not proactive enough and not “in the weeds” enough with thinking about “what needs to happen, concretely, to avert an AI catastrophe?.”
Even so, I think I know several DC EAs who are exceptionally competent and super tuned in and who’ll likely do impactful work down the line, or are already about to do such things. (And I’m not even particularly connected to that sphere, DC/policy, so there are probably many more really cool EAs/EA-minded folks there that I’ve never talked to or read about.)