Firstly, it seems like the skills of a staff engineer at your current role might be different from what many new EA or AI safety orgs need. For example, you probably handle a lot of design and dependencies currently. Writing, communication is probably a lot of your value. In contrast, in a small, new organization, you probably need to knock out a lot of smaller systems quickly. Your skills and the edge you have might be different between these orgs.
Does the above sound accurate to you, or is it misleading?
Secondly, retirement security was an important point to you. I didn’t fully understand this. My guess is that in your personal case, the security you could provide for yourself and your partner might be large compared to what an EA org (or really most jobs) could easily provide. So my read was that you were talking about more junior engineers and SWE outside of FAANG.
Is this guess wrong? For example, in the US / Cali, maybe I’m really ignorant and you need a large 7 figure nest egg to be safe.
Finally, in terms of financial security, would income stability, such as guaranteeing transitional income if an organization needed to shutdown, substitute for very large income? The idea here is that everyone trusts you, and you were funded to move to successful organizations, so you didn’t have to stay on or bail out zombie organizations.
HR is a big problem facing new companies—talent there is hard to find, too.
Could you write a little more here to make this more legible? Like, is there a book or blog post you can share?
To give context, it’s not clear to me what you mean by HR needs. Do you mean basic operational tasks involved in HR? I know there’s a recruiting org that might be pretty sophisticated, maybe that is what you mean? Maybe I’m really ignorant, but in many tech companies, modulo of the recruiters, it seemed that both talent attraction and team functionality is entirely up to management (e.g. the manager or skip of your “two pizza team”). HR was involved in only pretty fundamental processes, like scheduling interviews or paying out checks (and many of these were contracted out). I knew a few director/VP at a FAANG who said they didn’t understand HR and literally said they provided documentation for terminations.
To be clear, the above could be really dysfunctional/disrespectful and betray my ignorance.
Context/Subtext for my questions:
I thought your comment was more selfless than it seemed, because I didn’t think you were talking about yourself. I think you were talking about the choices other software engineers face.
Another point is that the skill match might suggest EtG, at least for your particular case.
Firstly, it seems like the skills of a staff engineer at your current role might be different from what many new EA or AI safety orgs need. For example, you probably handle a lot of design and dependencies currently. Writing, communication is probably a lot of your value. In contrast, in a small, new organization, you probably need to knock out a lot of smaller systems quickly. Your skills and the edge you have might be different between these orgs.
Yes, I agree but with a huge caveat: every person will progress through various stages of competency during their career. While many early-stage folks could contribute just as well at an early stage EA startup (and should consider it), in the context of the 80k Hours article that I was replying to, we need to be transparent with folks about what a typical career path looks like and what tradeoffs there are to consider down each of the startup vs EtG paths. Here’s the typical career progression for software engineers (though it’s general enough to map onto other fields).
New grad/early stage: needing direction from others on what to work on, executing that work.
Leading self: proposing work in the context of larger goals and then executing that work
Leading a small team: proposing work and technical direction for a small team of engineers. Major design, some direct contributions, work-stream shepherding, mentorship.
Leading a large, ambiguous area/leading multiple teams: proposing strategic direction shifts, aligning team leaders, building consensus without authority, major design work, little direct contributions, mentorship + cultivation at scale.
Leading the entire technical direction for a business function: everything of the previous role, except heavily influencing all of the non-tech functions in the organization.
Regarding which level of progression that individuals might achieve by the end of their career, there’s a bell curve distribution around the 3rd step. Only a handful will ever reach the 6th step of being an executive[2]. FAANG pays somewhere between $750k-$1.5M for step 5, though, and—while still rarefied—it’s attainable for top talent, so a possible EtG goal to plan for.
All of this is a long-winded way of saying that CS folks who are about to graduate shouldn’t throw away a job offer from FAANG for an EA startup, out-of-hand, if they think that they have career luck in their favor. It would be a hard call. If I were 22 and about to graduate today[3], I would give an EA startup 3-5 years to be successful before I switched tactics and tried for a FAANG or other top-of-market option.
Secondly, retirement security was an important point to you. I didn’t fully understand this. My guess is that in your personal case, the security you could provide for yourself and your partner might be large compared to what an EA org (or really most jobs) could easily provide. So my read was that you were talking about more junior engineers and SWE outside of FAANG.
Right: speaking about general SWE population considering private-sector versus non-profits which tend to pay less and also tend to provide less benefits like retirement account funding.
Is this guess wrong? For example, in the US / Cali, maybe I’m really ignorant and you need a large 7 figure nest egg to be safe.
In US/Cali Bay Area (where some EA startups are based), the median house price is $1.3M. So, for someone looking to put down roots in the Bay Area and retire within the same friend network close by, a nest egg of $2M isn’t an unreasonable guess; $3-4m if their spouse isn’t working and they start a family. If we expect EA-ers to come work for a startup in the Bay Area and then move to a lower cost of living place later, we should be transparent about that. (Or, we should be encouraging EA orgs to go remote-first to unlock paying top-of-market rates in rural areas.)
Finally, in terms of financial security, would income stability, such as guaranteeing transitional income if an organization needed to shutdown, substitute for very large income? The idea here is that everyone trusts you, and you were funded to move to successful organizations, so you didn’t have to stay on or bail out zombie organizations.
Yes, that would make the job offers of EAs more attractive to new-career and mid-career folks. It’s probably also applicable to all other roles that an EA would hire for.
HR is a big problem facing new companies—talent there is hard to find, too.
Could you write a little more here to make this more legible? Like, is there a book or blog post you can share?
To give context, it’s not clear to me what you mean by HR needs. Do you mean basic operational tasks involved in HR? Maybe I’m really ignorant, but in many tech companies, it seemed to me like both talent attraction and team functionality is entirely up to management (e.g. the manager or skip of your “two pizza team”). HR was involved in only pretty fundamental processes, like scheduling interviews or paying out checks (and many of these were sub contracted). In fact, I knew a few director/VP at a FAANG who said they didn’t really understand HR and literally said they provided documentation for terminations.
To be clear, the above could be really dysfunctional/disrespectful and betray my ignorance.
It’s common it tech to hear the sentiment of your social network that HR provides no value so I’m not surprised to see this. In Silicon Valley, there’s similar discounting of the value provided by folks in operations, support, logistics and finance.
A note on horizontal organization roles: there are types of roles that apply horizontal, cultural influence. For example, a wise person once said, “If you want to understand why an organization behaves the way that it does, look at the incentives of the people in that organization.”
I point to HR, specifically, because it’s an area where I’ve seen the most struggles in small-stage startups precisely because it is a horizontal force multiplier. Here’s some values that a functioning HR organization provides to a small stage startup:
A meritocratic system of promotion/career advancement that’s seen as fair by the employees. This includes transparency of the expected roles and responsibilities at each stage of career progression. Of course, this is includes some objective criteria for deciding to fire someone, and all of the legal implications thereof (as mentioned above), but that’s not the most important part. Retention is partially a function of aligning the hedonic treadmill with real career progress possibilities.
Setting norms on how individuals interact and, theoretically, backstopping those norms with enforcement. For example, an org might say that aggressive behavior in meetings and emails is not tolerated. This is just a theoretical rule unless org leaders actually back up those words with actions through the promotion process and, in extreme cases, HR-backed disciplinary action. It’s also the function of HR to repeat the company’s behavioral expectations, periodically.
Ensuring fair hiring practices is non-trivial. It’s common in startups to hand-waive over this problem. But actually objectively evaluating candidates and ensuring that bias doesn’t creep in and that pay is equal among all similar roles and levels is hard. Radical transparency can help here but it doesn’t just magically fix the problem.
Setting organizational goals against which the org is measured is sometimes seen as operations or Product Management but there’s an HR role there too: leaders of those sub-orgs that set those goals need to be held accountable and any exec/leadership compensation should be tied to business outcomes in a way that lower-level employees do not face. And all of those company benchmarks and the feedback cycle need to be done in front of the whole employee population, quarterly.
Assessing employee satisfaction and collecting feedback anonymously on an ongoing basis. This can be as simple as an anonymous Google Form that’s open for two weeks once a year. But, actually collating the data, slicing it by org, trending over time, and proposing cultural changes to address employee feed back is hard.
Benefits benefits benefits. This is a constantly evolving space. To some extent, this can be outsourced, but there should be someone on staff continually evaluating the changing landscape of offerings and competitor offerings and continually updating the employees about those changes and acting as a partner to fix problems when they come up.
I could go on but these are the ones that came to mind while I was writing this, and I think that I’ve exceeded the amount of time that I intended to spend on this. 😉
I acknowledge that this assumes a fully meritocratic progression; there are indeed many reasons that individuals might be given these roles without being qualified.
I recognize that founders of startups are not necessarily destined to lead 1,000 employee organizations but that they do need some mix of all of the skills in these stacks. And this is often why startups fail.
Like you, I also think it’s not-obvious that someone of Jason’s skillsets should be doing something other than earning-to-give. As a practical matter, most EA employers aren’t willing to pay >1M/year, except maybe in a few niche situations in ML/AI Safety or very successful EA fintech startups.
(I do vaguely think that if you just naively crunch the numbers about how great SWEs can contribute now vs. Open Phil’s last dollar, there probably exists opportunities somewhat above this bar however, including outside of AI Safety and weird crypto stuff).
There’s also a strong argument that someone like Jason has a clear comparative advantage relative to the rest of the movement in doing E2G stuff in BigTech, and maybe as a community this coordination problem is better solved with less career shuffles.
Hi Jason,
Do you mind humoring a few follow up questions?
Firstly, it seems like the skills of a staff engineer at your current role might be different from what many new EA or AI safety orgs need. For example, you probably handle a lot of design and dependencies currently. Writing, communication is probably a lot of your value. In contrast, in a small, new organization, you probably need to knock out a lot of smaller systems quickly. Your skills and the edge you have might be different between these orgs.
Does the above sound accurate to you, or is it misleading?
Secondly, retirement security was an important point to you. I didn’t fully understand this. My guess is that in your personal case, the security you could provide for yourself and your partner might be large compared to what an EA org (or really most jobs) could easily provide. So my read was that you were talking about more junior engineers and SWE outside of FAANG.
Is this guess wrong? For example, in the US / Cali, maybe I’m really ignorant and you need a large 7 figure nest egg to be safe.
Finally, in terms of financial security, would income stability, such as guaranteeing transitional income if an organization needed to shutdown, substitute for very large income? The idea here is that everyone trusts you, and you were funded to move to successful organizations, so you didn’t have to stay on or bail out zombie organizations.
Could you write a little more here to make this more legible? Like, is there a book or blog post you can share?
To give context, it’s not clear to me what you mean by HR needs. Do you mean basic operational tasks involved in HR? I know there’s a recruiting org that might be pretty sophisticated, maybe that is what you mean? Maybe I’m really ignorant, but in many tech companies, modulo of the recruiters, it seemed that both talent attraction and team functionality is entirely up to management (e.g. the manager or skip of your “two pizza team”). HR was involved in only pretty fundamental processes, like scheduling interviews or paying out checks (and many of these were contracted out). I knew a few director/VP at a FAANG who said they didn’t understand HR and literally said they provided documentation for terminations.
To be clear, the above could be really dysfunctional/disrespectful and betray my ignorance.
Context/Subtext for my questions:
I thought your comment was more selfless than it seemed, because I didn’t think you were talking about yourself. I think you were talking about the choices other software engineers face.
Another point is that the skill match might suggest EtG, at least for your particular case.
Yes, I agree but with a huge caveat: every person will progress through various stages of competency during their career. While many early-stage folks could contribute just as well at an early stage EA startup (and should consider it), in the context of the 80k Hours article that I was replying to, we need to be transparent with folks about what a typical career path looks like and what tradeoffs there are to consider down each of the startup vs EtG paths. Here’s the typical career progression for software engineers (though it’s general enough to map onto other fields).
New grad/early stage: needing direction from others on what to work on, executing that work.
Leading self: proposing work in the context of larger goals and then executing that work
Leading a small team: proposing work and technical direction for a small team of engineers. Major design, some direct contributions, work-stream shepherding, mentorship.
Leading a large, ambiguous area/leading multiple teams: proposing strategic direction shifts, aligning team leaders, building consensus without authority, major design work, little direct contributions, mentorship + cultivation at scale.
Leading the entire technical direction for a business function: everything of the previous role, except heavily influencing all of the non-tech functions in the organization.
A business executive[1]
Regarding which level of progression that individuals might achieve by the end of their career, there’s a bell curve distribution around the 3rd step. Only a handful will ever reach the 6th step of being an executive[2]. FAANG pays somewhere between $750k-$1.5M for step 5, though, and—while still rarefied—it’s attainable for top talent, so a possible EtG goal to plan for.
All of this is a long-winded way of saying that CS folks who are about to graduate shouldn’t throw away a job offer from FAANG for an EA startup, out-of-hand, if they think that they have career luck in their favor. It would be a hard call. If I were 22 and about to graduate today[3], I would give an EA startup 3-5 years to be successful before I switched tactics and tried for a FAANG or other top-of-market option.
Right: speaking about general SWE population considering private-sector versus non-profits which tend to pay less and also tend to provide less benefits like retirement account funding.
In US/Cali Bay Area (where some EA startups are based), the median house price is $1.3M. So, for someone looking to put down roots in the Bay Area and retire within the same friend network close by, a nest egg of $2M isn’t an unreasonable guess; $3-4m if their spouse isn’t working and they start a family. If we expect EA-ers to come work for a startup in the Bay Area and then move to a lower cost of living place later, we should be transparent about that. (Or, we should be encouraging EA orgs to go remote-first to unlock paying top-of-market rates in rural areas.)
Yes, that would make the job offers of EAs more attractive to new-career and mid-career folks. It’s probably also applicable to all other roles that an EA would hire for.
It’s common it tech to hear the sentiment of your social network that HR provides no value so I’m not surprised to see this. In Silicon Valley, there’s similar discounting of the value provided by folks in operations, support, logistics and finance.
A note on horizontal organization roles: there are types of roles that apply horizontal, cultural influence. For example, a wise person once said, “If you want to understand why an organization behaves the way that it does, look at the incentives of the people in that organization.”
I point to HR, specifically, because it’s an area where I’ve seen the most struggles in small-stage startups precisely because it is a horizontal force multiplier. Here’s some values that a functioning HR organization provides to a small stage startup:
A meritocratic system of promotion/career advancement that’s seen as fair by the employees. This includes transparency of the expected roles and responsibilities at each stage of career progression. Of course, this is includes some objective criteria for deciding to fire someone, and all of the legal implications thereof (as mentioned above), but that’s not the most important part. Retention is partially a function of aligning the hedonic treadmill with real career progress possibilities.
Setting norms on how individuals interact and, theoretically, backstopping those norms with enforcement. For example, an org might say that aggressive behavior in meetings and emails is not tolerated. This is just a theoretical rule unless org leaders actually back up those words with actions through the promotion process and, in extreme cases, HR-backed disciplinary action. It’s also the function of HR to repeat the company’s behavioral expectations, periodically.
Ensuring fair hiring practices is non-trivial. It’s common in startups to hand-waive over this problem. But actually objectively evaluating candidates and ensuring that bias doesn’t creep in and that pay is equal among all similar roles and levels is hard. Radical transparency can help here but it doesn’t just magically fix the problem.
Setting organizational goals against which the org is measured is sometimes seen as operations or Product Management but there’s an HR role there too: leaders of those sub-orgs that set those goals need to be held accountable and any exec/leadership compensation should be tied to business outcomes in a way that lower-level employees do not face. And all of those company benchmarks and the feedback cycle need to be done in front of the whole employee population, quarterly.
Assessing employee satisfaction and collecting feedback anonymously on an ongoing basis. This can be as simple as an anonymous Google Form that’s open for two weeks once a year. But, actually collating the data, slicing it by org, trending over time, and proposing cultural changes to address employee feed back is hard.
Benefits benefits benefits. This is a constantly evolving space. To some extent, this can be outsourced, but there should be someone on staff continually evaluating the changing landscape of offerings and competitor offerings and continually updating the employees about those changes and acting as a partner to fix problems when they come up.
I could go on but these are the ones that came to mind while I was writing this, and I think that I’ve exceeded the amount of time that I intended to spend on this. 😉
I acknowledge that this assumes a fully meritocratic progression; there are indeed many reasons that individuals might be given these roles without being qualified.
I recognize that founders of startups are not necessarily destined to lead 1,000 employee organizations but that they do need some mix of all of the skills in these stacks. And this is often why startups fail.
Full disclosure: I do not have a degree and am an anomaly. So, I can’t really speak with authenticity on this hypothetical.
Thanks so much! I thought this was incredibly informative and in-depth. There are some really valuable insights here.
Like you, I also think it’s not-obvious that someone of Jason’s skillsets should be doing something other than earning-to-give. As a practical matter, most EA employers aren’t willing to pay >1M/year, except maybe in a few niche situations in ML/AI Safety or very successful EA fintech startups.
(I do vaguely think that if you just naively crunch the numbers about how great SWEs can contribute now vs. Open Phil’s last dollar, there probably exists opportunities somewhat above this bar however, including outside of AI Safety and weird crypto stuff).
There’s also a strong argument that someone like Jason has a clear comparative advantage relative to the rest of the movement in doing E2G stuff in BigTech, and maybe as a community this coordination problem is better solved with less career shuffles.
Ozzie’s post on opportunity costs of technical talent is related.