Software engineering could be a great option for having a direct impact on the world’s most pressing problems, particularly in AI safety, but also in biosecurity and across other cause areas. This will probably be more impactful than earning to give.
As with operations staff, organisations need exceptional and mission-aligned software engineers. But many still find it difficult to hire.
Some myths:
You need an ML background to work as an engineer on AI safety.
Outside AI safety, the only useful software skill is front-end web development.
Effective organisations will pay far less than top companies.
None of these things are true.
In fact, many organisations have budgets of $10s of millions, and think that software engineers can substantially increase their cost-effectiveness (e.g. in his 80,000 Hours podcast, Chris Olah argues that Anthropic’s systems researcher could easily increase their efficiency by at least 10%). So even if you’re earning 7 figures, you could be more effective doing direct impact work.
This rest of this post contains an excerpt from my new career review of software engineering for 80,000 Hours, focusing on the parts most relevant to already-engaged EAs.
This review owes a lot to helpful discussions with (and comments from) Andy Jones, Ozzie Gooen, Jeff Kaufman, Sasha Cooper, Ben Kuhn, Nova DasSarma, Kamal Ndousse, Ethan Alley, Ben West, Ben Mann, Tom Conerly, Zac Hatfield-Dodds, and George McGowan. Special thanks go to Roman Duda for our previous review of software engineering, on which this was based.
Why might software engineering be high impact?
Software engineers are in a position to meaningfully contribute directly to solving a wide variety of the world’s most pressing problems.
In particular, there is a shortage of software engineers at the cutting edge of research into AI safety.
We’ve also found that software engineers can contribute greatly to work aiming at preventing pandemics and other global catastrophic biological risks.
Even for skilled engineers who could command high salaries, we think that working directly on a problem will probably be more impactful than earning to give.
Some examples of projects where software engineering is central to their impactful work:
Ought is an AI research lab attempting to build systems that apply machine learning to the task of helping people think.
The Secure DNA Project is attempting to build a secure global screening system for DNA sequences that could be used to engineer a global pandemic.
Wave is a startup building a way for people in developing countries to access financial services.
Most organisations, even ones that don’t focus on developing large software products, need software engineers to manage computer systems, apps, and websites. For example:
Effective Altruism Funds provides expert-managed funds to facilitate donors maximising the impact of their donations. They use software engineers to design and maintain their online platform.
Many people we’ve spoken to at these and other organisations have said that they have real difficulty hiring extremely talented software engineers. Many nonprofits want to hire people who believe in their missions (just as they do with operations staff), which indicates that talented, altruistic-minded software engineers are sorely needed and could do huge amounts of good.
Smaller organisations that don’t focus on engineering often only have one or two software engineers. And because things at small organisations can change rapidly, they need unusually adaptable and flexible people who are able to maintain software with very little help from the wider team.1
It seems likely that, as the community of people working on helping future generations grows, there will be more opportunities for practical software development efforts to help. This means that even if you don’t currently have any experience with programming, it could be valuable to begin developing expertise in software engineering now.
With the rise of machine learning, and the huge success of deep learning models like GPT-3, many experts now think it’s reasonably likely that our current machine learning methods could be used to create transformative artificial intelligence.
This has led to an explosion in empirical AI safety research, where teams work directly with deep neural networks to identify risks and develop frameworks for mitigating them. Examples of organisations working in empirical AI safety research include Redwood Research, DeepMind, OpenAI, and Anthropic.
These organisations are doing research directly with extremely large neural networks, which means each experiment can cost millions of dollars to run. This means that even small improvements to the efficiency of each experiment can be hugely beneficial.
There’s also often overlap between experimental results that will help further AI safety and results that could accelerate the development of unsafe AI, so it’s also important that the results of these experiments are kept secure.
As a result, it’s likely to remain incredibly valuable to have talented engineers working on ensuring that these experiments are as efficient and safe as possible. Experts we spoke to expect this to remain a key bottleneck in AI safety research for many years.
However, there is a serious risk associated with this route: it seems possible for engineers to accidentally increase risks from AI by generally accelerating the technical development of the field. We’re not sure of the more precise contours of this risk (e.g. exactly what kinds of projects you should avoid), but think it’s important to watch out for. That said, there are many more junior non-safety roles out there than roles focused specifically on safety, and experts we’ve spoken to expect that most non-safety projects aren’t likely to be causing harm. If you’re uncertain about taking a job for this reason, our team may be able to help you decide.
Software engineer salaries mean you can earn to give
In general, if you can find a job you can do well, you’ll have a bigger impact working on a problem directly than you would by earning money and donating. However, earning to give can still be a high-impact option, especially if you focus on donating to the most effective projects that could use the extra funds.
If you’re skilled enough to work at top companies, software engineering is a well-paid career. In the US, entry-level software engineer salaries start at around $110,000. Engineers at Microsoft start at $150,000, and engineers at Google start at around $180,000 (including stock and bonuses). If you’re successful, after a few years on the job you could be earning over $500,000 a year.
Pay is generally much lower in other countries. Median salaries in Australia are around 20% lower than salaries in the US (approximately US$80,000), and around 40% lower in the UK, Germany, Canada, and Japan (approximately US$60,000). While much of your earnings as a software engineer come from bonuses and equity, rather than just your salary, these are also lower outside the US.
If you do want to make a positive difference through donating part of your income as a software engineer, you may be able to increase your impact by using donation-matching programmes, which are common at large tech companies (although these are often capped at around US$10,000 per year).
You can read more about salaries at large tech companies below.
It’s important to note that many nonprofit organisations, including those focusing on AI safety, will offer salaries and benefits that compete with those at for-profit firms.
If you work at or found a startup, your earnings will be highly variable. However, the expected value of your earnings — especially as a cofounder — could be extremely high. For this reason, if you’re a particularly good fit, founding a tech startup and donating your earnings could be hugely impactful, as you could earn and donate extraordinary amounts.
Moving to a direct impact software engineering role
Working in AI safety
If you are looking to work in an engineering role in an AI safety or other research organisation, you will probably want to focus on back-end software development (although there are also front-end roles, particularly those focusing on gathering data from humans on which models can be trained and tested). There are recurring opportunities for software engineers with a range of technical skills (to see examples, take a look at our job board).
If you have the opportunity to choose areas in which you could gain expertise, the experienced engineers we spoke to suggested focusing on:
Distributed systems
Numerical systems
Security
In general, it helps to have expertise in any specific, hard-to-find skillsets.
This work uses a range of programming languages, including Python, Rust, C++ and JavaScript. Functional languages such as Haskell are also common.
We’ve previously written about how to move into a machine learning career for AI safety. We now think it is easier than we previously thought to move into an AI-safety-related software engineering role without explicit machine learning experience.
The Effective Altruism Long-Term Future Fund and the Survival and Flourishing Fund may provide funding for promising individuals to learn skills relevant to helping future generations, including new technologies such as machine learning. If you already have software engineering experience, but would benefit from explicit machine learning or AI safety experience, this could be a good option for you.
Anyone with a good understanding of how to build software can be useful in these small organisations, even if they don’t have much experience. However, if you want to work in this space, you’ll need to be comfortable getting your hands dirty and doing whatever needs to be done, even when the work isn’t the most intellectually challenging. For this reason, it could be particularly useful to have experience working in a software-based startup.
Much of the work in biosecurity is related to handling and processing large amounts of data, so knowledge of how to work with distributed systems is in demand. Expertise in adjacent fields such as data science could also be helpful.
There is also a big focus on security, particularly at organisations like SecureDNA.
Most code in biosecurity is written in Python.
If you’re interested in working on biosecurity and pandemic preparedness as a software engineer, you can find open positions on our job board.
Other important direct work
Nonprofit organisations and altruistic-minded startups often have very few team members. And no matter what an organisation does, they almost always have some need for engineers (for example, 80,000 Hours is not a software organisation, but we employ two developers). So if you find an organisation you think is doing something really useful, working as a software engineer for them might be an excellent way to support that work.
Engineering for a small organisation likely means doing work across the development process, since there are few other engineers.
Often these organisations are focused on front-end development, with jobs ranging from application development and web development to data science and project management roles. There are often also opportunities for full-stack developers with a broad range of experience.
Founding an organisation yourself is more challenging, but can be even more impactful. And if you’ve worked in a small organisation or a startup before, you might have the broad skills and entrepreneurialism that’s required to succeed. See our profile on founding new high-impact projects for more.
Reasons not to go into software engineering
We think that most people with good general intelligence will be able to do well at software engineering. And because it’s very easy to test out (see the section on how to predict your fit in advance), you’ll be able to tell early on whether you’re likely to be a good fit.
However, there are lots of other paths that seem like particularly promising ways to help solve the world’s most pressing problems, and it’s worth looking into them. If you find programming difficult, or unenjoyable, your personal fit for other career paths may be higher. And even if you enjoy it and you’re good at it, we think that will be true for lots of people, so that’s not a good reason to think you won’t be even better at something else!
As a result, it’s important to test your fit for a variety of options. Try taking a look at our other career reviews to find out more.
Software engineering—Career review
Link post
Summary
Software engineering could be a great option for having a direct impact on the world’s most pressing problems, particularly in AI safety, but also in biosecurity and across other cause areas. This will probably be more impactful than earning to give.
As with operations staff, organisations need exceptional and mission-aligned software engineers. But many still find it difficult to hire.
Some myths:
You need an ML background to work as an engineer on AI safety.
Outside AI safety, the only useful software skill is front-end web development.
Effective organisations will pay far less than top companies.
None of these things are true.
In fact, many organisations have budgets of $10s of millions, and think that software engineers can substantially increase their cost-effectiveness (e.g. in his 80,000 Hours podcast, Chris Olah argues that Anthropic’s systems researcher could easily increase their efficiency by at least 10%). So even if you’re earning 7 figures, you could be more effective doing direct impact work.
This rest of this post contains an excerpt from my new career review of software engineering for 80,000 Hours, focusing on the parts most relevant to already-engaged EAs.
This review owes a lot to helpful discussions with (and comments from) Andy Jones, Ozzie Gooen, Jeff Kaufman, Sasha Cooper, Ben Kuhn, Nova DasSarma, Kamal Ndousse, Ethan Alley, Ben West, Ben Mann, Tom Conerly, Zac Hatfield-Dodds, and George McGowan. Special thanks go to Roman Duda for our previous review of software engineering, on which this was based.
Why might software engineering be high impact?
Software engineers are in a position to meaningfully contribute directly to solving a wide variety of the world’s most pressing problems.
In particular, there is a shortage of software engineers at the cutting edge of research into AI safety.
We’ve also found that software engineers can contribute greatly to work aiming at preventing pandemics and other global catastrophic biological risks.
Aside from direct work on these crucial problems, while working for startups or larger tech companies you can gain excellent career capital (especially technical skills), and, if you choose, earn and donate substantial amounts to the world’s best charities.
How to do good as a software engineer
Even for skilled engineers who could command high salaries, we think that working directly on a problem will probably be more impactful than earning to give.
Some examples of projects where software engineering is central to their impactful work:
Ought is an AI research lab attempting to build systems that apply machine learning to the task of helping people think.
The Secure DNA Project is attempting to build a secure global screening system for DNA sequences that could be used to engineer a global pandemic.
Momentum is a startup building donation pages that encourage recurring donations to the world’s most effective charities.
Lightcone Infrastructure builds software-based infrastructure for longtermist projects.
Telis Bioscience is a startup attempting to radically accelerate drug development to mitigate risks from future pandemics.
Anthropic is a research company working to build reliable, interpretable, and steerable AI systems.
Redwood Research conducts applied research into the challenge of aligning artificial intelligence.
Wave is a startup building a way for people in developing countries to access financial services.
Most organisations, even ones that don’t focus on developing large software products, need software engineers to manage computer systems, apps, and websites. For example:
Effective Altruism Funds provides expert-managed funds to facilitate donors maximising the impact of their donations. They use software engineers to design and maintain their online platform.
Our World in Data, founded by Max Roser, collects and presents data on many of the world’s most pressing problems. They use software engineers to maintain their website, analyse data, and develop their open-source data visualisation tool.
Here at 80,000 Hours, our team includes two software engineers working on our website and other technology.
The Centre for Effective Altruism’s tech team supports a variety of projects across the effective altruism community, such as the Effective Altruism Forum.
Many people we’ve spoken to at these and other organisations have said that they have real difficulty hiring extremely talented software engineers. Many nonprofits want to hire people who believe in their missions (just as they do with operations staff), which indicates that talented, altruistic-minded software engineers are sorely needed and could do huge amounts of good.
Smaller organisations that don’t focus on engineering often only have one or two software engineers. And because things at small organisations can change rapidly, they need unusually adaptable and flexible people who are able to maintain software with very little help from the wider team.1
It seems likely that, as the community of people working on helping future generations grows, there will be more opportunities for practical software development efforts to help. This means that even if you don’t currently have any experience with programming, it could be valuable to begin developing expertise in software engineering now.
Software engineers can help with AI safety
We’ve argued before that artificial intelligence could have a deeply transformative impact on our society. There are huge opportunities associated with this ongoing transformation, but also extreme risks — potentially even threatening humanity’s survival.
With the rise of machine learning, and the huge success of deep learning models like GPT-3, many experts now think it’s reasonably likely that our current machine learning methods could be used to create transformative artificial intelligence.
This has led to an explosion in empirical AI safety research, where teams work directly with deep neural networks to identify risks and develop frameworks for mitigating them. Examples of organisations working in empirical AI safety research include Redwood Research, DeepMind, OpenAI, and Anthropic.
These organisations are doing research directly with extremely large neural networks, which means each experiment can cost millions of dollars to run. This means that even small improvements to the efficiency of each experiment can be hugely beneficial.
There’s also often overlap between experimental results that will help further AI safety and results that could accelerate the development of unsafe AI, so it’s also important that the results of these experiments are kept secure.
As a result, it’s likely to remain incredibly valuable to have talented engineers working on ensuring that these experiments are as efficient and safe as possible. Experts we spoke to expect this to remain a key bottleneck in AI safety research for many years.
However, there is a serious risk associated with this route: it seems possible for engineers to accidentally increase risks from AI by generally accelerating the technical development of the field. We’re not sure of the more precise contours of this risk (e.g. exactly what kinds of projects you should avoid), but think it’s important to watch out for. That said, there are many more junior non-safety roles out there than roles focused specifically on safety, and experts we’ve spoken to expect that most non-safety projects aren’t likely to be causing harm. If you’re uncertain about taking a job for this reason, our team may be able to help you decide.
Software engineer salaries mean you can earn to give
In general, if you can find a job you can do well, you’ll have a bigger impact working on a problem directly than you would by earning money and donating. However, earning to give can still be a high-impact option, especially if you focus on donating to the most effective projects that could use the extra funds.
If you’re skilled enough to work at top companies, software engineering is a well-paid career. In the US, entry-level software engineer salaries start at around $110,000. Engineers at Microsoft start at $150,000, and engineers at Google start at around $180,000 (including stock and bonuses). If you’re successful, after a few years on the job you could be earning over $500,000 a year.
Pay is generally much lower in other countries. Median salaries in Australia are around 20% lower than salaries in the US (approximately US$80,000), and around 40% lower in the UK, Germany, Canada, and Japan (approximately US$60,000). While much of your earnings as a software engineer come from bonuses and equity, rather than just your salary, these are also lower outside the US.
If you do want to make a positive difference through donating part of your income as a software engineer, you may be able to increase your impact by using donation-matching programmes, which are common at large tech companies (although these are often capped at around US$10,000 per year).
You can read more about salaries at large tech companies below.
It’s important to note that many nonprofit organisations, including those focusing on AI safety, will offer salaries and benefits that compete with those at for-profit firms.
If you work at or found a startup, your earnings will be highly variable. However, the expected value of your earnings — especially as a cofounder — could be extremely high. For this reason, if you’re a particularly good fit, founding a tech startup and donating your earnings could be hugely impactful, as you could earn and donate extraordinary amounts.
Moving to a direct impact software engineering role
Working in AI safety
If you are looking to work in an engineering role in an AI safety or other research organisation, you will probably want to focus on back-end software development (although there are also front-end roles, particularly those focusing on gathering data from humans on which models can be trained and tested). There are recurring opportunities for software engineers with a range of technical skills (to see examples, take a look at our job board).
If you have the opportunity to choose areas in which you could gain expertise, the experienced engineers we spoke to suggested focusing on:
Distributed systems
Numerical systems
Security
In general, it helps to have expertise in any specific, hard-to-find skillsets.
This work uses a range of programming languages, including Python, Rust, C++ and JavaScript. Functional languages such as Haskell are also common.
We’ve previously written about how to move into a machine learning career for AI safety. We now think it is easier than we previously thought to move into an AI-safety-related software engineering role without explicit machine learning experience.
The Effective Altruism Long-Term Future Fund and the Survival and Flourishing Fund may provide funding for promising individuals to learn skills relevant to helping future generations, including new technologies such as machine learning. If you already have software engineering experience, but would benefit from explicit machine learning or AI safety experience, this could be a good option for you.
If you think you could, with a few weeks’ work, write a new feature or fix a bug in a major machine learning library, then you could probably apply directly for engineering roles at top AI safety labs (such as Redwood Research, DeepMind, OpenAI, and Anthropic), without needing to spend more time building experience in software engineering. These top labs offer pay that is comparable to pay at large tech firms.
If you are considering joining an AI safety lab in the near future, our team may be able to help.
Working on reducing global catastrophic biological risks
Reducing global catastrophic biological risks — for example, research into screening for novel pathogens to prevent future pandemics — is likely to be one of the most important ways to help solve the world’s most pressing problems.
Through organisations like Telis Bioscience and SecureDNA (and other projects that might be founded in the future), there are significant opportunities for software engineers to contribute to reducing these risks.
Anyone with a good understanding of how to build software can be useful in these small organisations, even if they don’t have much experience. However, if you want to work in this space, you’ll need to be comfortable getting your hands dirty and doing whatever needs to be done, even when the work isn’t the most intellectually challenging. For this reason, it could be particularly useful to have experience working in a software-based startup.
Much of the work in biosecurity is related to handling and processing large amounts of data, so knowledge of how to work with distributed systems is in demand. Expertise in adjacent fields such as data science could also be helpful.
There is also a big focus on security, particularly at organisations like SecureDNA.
Most code in biosecurity is written in Python.
If you’re interested in working on biosecurity and pandemic preparedness as a software engineer, you can find open positions on our job board.
Other important direct work
Nonprofit organisations and altruistic-minded startups often have very few team members. And no matter what an organisation does, they almost always have some need for engineers (for example, 80,000 Hours is not a software organisation, but we employ two developers). So if you find an organisation you think is doing something really useful, working as a software engineer for them might be an excellent way to support that work.
Engineering for a small organisation likely means doing work across the development process, since there are few other engineers.
Often these organisations are focused on front-end development, with jobs ranging from application development and web development to data science and project management roles. There are often also opportunities for full-stack developers with a broad range of experience.
Founding an organisation yourself is more challenging, but can be even more impactful. And if you’ve worked in a small organisation or a startup before, you might have the broad skills and entrepreneurialism that’s required to succeed. See our profile on founding new high-impact projects for more.
Reasons not to go into software engineering
We think that most people with good general intelligence will be able to do well at software engineering. And because it’s very easy to test out (see the section on how to predict your fit in advance), you’ll be able to tell early on whether you’re likely to be a good fit.
However, there are lots of other paths that seem like particularly promising ways to help solve the world’s most pressing problems, and it’s worth looking into them. If you find programming difficult, or unenjoyable, your personal fit for other career paths may be higher. And even if you enjoy it and you’re good at it, we think that will be true for lots of people, so that’s not a good reason to think you won’t be even better at something else!
As a result, it’s important to test your fit for a variety of options. Try taking a look at our other career reviews to find out more.
You can read the full review here.