Part 1: EA tech work is inefficiently allocated & bad for technical career capital

Arepo25 Jul 2021 13:03 UTC
59 points
13 comments14 min readEA link

Epistemic status of sequence: empirically untested. Therein lies the excitement …

Tl;dr of the sequence

In part 1, I argue that tech work at EA orgs has three predictable problems:[1]

  • It’s bad for developing technical skills

  • It’s inefficiently allocated

  • It’s difficult to assess hires

The common root of these problems is siloed small/​nonexistent tech departments across EA orgs.

In part 2 I lay out the case that moving tech work to agencies would largely address these problems.[2] Qualitatively I would expect them to either increase the availability of EA tech staff, increase their career capital or both, and to help assess new tech hires. Quantitatively, I argue that if all EA nonprofits with an existing tech department were to fold their tech departments into an agency, it would save the movement approximately 11-43% of the cost of a software developer per organisation in efficiency gains from greater productivity.

In part 3, I look at three modes of funding outsourced EA tech work, which cut across both agencies and individuals: donor-funded[3], low-bono and full-profit. I’ll argue that there are several advantages of a donor-funded agency over the other funding modes, both in the value it could add to the space of EA tech work and in the appeal of working for it as a developer. I’ll then look at some of the unique challenges a donor funded agency would face (such as lack of market incentives), and consider ways to resolve them and in some cases turn them to its advantage.

In the final part, I look at the pros and cons of an alternative mode—of effectively donor-funding an ‘agency’ that’s a branch of an existing org—and suggest tentatively that it might be a good option to get started, planning eventually to spin the orgs off into independence. Having focused on tech agencies throughout I’ll briefly look at how much the overall reasoning could apply to the case for other specialties.

I’ll prompt again for input at the very end, but feedback would be especially valuable from three groups of people (feel free to PM me if you’d prefer not to answer publicly) - though I do recommend reading the whole sequence first:

  • software developers: would you be interested in working for either a full-profit agency or a donor-funded one?

  • project managers at EA orgs, or people thinking of starting EA orgs: would you consider working with such an agency if it existed?

  • grant managers at OpenPhil, EA Funds et al: would you consider funding or contributing to a tech agency that didn’t charge its ‘clients’?

Part 1 Introduction

For about three years I worked as a software developer at an EA organisation, mostly as the only active developer on the team. This was a rewarding time overall, and I strongly believed and believe in their mission; however, working in that environment as a lone developer brought with it some fundamental problems that seemed likely to generalise to many other EA nonprofits. EA nonprofits typically have a) 0 or 1 software developers, and rarely more than 2, and b) a lot of conceptual overlap in their tech work requirements—to wit, various flavours of web development—yet very little systemic cooperation between these departments.

In this post I explore three distinct negative effects of these common traits:

  1. EA tech work is bad for developing technical skills

  2. EA tech work is inefficiently allocated

  3. EA tech work is difficult to assess hires for

EA tech work is bad for developing technical skills

As a long-time advocate of intra-movement support processes, I find this effect the most concerning of the three.

Four years ago Ben Todd observed that the EA movement struggled to recruit software engineers. Two years ago Jeff Kaufman observed a similar difficulty. Some people in those threads speculated that the difficulty might come from low salaries; others thought it was due to EA tech work being unrewarding.[4]

I speculated that, given my own experience plus what I’d learned anecdotally of other EA orgs, these concerns would generalise to the wider movement. To test this theory, I polled and then ran follow-up interviews with EAs who’d chosen not to work at EA orgs, and also interviewed the existing software developers at some EA organisations.

Interviews with software developers not working at an EA org

Last year I created a series of Facebook polls, in the ‘Software, Data, and Tech Effective Altruism in London’, the ‘Software, Data, and Tech Effective Altruism’, and the ‘Effective Altruism Polls’ groups (posted Feb 20, Jan 21, and Feb 21 respectively)[5] asking the question ‘Software developers both in industry and at EA orgs, what reservations about/​critiques of the EA tech sector do you have?’ If you can’t or don’t want to view those groups or want to see the totals in one place, the results are grouped here.

The three most common choices were

  1. I don’t think I would get enough career capital (12 agreed)

  2. EA orgs don’t pay enough/​not good enough perks (11 agreed)

  3. I want to work in a larger tech department (5+ people) (10 agreed)

I believed that 1 and 3 were highly overlapping, and sent some semi-structured follow-up questions to respondents to the first poll, based on the responses they’d given, to test this belief and to explore the question of pay. You can see the pseudonymised answers in full. They’re difficult to summarise, but some themes stuck out.

On pay/​perks

I asked those who’d agreed about low pay/​perks what they imagined the salaries would be, vs the minimum salary they’d accept, and got the following answers:

  • Guessed £25k, would be willing to start at £35k (CS degree, no paid coding experience)

  • Guessed £40k, would be willing to start at £50k (8 years experience)

  • Guessed £38-50k, would be willing to start at £45k (2-4 years experience)

  • Guessed £50k, would be willing to start at £60-70k (experience ‘depends what you count, between 1.5-5 years’)

  • Guessed ‘10-20k less than other places at least’, and ‘wouldn’t move jobs for less than £70k, but I wouldn’t want to stick at £70k forever as a few years later I’d expect to be on £100k as a senior engineer’ (5 years experience)

  • One asked me to not to publicise their figures, but guessed in the same range as the others

This is very limited data, and very dependent on experience and location (all the people who answered this question were based or expected to be based in London) but it’s interesting that a) their guesses at EA salaries are a long way below market rate for their experience (base rate of about £48,000-53,000 in London going by Glassdoor average for a mid-level developer/​engineer, which probably means about 3+ years of experience), b) their starting rates still seem a little below market rate for their experience, and c) their guesses are mostly less than I was paid by the time I left (then with about 5 years experience), and much less than the salary of one of the two EA-employed software developers who shared theirs with me (see below). They’re also in the range of the two EA software jobs currently open and specifying salary at the time of writing (£50,000-70,000 for CEA and £53,000-65,000 for GWWC), though CEA and GWWC might have higher budgets than the average EA org.

The perk suggestions were a mix of benefits, working flexibility, and learning support (ie career capital):

  • ‘Standard trendy startupy perks appeal to me (free meals, good time off, nice offices)’

  • ‘An awesome perk would be the ability to work part time (which would also be really helpful if I was at the end of my career and/​or bringing up a family at that point in my life).’

  • Min salary ‘would be higher for an org which I thought didn’t have good potential for personal development (no real eng department/​culture, simple tasks like maintaining a website) and lower for an org where I thought I would learn a lot (e.g: research engineer position at imperial)’

  • ‘There’s lots of benefits and perks that would be nice to have: Wellness budgets or generous wellness offerings, Learnings budgets or generous learnings offerings; potentially it could be about working fewer hours (like, half a friday off) or diverting that to pure learning; strong community building with the workplace; and of course if it’s a startup, then equity’

Another poll option that got 5 votes was ‘Not enough remote positions/​the main hubs are too far away from me’, further corroborating the ‘flexibility’ concerns—though in retrospect the phrasing makes it unclear how many of those people would actually want remote work.

So inasmuch as perks are translatable to salary and actual EA software dev salaries are substantially higher than EAs working elsewhere seem to believe, salary may not be as much of a blocking factor in the EA world as the responses suggest.

On career capital

The respondents who clarified after voting for ‘career capital’ or ‘want to work in a larger tech department’ all mentioned limited learning opportunities, and in most cases corresponding CV concerns:

  • ‘the number of people I can learn from and the potential to have mentors that work in a similar function to you but not in your direct team’

  • ‘I think dev experience and rate of technical improvement, potentially with CV effects too.’

  • ‘a mix of not enough software dev experience and doesn’t look good on CV, I’d say at a 70⁄30 ratio … Working at a small non-profit signals to potential future employer that I haven’t been exposed to best practice, and other benefits usually associated with bigger, for-profit organisations (This concern was particularly significant at the beginning of my career, it’s decreasing with every additional year of experience)’

  • ‘My thoughts on career capital was that, for the EA aligned or eng jobs I had seen, the level of engineering ability of those orgs was far less than say that at the average good tech firm. Hence working at them would lead to substantially less engineering skill development and far less opportunity to tackle interesting technical challenges/​have good mentoring than I had received in the private sector.’ (via yes/​no answers to my Qs): ‘wouldn’t have got enough software dev experience’ and ‘working at an EA org just wouldn’t look that great on my CV’

  • ‘1. I wouldn’t have got good enough software dev (in my case Data Science + Product Management) experience. Also it’s slightly specific since my current role is quite rare to find (at least that was my experience) so it’s also the type of role (it’s called “Deployment Strategist”, which is somewhat of a solution architect, and it’s between PM and actually building the technical solutions). 2. working at an EA org just wouldn’t look as great on my CV as my current alternative. Given that I chose to go to a new unknown university (Minerva), it’s really helpful for me to have a good familiar organization as a stamp on my CV rather than an obscure startup or org.’

Interviews with current EA developers

I messaged various software developers currently working at EA orgs where I could find contact information for them, though this only led to three interviews and one informal conversation that I was unable to follow up on. Those three I asked various questions about their job satisfaction on a 1-7 rating scale.

Three of the four of them were very happy with their job (satisfaction 6). Many forms of selection bias could apply here; nonetheless, this is evidence against the idea that EA tech jobs _overall) are unrewarding. But three of the four raised concerns around their career development, in particular the lack of other developers to work with, and the one who didn’t described himself as having a much broader role: ‘something closer to a broad, founder-y position, where I get involved in all sorts of things, including web design, product strategy, writing, operations, management and hiring’.

(You can access this table as a PDF here)

Developer C also mentioned that their org had struggled to hire for a tech role, ‘since within EA in particular, there are lots of developers who have ‘stronger technical skills than we need, relative lack of interest in web development’. Taking this at face value, it seems like the technical work at the organisation was relatively unchallenging, but that their broader set of responsibilities made up for this for them. A similar generalism/​level of autonomy seems to have contributed to Developer A and B’s overall satisfaction.

That said, Developer C also mentioned ‘not enough breadth of interest /​ competence in non-technical areas’ as a difficulty in hiring for the role, so we can’t assume much about what proportion of EA developers would be motivated by these elements—they won’t matter much to skilled non-EA developers who might have considered the role. As Developer B noted, the experience from them is probably much less valuable if you ever expect to work at non-EA organisations.

So inasmuch as we can infer from N=3-and-a-bit, I found that tech jobs have higher overall job satisfaction than I’d expected, if the selection bias effect wasn’t too strong—but the concern that the jobs can counterfactually retard your technical skills seems well founded.

Summing up

Hiring now may not be as difficult as it was when Ben Todd and Jeff Kaufman posted their observations. One of the tech staff I interviewed above claimed ‘the landscape has changed dramatically in the last 4 years. All roles at EA orgs have become much more competitive.’

But the nature of EA tech work hasn’t particularly changed since then. If we believe it was—and therefore still is—bad for technical skill development, then the cost has just shifted from the organisations (paying in time and effort to recruit) to their tech staff (paying in counterfactual career capital).

The most rewarding EA tech jobs seem to go well beyond just ‘software development’. But not all EA tech jobs have this added responsibility, and it doesn’t address the problem raised by many would-be and actual EA developers of developing technical skills. As for shifting the cost to developers: drawing from EAs’ future career capital will have all kinds of harmful effects on the movement longer term, even if it’s ‘working’ at the moment.

EA tech work is inefficiently allocated

Small tech departments are inefficient when they have common values. I’ll explore this more rigorously in part 2, but it’s quite an intuitive problem. The rate at which tech work comes in constantly fluctuates and some projects are vastly more valuable than others. This means that at any given time, it’s likely that developers at some organisation are largely wasting their time on low value work while developers at another org—that might have very similar goals or at least similar terminal values—are overworked on some major project that needs to be released ASAP.

To attempt to quantify this effect, I’ve used a simplifying model which I’ll detail in the appendix to the next part, but the short version is that this inefficiency seems to lose each single-developer org value very roughly equal to half of the cost of employing the developer alternately in money wasted by overstaffing and output lost by understaffing. How much of this waste an agency could prevent depends on further assumptions—also laid out in part 2’s appendix.

There are also many more EA orgs with no in-house developer (there’s a wiki of EA orgs, roughly 90% of which don’t seem to have one). Additionally, sometimes someone outside an EA org—or entirely outside the EA space—will propose a high value one-off project and then take little or no further ownership, often meaning it doesn’t get done (although in one of those cases the proposer ended up doing it himself). We could consider such projects pico-orgs[6], since they’re amenable to the same logic as macro-orgs (by which I mean any 1+ person organisation).

Each <1-developer-org constitutes a potential value loss of whatever value its tech projects would have when implemented, minus the pro-rata cost of developer time it would take to implement them. In some cases this will be a low enough loss that we would prefer to counterfactually spend the money elsewhere; in others it will be high enough that we would gladly pay pro rata for an in-house developer if that were an option. Instead, we have to choose between not doing the projects and paying perhaps 3x the pro rata cost for a traditional software agency (see parts 2–3 for more on this comparison). In practice such orgs (especially pico-orgs) will often not apply for funding such an expense, even if it would still be worthwhile. Sometimes volunteers will step up, but reliable volunteers are in short supply (more in part 3 on the comparative value of volunteers).

The problem of fluctuating tech work could perhaps be mitigated by part time staff, but this only helps to the extent that an org can find a developer whose part time preferences closely match the org’s requirements, which will continue to fluctuate.

The problem is accentuated by the way tech work is split into many specialisations. Web development alone approximately consists of frontend, backend and devops work—and many more sub-specialisations. While there are many ‘full stack’ developers (ie generalists between the three) they inevitably have some preference, and inevitably would benefit from being able to consult with someone with different strengths.

A problem unique to EA?

As far as I know, this is not something EAs have thought much about, possibly because it’s much less of a concern in the for-profit space. A for-profit company only focuses on its own efficiency relative to its competitors’ - the absolute efficiency of its sector is otherwise irrelevant to it. Moreover a successful startup will gradually grow to the point where it can afford enough tech staff to substantially balance out fluctuating work, retaining some cross-department inefficiencies, but with better balanced workloads overall since reassignments causing much less friction than hiring and firing.

Conversely, the EA space as a whole cares about its absolute efficiency, not that of any given organisation relative to its ‘competitors’. And even the most successful EA nonprofits tend to stay relatively lean—I know of only a couple with even two full time software devs.

Meanwhile, EA for-profits will typically want to grow, but obviously start small, and still be concerned with the efficiency of their space. So until they reach capacity for a multi-person tech team, they have the same problems as nonprofits.

Summing up

The EA space has a very high proportion of small orgs, losing a lot of potential value.

EA tech work is difficult to assess hires for

To a first approximation judging a potential hire is harder the more senior they are relative to your current best employee in that field (presuming they’re doing the interviewing). This is somewhat less of a concern for tech, where there are relatively objective tests of some of the relevant skills; nonetheless, if you have 0 developers, which many EA orgs do when they’re hiring, it’s going to be difficult to make an informed decision.

When you have 1 developer it’s still going to be difficult 50ish% of the time when you’re trying to hire someone with more experience than them, and so on—tending towards 0% as your tech department grows.

Next: Part 2: The advantages of agencies

In which I explore how an EA tech agency could address these concerns.

  1. ↩︎

    Two terminology clarifications that apply to this whole sequence:

    Firstly, when I talk about EA organisations, I mean ‘nonprofit EA organisations’ unless otherwise specified. I’m cautiously optimistic that an agency of the type I discuss in subsequent posts could also help for-profit EA startups, with various caveats: 1) it might need a clear definition of what qualified as an EA for-profit, and for-profits would have an incentive to game this, 2) there may be legal restrictions on the support a non-profit could offer a for-profit, and 3) for-profits typically aim to grow much faster and therefore have a smaller window in which outsourcing would be better for them than in-house hiring.

    Secondly, I’m using ‘tech worker’ and ‘coder’ primarily as synonyms for ‘web development worker’ and closely adjacent roles, since those seems to be the main area of EA tech work other than highly specialised AI/​research work—the latter of which has very different considerations.

    I suspect that similar patterns as those I describe here apply to a lesser extent to somewhat adjacent areas like data science. In the longer run I’d hope an agency could include them, if only since it gives more breadth of experience to the team, and hence more room for lateral movement to those who want to try something new. But in the short term, I’m imagining a donor-funded tech agency would start by primarily addressing web development, hopefully prove the concept, and only afterwards work its way outwards, to the extent that doing so seemed advisable. Even short term there may be other roles needed, such as (tech-specialised) project managers, but since I’m agnostic on how much the overall argument would apply to them I’m leaving that deliberately open.

  2. ↩︎

    I’m using the term ‘agency’ throughout because it seems less presumptuous than ‘consultancy’, but in practice I would expect such an agency to eventually provide a shared tech service to the whole EA space, such that over time I would expect it to offer more than just coding-on-demand. A donor funded agency as described in part 3 would have to start doing this sooner, since its domain (as discussed in more detail there) would include finding projects that individual EA orgs might have missed. Plausibly ‘shared service organisation’ would be a better term, but a Google search yields at best a handwavey definition of what that actually means, and it won’t be familiar to most people, so I’ve stuck with the more evocative term.

    While I was writing this, Luke Muehlhauser published a general call for more EA consultancies. His argument is a high level one for a) the general case, rather than tech specifically, and b) not (as I’ll discuss in part 3) contrasting funding models, so our arguments don’t overlap much.

  3. ↩︎

    ‘Donor-funded’ could describe any charity or nonprofit, but in this sequence I’m imagining it implies funding almost exclusively from a small number of individual entities as opposed to sourced from large numbers of small donations. The distinction isn’t super-important—in principle, donor-funded tech agencies and individual workers could source funding from either, but in practice it seems far more likely that they’ll either be funded by major donors or not at all.

  4. ↩︎

    Eg comments on Ben’s thread: ‘I decided not to apply for the 80k job because it was WordPress, which is horrible to work with and bad for career capital as a developer’ (comment link), ‘I’m concerned the jobs won’t be technically challenging enough’ (comment link)- though a couple of comments challenge the idea that they’re the driving reason for hiring difficulty.

    Comment on Jeff’s thread: ’I took a significant pay cut to work as a software engineer for CEA, considering the difference to be essentially a donation … I treated it like a standard tech job with flexible hours, and found that I was judged pretty harshly for that by my co workers.

    ‘I was also under the impression that I’d be the manager of the team when I joined and would work on growing the team and mentoring new folks, but it turned out that I was placed under the single existing engineer who had less experience than I did, and after hiring and training one person, the team stayed stagnant and I did not get the opportunity to mentor and train that I’d come in excited to do.’

  5. ↩︎

    To my knowledge, this is the biggest FB tech group—it’s members seem to be largely a superset of the ‘Software, Data, and Tech Effective Altruism’ group.

    I do worry that the phrasing was leading—at the very least, I specifically asked about ‘reservations’ rather than ‘overall view’. I think the number of respondents who picked ‘I enjoy my current job too much to leave but would want to join an EA org if I left’ (4 of 24 total participants) provides some calibration here, but this is a potential source of bias.

  6. ↩︎

    Definitely a silly term, but Googling ‘micro-organisation’ yields definitions of ‘a small business that employs few people’ so would be confusing; a ‘nano-organisation’ might then be one that employs one person and might also confuse with nanotech organisations, which show up when you Google the phrase; a pico-organisation has no such associations, no meaningful Google hits and, following down the SI prefix chain by order if not cardinality would be the first prefix described an ‘organisation’ of less than one person.

Arepo25 Jul 2021 13:03 UTC
59 points
13 comments14 min readEA link