EA needs consultancies

lukeprog28 Jun 2021 15:18 UTC
240 points
88 comments8 min readEA link

Problem

EA organizations like Open Phil and CEA could do a lot more if we had access to more analysis and more talent, but for several reasons we can’t bring on enough new staff to meet these needs ourselves, e.g. because our needs change over time, so we can’t make a commitment that there’s much future work of a particular sort to be done within our organizations.[1] This also contributes to there being far more talented EAs who want to do EA-motivated work than there are open roles at EA organizations.[2]

A partial solution?

In the public and private sectors, one common solution to this problem is consultancies. They can be think tanks like the National Academies or RAND,[3] government contractors like Booz Allen or General Dynamics, generalist consulting firms like McKinsey or Deloitte, niche consultancies like The Asia Group or Putnam Associates, or other types of service providers such as UARCs or FFRDCs.

At the request of their clients, these consultancies (1) produce decision-relevant analyses, (2) run projects (including building new things), (3) provide ongoing services, and (4) temporarily “loan” their staff to their clients to help with a specific project, provide temporary surge capacity, provide specialized expertise that it doesn’t make sense for the client to hire themselves, or fill the ranks of a new administration.[4] (For brevity, I’ll call these “analyses,” “projects,” “ongoing services,” and “talent loans,” and I’ll refer to them collectively as “services.”)

This system works because even though demand for these services can fluctuate rapidly at each individual client, in aggregate across many clients there is a steady demand for the consultancies’ many full-time employees, and there is plenty of useful but less time-sensitive work for them to do between client requests.

Current state of EA consultancies

Some of these services don’t require EA talent, and can thus be provided for EA organizations by non-EA firms, e.g. perhaps accounting firms. But what about analyses and services that require EA talent, e.g. because they benefit from lots of context about the EA community, or because they benefit from habits of reasoning and moral intuitions that are far more common in the EA community than elsewhere?[5]

Rethink Priorities (RP) has demonstrated one consultancy model: producing useful analyses specifically requested by EA organizations like Open Philanthropy across a wide range of topics.[6] If their current typical level of analysis quality can be maintained, I would like to see RP scale as quickly as they can. I would also like to see other EAs experiment with this model.[7]

BERI offers another consultancy model, providing services that are difficult or inefficient for clients to handle themselves through other channels (e.g. university administration channels).

There may be a few other examples, but I think not many.[8]

Current demand for these services

All four models require sufficient EA client demand to be sustainable. Fortunately, my guess is that demand for ≥RP-quality analysis from Open Phil alone (but also from a few other EA organizations I spoke to) will outstrip supply for the foreseeable future, even if RP scales as quickly as they can and several RP clones capable of ≥RP-quality analysis are launched in the next couple years.[9] So, I think more EAs should try to launch RP-style “analysis” consultancies now.

However, for EAs to get the other three consultancy models off the ground, they probably need clearer evidence of sufficiently large and steady aggregate demand for those models from EA organizations. At least at first, this probably means that these models will work best for services that demand relatively “generalist” talent, perhaps corresponding roughly to the “generalist researchers” category, plus some of the “operations” category, in this survey of EA organizational needs.[10] Ongoing services may be a partial exception because in that category, demand from each interested client is relatively stable over time,[11] so one might only need demand from 2-3 EA organizations to justify a full-time role providing that service at an EA consultancy.

Below, I comment on the current demand for each of these three models of EA consultancy. Based on polling other Open Phil staff, I think there is substantial demand for all four types of services from Open Phil alone, but I know less about demand from other EA organizations.[12]

Projects

For example, I wish there was an EA consultancy I could pay to do the market research on how much EA organization demand there is for each of these types of services. :)

Here’s an initial brainstorm of project types for which there might be substantial ongoing demand from EA organizations, perhaps enough for them to be provided by one or more EA consultancies:

  • Impact assessment, e.g. trying to estimate the counterfactual impact of a grant made or project run a few years ago, by interviewing 5-20 people, gathering relevant facts, and putting some numbers on the magnitude of relevant changes in outcomes variables and counterfactual credit to different actors.

  • EA event organization and management

  • Statistics /​ data science assistance

  • Web development projects for which EA context and habits are helpful, e.g. for new EA discussion platforms, forecasting/​calibration software, or interactive visualizations of core EA ideas.

  • Polling, survey research, and online experiments (e.g. via Positly+GuidedTrack) on EA-relevant questions[13]

  • Marketing pushes for EA things, i.e. figuring out which marketing tools best fit the thing to be promoted and the intended audience, and then executing

  • Run an EA-related RFP, filter the responses, summarize the strongest submissions for the client to consider funding

  • Policy development and advocacy

  • Run a fellowship program, filter the responses, summarize the strongest candidates for the client to consider funding, find and manage the training resources and connection opportunities for the fellows

  • Run a training program for staff /​ contractors /​ grantees /​ collaborators, a la a superforecasting workshop but with EA-specific content, and perhaps extending longer than one day

  • Design and run a prize program

  • Design, test, and iterate a training program, a MOOC, an undergraduate course, a summer school program, or other educational materials

  • In general, pilot projects for ideas that, if successful, could perhaps become an ongoing program/​organization

  • Do 80% of the work for a recruitment round for a full-time role at the client organization

  • Help communicate some research (that perhaps can’t itself be done by consultants) to non-specialist audiences such as policymakers or the general public

  • Other ideas?

I’m not sure how much overall demand there is for such projects to be run by EA consultancies,[14] but there is substantial demand for some of them at Open Phil alone (see footnote[15]).

Ongoing services

Again, an initial brainstorm:

  • Initial vetting stages of job applicants

  • On-demand EA life/​career coaching[16]

  • On-demand EA-aware mental health services[17]

  • EA-aware legal services and HR services[18]

  • Some kinds of content writing

  • Writing support (feedback, copyediting, design)

  • Donor services

  • A community fund /​ DAF provider

  • Fiscal sponsorship for new projects without their own incorporation (yet)

  • Other ideas?

Here again, I don’t have a good sense of how much overall demand there is for such ongoing services from EA consultancies,[19] but there is some demand from Open Phil alone.

Talent loans

I frequently think something like “If I could hire an analytically strong EA to work with me for 2 months on X, I would do it, but I can’t hire anyone with that skill level for just 2 months, and also vetting hundreds of applicants and interviewing ~10 of them just to enable 2 months of work wouldn’t be worth it.”

But if McKinsey-style EA consultancies existed and had a track record for hiring conscientious, analytically strong people, then I could effectively hire such EAs for 2 months at a time (via a contract with the consultancy), with the consultancy already having done >90% of the necessary vetting and training.[20]

Talent loans would often serve a similar purpose as outsourced analyses or projects, and I’d need more experience with all three to have a good sense of when I prefer a talent loan to an outsourced analysis or project. However, my initial guess is that I personally might have a need for two 2-4mo EA talent loans per year on average.

I’m not sure how much demand there is for this from others at Open Phil or other EA organizations.[21] Rethink Priorities has made a small number of talent loans before, to Longview and Effective Giving.

Thoughts on offering these services

There are various books, courses, etc. on how to start and run a successful consulting business. I don’t know how good they are, or how relevant their advice is to EA consultancies, but they might be worth a glance.[22]

Probably any single consultancy should provide only one or a few of the services above, not all of them.

If you want to offer some of these services yourself, you could do a bit of market research on how much demand there is for the specific service(s) you think you can provide, and then start pitching potential clients to contract you for an initial chunk of work. Here are some potential obstacles and ways to address them:

  • In this post I’ve mostly been thinking about the need for somewhat-established many-person consultancies, which can develop reputations for good client service and good selection and management of individual consultants. Individual freelance consultants can also be helpful, but they can be less convenient for clients, because then the client needs to put more work into vetting and managing the work of each individual consultant, instead of relying on an external firm for that. To overcome this problem you could try to get a job at an existing EA consultancy like Rethink Priorities, though there are very few such positions today.

  • Some EA organizations may not have the budget to experiment with external consultants. But, you could encourage them to include some funding for EA consultant experiments in their next grant proposal.

  • Your potential clients probably don’t have much time to try things out with an “unproven” consultant. To overcome this, you could complete some example work of the sort you’d like to provide to clients, make it extremely “legible” to prospective clients (i.e. fast and easy to evaluate for quality and plausible usefulness), and then send it to potential clients. E.g. the reason I gave Rethink Priorities a grant to do more work on moral weight is that Jason Schukraft had previously written several reports on moral weight that I found helpful,[23] and I think he knew Open Phil might find that specific kind of work helpful because it followed very directly from the “open questions” listed in my moral patienthood report, and pursued those questions from a similar perspective/​framework.

  • You might not have as good a picture of the client’s needs as you think you do. There are lots of very subtle things that can make even high-quality work essentially unusable by the client. The best way to address this is to get a call with the potential client and ask them questions to understand in detail what they need and why, but it might be hard to get their time unless you’ve already done some work that is “close enough” to being useful to the client that they can recognize that the call might be worth the time.

  • You might not be as good at providing the service as you think you are. If you’ve addressed the challenges above and you’re still not getting any paid consulting work, that might be an indicator that your potential clients don’t think you’re as good a fit for providing those services as you think you are, in which case you should consider moving on and doing something else with your time and energy. Or perhaps get more experience (e.g. at one of the large generalist non-EA consulting firms) and then try again.

Some of these services could perhaps be offered not by new organizations, but by existing organizations deciding to offer particular services alongside their other work. For example Rethink Priorities could expand the range of services it offers, or 80,000 Hours could offer on-demand career coaching while continuing its other work.

Some additional notes of caution

The consultancies model looks promising to me given what I’ve seen in other industries and the constraints I’ve observed when Open Phil considers or tries to hire more staff. That said, I don’t want to oversell it. In addition to the list of challenges in the previous section, I should say:

  • One obvious failure mode is that EA consultancies, like many non-EA consultancies, might simply cost a lot but provide little value beyond generic advice, sharp-looking slide decks, and a façade of external justification for something a manager had been planning to do anyway. If this happens then I’d like to think EA client organizations would simply stop commissioning those services.

  • In general, it can be difficult for consultants to understand the goals and heuristics of their clients in enough detail to know how to “hit the mark,” without all the context that one can acquire as a full-time employee of that client. Perhaps especially in EA, even things that seem like minor details and debatable judgment calls can make the ultimate product effectively useless from the client’s perspective. This might be a fundamental problem that limits the utility of the consulting model, at least within EA, to a pretty small set of services.

  • Should a talented EA provide services via a consultancy, or do more entrepreneurial work that isn’t specifically requested by EA clients, or do something else? It’s debatable which of these will be more impactful. My guess is that experimentation, personal fit, and career capital development should play major roles when choosing between these options.

  • I haven’t spent as much time thinking through possible objections and reasons for skepticism about the advice in this post as I sometimes do, for time reasons. I hope that the community will discuss the pros and cons of my advice here in more detail in the comments.

Acknowledgements: I got helpful feedback from several people in the EA community on earlier drafts of this post but unfortunately forgot to ask permission to name them here, except for some people I name and quote or paraphrase in specific footnotes.

Notes

  1. ↩︎

    Also, there can be large costs to hiring someone who turns out to not be a strong fit.

  2. ↩︎

    See e.g. Reflections on Our 2018 Generalist Research Analyst Recruiting and After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation.

  3. ↩︎

    Some think tanks do lots of work that is specifically commissioned by clients (the consultancy model), but more often they produce outputs that weren’t specifically requested (the entrepreneur model), including work that is aimed at affecting the behavior of specific actors in a specific way (e.g. GPI’s work on “patient philanthropy”). EA needs both; this post is focused on the need for consultancies. EA has many “entrepreneurship” organizations, including several funded by Open Phil, and we have benefited from their work. Within longtermism (which I know best), I think of e.g. FHI, MIRI, 80,000 Hours, and GPI.

  4. ↩︎

    Working on such projects for a client, or especially being “loaned” to a client for a time, provides both the consultant and the client a strong opportunity to evaluate each other for fit w.r.t. a full-time role with the client, but in a “safe” context in which there is no default expectation of a full-time offer from the client, and the consultant’s job security with the consultancy remains intact.

  5. ↩︎

    E.g. reasoning that is calibrated, reasoning-transparent, rigorous but willing to draw from any genre of evidence, focused on maximal counterfactual impact, weirdness-tolerant, and impartial (in the moral sense), all at the same time.

  6. ↩︎

    Rethink Priorities has done commissioned work on animal consciousness, animal welfare interventions, lead exposure, charter cities, and agricultural land redistribution (commissioned by Open Phil), and on the EA community itself (commissioned by Center for Effective Altruism and 80,000 Hours). Open Phil has found the work we commissioned to be of sufficiently high quality to be useful to us, though I can’t speak to the quality of their other work. They have also produced work specifically requested by (and in some cases paid for by) The Humane League, Farmed Animal Funders, Mercy for Animals, Animal Equality, and Wild Animal Initiative, but I’m not familiar with that work. And between client-requested projects, they have produced a variety of non-requested analyses that seem generally useful to the EA community.

  7. ↩︎

    Why not just use RFPs? I’m more optimistic about the consultancy model because it can more often leverage an existing relationship with an existing organization that is known to have hit some quality threshold for similar-ish projects in the past. In contrast, with RFPs the funder often need to build a new relationship for every funded project, has much less context on each grantee on average, and grantees are less accountable for performance because they have a lower expectation for future funding from that funder compared to a consultancy that is more fundamentally premised on repeat business with particular clients.

  8. ↩︎

    Some EA organizations provide significant services to the EA community, in part due to expressions of interest from other EA organizations, e.g. CEA’s work on community health and community discussion platforms. But that is different from more narrowly scoped services being delivered in a particular way for a particular time period under contract with a specific client. One organization (besides Rethink Priorities) that might qualify as an EA consultancy as I use the term here is The Good Growth Co, but I don’t know much about them yet. Two other possible exceptions are Longview and Founders Pledge, which provide donor services to some people who are perhaps “EA-curious,” though they don’t charge their clients for their services. CLTR may be another example.

  9. ↩︎

    If this was feasible to do while maintaining quality, I’d probably want to commission enough ongoing analysis from RP on AI governance research questions alone to sustain >10 FTEs there. (A group like GovAI doesn’t fill this need because they generally don’t do projects requested by clients, and they typically want to produce work optimized for academic publication rather than for informing action at EA organizations.) To be clear, I don’t think it would be ideal for a consultancy such as RP to have just one client commissioning >90% of its work; that would seem to restore some of the dynamics that the consultancy model is meant to avoid.

  10. ↩︎

    Generalist researchers and operations were the two categories of talent most commonly demanded by the surveyed organizations.

  11. ↩︎

    Almost by definition, that is.

  12. ↩︎

    I comment elsewhere throughout this post and its footnotes about Open Phil demand for specific services. Beyond that, I got several comments from other Open Phil staffers about demand across all four types of services, along the lines of “I’m not sure, but I think our demand for these things is kind of a lot” or “I frequently want one or more of these things.” Likewise, Seán Ó hÉigeartaigh of CSER and CFI said he expects there would be significant demand for some of these services at CSER/​CFI, and that some others at CSER/​CFI reported the same opinion.

  13. ↩︎

    Rethink Priorities has done a fair bit of this before, for Open Philanthropy, Center for Effective Altruism, Forethought Foundation, The Humane League, Mercy for Animals, Animal Equality, and the Humane Society for the United States.

  14. ↩︎

    Max Dalton of CEA told me he thinks CEA has some demand for such project services.

  15. ↩︎

    Examples for which other Open Phil staff members told me they plausibly or probably could have made (or could make) good use of relatively generalist EA consultants for short-term projects include: (1) a recent ballot measure project, (2) surge capacity for work test grading for recruiting rounds, (3) figuring out what it would take for Open Phil to have a significant campus recruiting presence, and various other things.

  16. ↩︎

    This is currently available from Lynette Bye and Daniel Kestenholz. 80,000 Hours offers career coaching but not “on demand.”

  17. ↩︎

    I’ve heard that this is available from Ewelina Tur and Damon Sasi and perhaps others, but I don’t know anything beyond that.

  18. ↩︎

    For example, Open Phil wants legal advice that attempts to quantify the risks of different options rather than giving advice consistent with minimizing all risk, and willingness to quantify risk is common among EAs and rare among non-EA lawyers. (In part for this reason, we recently hired EA lawyer Molly Kovite as our in-house counsel.) Likewise, it would be helpful to have EA-familiar HR consultants that could better understand issues raised by our EA staff members and understand certain things in our work culture that are common in EA and less common elsewhere. I’m not sure that EA-friendly legal or HR services need to be their own firms, though; perhaps they could be provided by a handful of EA-friendly or EA-aware lawyers and HR experts at one or more larger firms that EA organizations can hire whenever the legal advice or HR services they need would especially benefit from EA context, a willingness to quantify the risks of different options, etc.

  19. ↩︎

    Max Dalton of CEA told me he thinks CEA has some demand for such ongoing services.

  20. ↩︎

    I would still need to vet specific consultants for specific projects to some degree, but I would know they were already selected for strong performance on analytically demanding work tests and (typically) prior projects, as well as being filtered for general conscientiousness, communication clarity, ease of interaction, etc.

  21. ↩︎

    Max Dalton of CEA told me he thinks CEA has some demand for such talent loans, especially for research.

  22. ↩︎

    Some example books that turned up in a quick search include Getting Started in Consulting, Consulting Success, An Insider’s Guide to Building a Successful Consulting Practice, The Boutique, and Coaching and Consulting Made Easy.

  23. ↩︎

    In this case, Open Phil funded Jason’s initial reports on moral weight, though we didn’t commission them — instead, some portion of an earlier grant included funding for projects chosen by Rethink Priorities, and they chose to use some of that funding to write reports on moral weight.

lukeprog28 Jun 2021 15:18 UTC
240 points
88 comments8 min readEA link