Founded the Effective Altruism Consulting Network (EACN) and cFactual.
Worked previously at various consulting firms (incl. 3 years at BCG) and as a research assistant. I served on the boards of EA GER and EA AT. Find here more information about my experience and consulting projects I did within the EA space.
In my free time, I like to get nerdy about climbing as well as improving personal and organizational decision making among other things.
Some shameless self-promotion as this might be relevant to some readers: I work at cFactual, a new EA strategy consultancy, where one of our three initial services is to optimize ToC’s and KPI’s together with organizations. Illustrative project experience includes the evaluation of the ToC and design of a KPI for GovAI’s fellowship program, building a quantitative impact and cost-effectiveness model for a global health NGO, internally benchmarking the impact potential of two competing programs of an EA meta organization with each other, doing coaching with a co-founder of a successful longtermist org around Fermi-estimates and prioritization of activities as well as redteaming the impact evaluation of a program of a large EA organization.
Thanks for creating this post! Sharing some thoughts on the topic based on my experience creating and redteaming theories of change (ToCs) with various EA orgs (partly echoing your observations and partly adding new points; Two concrete project examples can be found here and here).
Neglectedness of ToC work (basically echoing your claim). Due to the non-pressing nature and required senior input, ToC/strategy work seems to get deprioritized very often, e.g., I have been deprioritizing updating our ToC for three months due to more pressing work. I think the optimal time spent thinking about your ToC/prioritization/strategy depends on the maturity of your project and is hard to get right, but based on my experience, most of us spend too little time on it (just like we tend to spend too little time exploring our career options as it is worth to invest 800 hours in our career planning when we can increase its impact by 1% in expectation). Assuming your org has ten staff, which work 200 days a year for 8 hours per day, you would want to invest 16k hours in figuring out how staff spends their time best if it is likely to result in a 1% impact increase
More than one ToC. I think most orgs should have a ToC on the org, team and individual level as well as for each main program/activity. It seems not optimal to work on something without even having thought through how this will change the world for at least 3min (and if there are alternatives, how you could achieve the same with less work)
Different levels of granularity. Depending on the purpose and the context of your ToC, you can have a three-row ToC (e.g., for small projects you are exploring), a flow-chart (e.g., to communicate the ToC of your org clearly, see examples in this post) and/or an exhaustive document showing lots of reasoning transparency, alternatives you considered etc. (e.g., to lay out the ToC of your research agenda)
Developing a ToC. One simplified approach to develop a ToC on an org level which worked well with some clients but always needs tailoring looks very roughly like this: (1) Map out all potential sources of value of today (and potentially in the future), (2) Prioritize them, (3) Create a flow chart for the most promising source of value (potentially include other sources of value or create several flow charts), (4) Think through the flow of impact/value end to end as a sanity check, (5) collect data (e.g., talk to experts, run small experiments) to reduce uncertainties, (6) re-iterate. See more here
Strategic implications and influencing decisions (previously mentioned but I think this is a point many people found useful and is not stressed enough in typical ToC literature). Your ToC should inform key decisions and ultimately how you allocate resources (your staff’s time or money). I never experienced that we were certain about all the sources of value and the causal relationships between each step or did not have a hypothesis on how to have even more impact with an adapted or new program. So the ToC work always had at least some implications for allocating time. One great example is the Fish Welfare Initiative, which included resolving uncertainties around their ToC in their yearly priorities (See slides 13 and 24)
Areas for improvement. (1) Trying to map every casual pathway and not focusing on the most important ones, (2) Deferring too much to others on what they perceive as valuable, e.g., the target group, and not doing enough first principles/independent thinking and data collection, (3) Not considering counterfactuals at all and/or not considering that there are likely several counterfactual worlds, and (4) Not laying out key assumptions and uncertainties (previously mentioned in the post but seems valuable to highlight that this also reflects my experience) among other things
Note that I likely have a significant sample bias, as organizations are unlikely to reach out to me if they have enough time to think through their ToC. Additionally, please read this as “random thoughts which came to Jona’s mind when reading the article” and not as “these are the X main things EA orgs get wrong based on a careful analysis”. I expect to update my views as I learn more
Thanks for sharing and your great work during the last year. Having talked to you several times, I was and am impressed with your systematic approach to finding product-market fit/high expected impact opportunities, your ability to build MVPs to test ideas quickly, and your courage to discontinue programs that do not meet your bar.
I think the latter is hard especially after investing weeks of work into programs and it is easy to trick oneself into motivated reasoning, about why it might be worth continuing the program. I admire you for having the courage to make tough judgment calls. Probably most of us should stop mediocre activities (earlier)
What do you specifically mean by “maturing in management, generally”? I noticed that people tend to have very different things in mind when they are talking about “Improving management in EA” so could be worth clarifying
Hmm. Obviously, career advice depends a lot on the individual and the specific context, all things equal, I tentatively agree that there is some value in having seen a large “functioning” org. I think many of these orgs have also dysfunctional aspects (e.g., I think most orgs are struggling with sexual harassment and concentration of formal and informal power) and that working at normal orgs has quite high opportunity costs. I also think that many of my former employers were net negative for some silly which I think are highly relevant, e.g., high-quality decision making
Love the idea of a having call and a pilot project (if this is what is most useful). We might even explore the options for pro bono work in the EACN as I know that some partners in BCG are looking for strong partnerships in their regions. I imagine that might also be the case for McKinsey, Accenture, Bain, … .
I also agree that almost all consultancies already do EA-aligned work (not to the extent, we would like them to of course) and have expertise in many relevant fields. E.g., my last project was to do an impact assessment (incl. counterfactual impact etc.) of a 300+M€ government grant, which addressed an EA cause area. At Accenture, BCG and Capgemini members of the EACN are actively reaching out to partners to push EA relevant topics even more. So we have a broad network of contact persons within the EACN and the different firms we could reach out to depending on the needs.
Thanks for highlighting this offer again and sharing your feelings, Catherine!
I like how you highlight that the forum is just one element of EA. Personally, I also distinguish quite strongly between EA as a question and set of evolving ideas and the EA community (which is obviously a part of EA).
Historically, I found it super valuable to talk with you through various sensitive community-building considerations and benefited a lot from your experience managing countless tricky situations I wasn’t even aware of. Thanks for doing that important and hard behind-the-scenes work!
Thank you so much! If you are unsure whether you should apply for consulting, you can also reach out to Jan-Willem van Putten, who does a fantastic job in giving insights to students on behalf of the EACN
Registered. It also seems valuable to talk to impact-driven people who seriously considered quitting but then decided to finish their PhD as (a) it is not obvious to me that quitting is always the right choice and (b) it might be useful to know common reasons why people decided to continue working on their PhD.
Thanks for creating this post! +1 to the general notion incl. the uncertainties around if it is always the most impactful use of time. On a similar note, after working with 10+ EA organizations on theories of change, strategies and impact measurement, I was surprised that there is even more room for more prioritization of highest leverage activities across the organization (e.g., based on results of decision-relevant impact analysis). For example, at cFactual, I don’t think we have nailed how we allocate our time. We should probably deprioritize even more activities, double down even more aggressively on the most impactful ones and spend more time exploring new impact growth areas which could outperform existing ones.
Thanks for the question, Merlin. Please note that we have a small sample size and are still refining our models of what skillsets are most relevant for more EA-aligned consulting.
Three things that have been useful: 1) Structuring problems, projects and meetings well, 2) Being able to switch quickly between different levels of abstraction quickly and constantly: Thinking carefully about a key assumption in an excel model in one moment and thinking about how the results change the big picture for a CEO in the next moment 3) More vaguely: Just having seen and worked with a lot of organization, projects and leaders, probably shaped our intuitions
Three things that were less useful than I thought: 1) Executional speed—On the margin, we care much more about what we work on and getting it right compared to getting things done; 2) Qualitative data selection e.g., interviewing—during the pilot project we did a lot more independent thinking and then specific testing of key uncertainties with key stakeholders. In traditional consulting, we would have conducted interviews earlier to develop our hypotheses (which has the cost of becoming potentially an echo chamber); 3) Project planning—during all pilot projects we adapted our activities quite a lot based on our updated models, what would be most impactful
Thanks for clarifying! I think Training for Good looked into “scalable management trainings”, but had a hard time identifying a common theme, which they could work on (This is my understanding based on a few informal chats. This might be outdated and I am sure they have a more nuanced take). Based on my experience, different managers seem to have quite different struggles which change over time and good coaching and peer support seemed to be the most time-effective interventions for the managers (This is based on me chatting occasionally to people and not based on proper research or deep thinking about the topic)
Thanks for sharing, Catherine! I apply many of your tips and agree that they are super useful. Additional questions I ask myself quite often:
What is the goal I want to achieve? This is the question which helps me to structure my thinking and approach the most
Am I asking the right question? Next to regularly not thinking through all the options I have, I also realize often that I am not asking the question I really care about in the first place
Can I make a prediction about my decision? This helps me a lot to keep track of my decisions e.g., at cFactual we have a “prediction of the week” to calibrate ourselves on outcomes we expect to see, identify differences in reasoning about important topics among team members, …
Do I weigh all arguments/considerations equally or do I believe one argument is 10x more relevant than others?
Some tools for group decision-making we use:
Our google doc company template has as default drop-downs to always indicate the status of the document, time spent, what the stage of the document is (Strawmen, key arguments or flashed out document) and a section for a few words on epistemic status (“braindump/ 5min of desk research/ I am an expert/...”). This helped us a lot to remain focused and have higher quality discussions with a time investment of 20sec
We try to quantify our preferences, e.g., Instead of saying: I am in favour of option A, we aim to write: I am 55% in favour of A. This helps us quite often to make a judgement call without forward and backward writing of comments
If there are larger decisions I want to think through more rigorously, I quite often use this mental structure as a starting point (and then adapt it): Recommendation/conclusion incl. my certainty in the conclusion, alternative options, my arguments for the recommendation, my arguments against, key uncertainties, key assumptions, downside risks and predictions
Probably stating the obvious for many here: I think the CFAR handbook also has great prompts for people who are interested in the topic
+1 . Here are some more considerations, which can help to develop a theory of change for a workplace/professional group. And here are three examples for different workplace/professional groups:
the EA finance network (probably higher focus on donations compared to other groups),
the UK civil service and policy network (probably higher focus on helping people advance within their career/influencing institutions compared to other groups) and
the Effective Altruism and Consulting network (probably higher focus on career transitions compared to other groups)
Would add to the list:
get funded from the EA infrastructure fund to figure out what your high impact career path is
work in grant making
Organizations I talked to, which seem to be especially excited about hiring former consultants (and also have many) include OpenPhil, Charity Entrepreneurship and the Centre for Effective Altruism (not speaking on their behalf though)
Quick update: we launched an EA-aligned strategy consultancy, partly motivated by this post and the feedback we received from our pilot projects: https://forum.effectivealtruism.org/posts/a65aZvDAcPTkkjWHT/introducing-cfactual-a-new-ea-aligned-consultancy-1
Posting as an individual who is a consultant, not on behalf of my employer
Thanks for the great post and the insightful comments! Building on your thoughts some additional comments from a consultants perspective (Worked two years at BCG on 10+ projects in the public, private and social sector; founded the Effective Altruism and Consulting Network; was Vice-Pres. for EA Austria):
On the need for consulting services: Generally speaking, I agree that consultancy can in specific circumstances (e.g., clearly defined objective, no expertise or free resources inhouse, enough capacity on the client-side to provide input and guidance, …) unlock value for the EA community and we already got several requests from different EA orgs via the effective altruism and consulting network (e.g., on calculating a business case, researching some data, assessing the feasibility of some planned projects). I also believe that investments in consultancies can be a waste of money if the project isn’t clearly defined. There is also a variance of quality in the market (the same general principle as with every purchasing decision apply: you need to find the right fit for your needs, the more money you spent the higher the likelihood of high-quality output, …). Additional efficiency gains for the EA community (esp. for local groups) might lay in setting up a shared service centre for EA orgs . Shared service centres offer standardized services (some of them mentioned above) such as support with marketing/layouts, HR services, reviews/translations, … . The difference to consultancy services is that a shared service centre covers much more the routine tasks every org has to deal with and is meant to be long-term support (vs. short-term/project engagements of consultants).
On the need for an EA-specific consultancy: While I generally believe that there might be specific services, where it is useful to have EA experience and build expertise over the years (e.g., EA trainings, the great work RP is doing), I would generally argue that many of the tasks listed above can be done by non-EA consultancies as
many of them need little to none EA specific input (e.g., the technical side of web development)
customization is needed for almost every client and a lot of the required input can be provided by the EA org/client with little time invest (see also the comment of JeremyR; e.g., for standard HR processes every org has)
you can benefit from years of experience from different industries normal consultants bring to the table
you as the client are in the driver seat of structuring the project so you get out of it, what you want (see also next bullet)
On the value of consulting services and the role the client plays: Additionally on the points mentioned above on the mixed quality of providers and projects, I would also argue that most people underestimate the role the clients play in delivering a successful project. This starts by assessing whether a consulting engagement is the right solution for the problem, deciding how much money you spent and choosing the consultancy/team. Clients have immense power to tailor the team and project set-up as well as deliverables based on their needs. Examples of good practices I have seen were partnering client and consulting team members to make sure knowledge transfer takes place in both directions as well as the client asking for ways how we could be wrong.
One way to drive this further would be to (1) assess and structure the different (future) needs and concerns etc., (2) identify relevant segments for external support, (3) define the best delivery model for the prioritized needs (inhouse, shared service centre, EA consultancy, freelancer, normal consultancy)