Bad Actors are not the Main Issue in EA Governance
Background
While I have a technical background, my career has been spent working with corporate boards and management teams. I have seen first-hand how critical leadership are to the success of organizations. Organizations filled with competent people can fail miserably if individuals do not have the right interpersonal skills and humility.
I have worried about governance within EA for a while. In October, I launched the EA Good Governance Project and wrote that “We have not yet experienced a scandal / major problem and have not yet started to think through how to avoid that happening again” . Now, 4 months later, we’ve had our fair share and people are open to change.
This post is my attempt to put some thoughts together. It has been written in a rather rushed way given recent news, so apologies if some parts are poorly worded.
Introduction
I have structured my thoughts in 4 sections, corresponding to the 4 key ways in which leadership can fail:
1) Bad actor
2) Well-intentioned people with low competence
3) Well-intentioned high-competence people with collective blind spots
4) Right group of people, bad practices
Bad Actors
Much discussion on the forum in recent months has focused on the concept of a bad actor. I think we are focusing far too much on this concept.
The term comes from computer science where hackers are prevalent. However, real life is rarely this black and white. Never attribute to malice that which is adequately explained by incompetence (Hanlon’s razor).
The bad actor concept can be used, consciously or unconsciously, to justify recruiting board members from within your clique. Many EA boards comprise groups of friends who know each other socially. This limits the competence and diversity of the board. Typically the people you know well are exactly the worst people to provide different perspectives and hold you to account. If they are your friends, you have these perspectives and this accountability already and you can prevent bad actors through referencing, donation history and background checks.
Key takeaway: Break the clique
Competence
There’s an old adage: How do you know if someone is not very good at Excel? They will say they are an expert. With Excel, the more you know, the more aware you are of what you don’t know. I think leadership is similar. When I had 3-5 years of professional experience, I thought I could lead anything. Now I know better.
Some aspects of leaderships come naturally to people, but many have to be learned by close interaction with role models. When you are a community without experienced figures at the top, this is hard. We should not expect people with less than 10 years of professional experience to be a fully rounded leader. Equally, it’s possible to be successful without being a good leader.
I think many of us in the community have historically held EA leaders on a pedestal. They were typically appointed because of their expertise in a particular field. Some of the brightest people I’ve ever met are within the EA community. However, we then assumed they had excellent people judgment, a sound understanding of conflicts of interest, in-depth knowledge of real estate investments and an appreciation for power dynamics in sexual relationships. It turns out some of them don’t. This shouldn’t come as a big surprise. It doesn’t mean they can’t be a valuable contributor to the community and it certainly doesn’t make them bad actors.
Key takeaway: We need to elevate the importance of soft skills and learn from models of effective leadership in other communities and organizations
Blind Spots
The more worrying thing though is how those people also believed in their own abilities. In my career, I have met a large number of company CEOs. I truly believe that the difference between a poor CEO and a good CEO is not their own expertise but their ability to harness other people’s expertise. This means knowing who to trust on a particular issue and how to get the right decision out of them. We all have blind spots and we all benefit from decision-making that draws on the right expertise at the right moment. Being open to external expertise is good epistemics. Let’s not recreate the errors of others simply because we believe we are superior.
We naturally like and find it easier to work with people who are similar to ourselves. It makes life simpler: decision-making is fast, cognitive dissonance is less, and we get our own way more often.
Key takeaway: Recognizing one’s own faults is one of the hardest skills to develop
Bad Practices
So how do good leaders get decisions right? They hire people who are different to them. They create the conditions for that person to challenge their views, e.g. by investing time to understand opposing viewpoints, by talking last in discussions if they risk being too dominant. They know when to make a decision
What does this mean for CEOs? CEOs should bring important and difficult questions to the Board for input. They should empower the board by giving them the information and context they require. They should create the right incentives for employees to voice their honest opinions. They should provide the structure for getting decisions made. And they should lead by example.
What does this mean for Board? Boards should understand that they are accountable only to themselves. They should avoid execution (there’s a lot of evidence that companies with a single Chairman-CEO underperform). They should talk less during discussions where they are not independent (e.g. they came up with the proposal). They should critically evaluate their own performance and composition.
Key takeaway: Think about how your every interaction promotes cultures for effective decision-making.
Personal Note
I spent a few years earning to give before coming across EA. During that time, I attempted and mostly failed to answer questions about the most effective ways to make the world better. EA provides people with a way to answer those questions and funnel their energy into it. I sincerely believe that EA has had and will have an enormous positive impact. Please don’t give up on EA.
- Some governance research ideas to prevent malevolent control over AGI and why this might matter a hell of a lot by (23 May 2023 13:07 UTC; 63 points)
- EA & LW Forum Weekly Summary (20th − 26th Feb 2023) by (27 Feb 2023 3:46 UTC; 29 points)
- Posts we recommend from last week (Digest #125) by (24 Feb 2023 3:49 UTC; 22 points)
- 's comment on More Centralisation? by (10 Mar 2023 10:30 UTC; 10 points)
- EA & LW Forum Weekly Summary (20th − 26th Feb 2023) by (LessWrong; 27 Feb 2023 3:46 UTC; 4 points)
What is the main issue in EA governance then, in your view? It strikes me [I’m speaking in a personal capacity, etc.] the challenge for EA is a combination of the fact the resources are quite centralised and that trustees of charities are (as you say) not accountable to anyone. One by itself might be fine. Both together is tricky. I’m not sure where this fits in with your framework, sorry.
There’s one big funder (Open Philanthropy), many of the key organisations are really just one organisation wearing different hats (EVF), and these are accountable only to their trustees. What’s more, as Buck notes here, all the dramatis personae are quite friendly (“lot of EA organizations are led and influenced by a pretty tightly knit group of people who consider themselves allies”). Obviously, some people will be in favour of centralised, unaccountable decision-making—those who think it gets the right results—but it’s not the structure we expect to be conducive to good governance in general.
If power in effective altruism were decentralised, that is, there were lots of ‘buyers’ and ‘sellers’ in the ‘EA marketplace’, then you’d expect competitive pressure to improve governance: poorly run organisations will be wracked by the “gales of creative destruction” as donors go elsewhere.
If leaders in effective altruism were accountable, for instance, if EVF became a membership organisation and the board were elected by its (paying?) members, that would provide a different sort of check and balance. I don’t think it’s reasonable for individual donors, i.e. Dustin Mosktovitz and Cari Tuna, or cause-specific organisations, to submit their money to the democratic will, but it seems more sensible for central organisations, those that are something like natural monopolies and ostensibly serve the whole community, to have democratic elements.
As it is, the governance structure across EA is, essentially, for its leaders to police themselves—and wait for media stories to break. Particularly in light of recent events, it’s unclear if this is the optimal approach. I am reminded of the following passage in Pratchett.
Upvoted. I think there is an assumption lurking in here, though. It seems likely that EVF would be a pale shadow of itself without Open Phil and other big-donor money. I dont member fees can plausibly replace that.
So there seemingly has to be an assumption that the big donors should continue funding EVF even if they strongly disagree with who the demos elects, as opposed to funding whatever the old guard starts up after being deposed. There seems to be some tension with the idea that donors aren’t obliged to defer to the democratic will.
Something OpenPhil could do is fund explicit alternatives to core EA orgs. They don’t have to be ‘competitors’ per se, in that there’s no market they have to vie for, but they could do the same thing in a different context—events in Europe vs events in North America for example, or longtermist events vs animal welfare events, and be available for the occasional encroachment into the others’ remit if someone asks them to. That way you could have multiple organisations with similar competence, and a visible signal if one was consistently doing better than the others, and funding could explicitly follow that signal.
This would be a lot easier the more focused the orgs/the less mission creep they had, since then the comparisons would be a lot higher fidelity. EVF’s current structure doesn’t seem ideal for this.
This is an interesting idea, but I don’t know how feasible or realistic it is. I find it really helpful to think of EA as a marketplace for maximum impact goods and services. Like your local farmers’ market, but instead of selling fruit and vegetables, people are offering charities, charity recommendations, and so on. (I’ve been reflecting on this for a while, and might write it up as a standalone post). The analogy doesn’t have to be perfect to be informative. In this framing, what we’d want, presumably, is for there to be lots of competition and choice, so that consumers get a better deal. You don’t want there to be just one guy that sells fruit, and only sells oranges, for example.
The challenge for EA is that one buyer dominates the market: Open Philanthropy is, I think, 50+% of total. In effect, you’re going to get about as much variety as OP wants, when they do their own ‘shopping’: if OP wants to buy your fruit, you’re in business; if they don’t, it’s much harder to survive in the market. If they want there to be a greater variety of sellers, they can create demand for them—equivalent to saying “hey, if you start selling apples in the market, we’ll buy 10 crates” or whatever. But it’s not clear if it’s a good use of their money, by their lights, to create a market for products they don’t really want and other people may not want. It puts the new vendors in an odd, risky position too if they only exist because they have one, questionably enthusiastic consumer. They could see it as a good use of money: they’d be subsiding the creation of products in the hope of creating demand and bringing other people into the market. Hence, it’s not obvious the main buyer in a marketplace would want to creat variety vs just seek out the specific things they want.*
*Mutatis mutandis, there are the same issues if major funders engage in regranting, which is like giving your friends some of your money and asking them to buy stuff in the market. If your friends have the same preferences to you, they’ll buy the same stuff, so there’s no point. If they buy stuff you’d hate, then you’ll think you shouldn’t have given them money. Either way, unless you keep giving them money, it’s only an artificial spike in consumption.
I’m thinking more about separating existing markets, or perhaps setting up new orgs to catch the overflow from when existing ones expand. Fwiw FTX Foundation explicitly wanted to do this—they had a category for ‘competition to existing EA orgs’. This could have been because as a different org they were sceptical of OP’s priorities, though this seems unlikely given the substantial overlap between the people involved, so it seems more like a genuine desire for a better marketplace.
This seems too simplistic and worrying if we genuinely think this is how OP think. Most people recognise that there’s a wisdom in crowds, and epistemic advantage to a diversity of thoughtful viewpoints.
Ah. I should probably have flagged I’m pointing to an ideal world. As goes (almost) without saying, all this is easier said than done. You could say that EA’s challenge is that it has only one mega-donor. It seems much better that it has one than none, but having just one creates distinctive governance challenges compared to if there was no mega-donor but lots of small donors; this latter scenario is a bit closer to a democracy in terms of how power is distributed.
But your comment does rather make my point. In a world where there is only one major donor, that is, effectively a monopsony (a market with single buyer) you could say “look, all this talk about good governance is nice, but it’s practically irrelevant; the only question to ask is ‘what does the big funder want?’”.
I don’t think donors can or should be forced to defer to the democratic will. People, qua citizens in their democracy, can be forced to do things, but charity is a necessarily private act. I think it would be good if EVF had some democratic elements and also that someone paid for it to keep going. But, who am I to tell people what they ought to do, a moral philosopher, or something? :)
Why so?
I wrote in this direction a few years ago, and I’m very glad to see you clearly stating these points here.
From What’s the best structure for optimal allocation of EA capital? –
Thank you for this post and the work you’re doing. Given the small size and newness of the EA community and many orgs/projects, I’m personally also very worried about something like “right group of people, great practices, unforeseen or unpreventable dependencies that lead to major risk of collapse should a few key things go wrong in succession.” My impression is that in some cases things have been going very well but the external pressures have been so substantial and consistent the past few months that even very stable teams are trembling. Sound practices can help mitigate this, but I also want to see more people feeling ok saying, “this is a shit time and we’re treading water, but we’re able to tread water until we reach shore because we prioritized a healthy infrastructure beforehand.”
Dear all
An interesting thread.
For what is worth, I have over 15 years experience observing governance failures at different levels in organization where I work (that shall name anonymous), as well a “bad actor” incident within one of the multiple project specific teams that I have developed and then disbanded or transferred once the work was complete. I fully agree with the analysis and the first part of the response/mitigation measure proposed by Grayden.
In addition, looking the EA sector personnel profiles and reading some of their posts, what strikes me is the amount of intelligence and energy demonstrated, in parallel with apparent limited life experience and in some cases simply a lack of wisdom. This in turn I feel reflects the recent surge in interest in EA. Previously it was a niche and fairly academic , and thus a bit grey haired. Then for a few years it has started to attract a lot of bright young talent, culturally trained for better or worse in part by the tech startup sector.
The timeless reality is that young energetic people with insufficient mentoring and/or oversight make mistakes, cause or sustain some damage and then learn from this. That happened to myself as well, more than once. The problem is when these young explorers are also leaders, in a position of serious authority or influence and in the public spotlight. Then the fall can be long and the collateral damage can be immense.
The clean up and recovery, as I know all too well, can take literally years, even for small to medium sized incidents. Major incidents can simply kill off entire teams, initiatives and organizations or leave a permanent mark.
EA right now is sadly in the global reputation doghouse and proving to be an easy target for critics. A 101 good governance improvement campaign will certainly help, starting at the top as Grayden proposes. I would also suggest that the movement look again at its team profiles and internal training.
What may work better in the long run is more of a balanced blend of youthful energy, external viewpoints and deeper older experience : neither dampening what makes EA so interesting and potentially useful, nor letting individuals drive the movement off the cliff.
Happy to see this, thanks for putting it together.
For what it’s worth, I roughly agree with a lot of this. I personally see EA challenges now very much as “maturing in management, generally” as opposed to anything very specific, like, “stopping a few bad actors”.
I expect that many senior people roughly would agree.
What do you specifically mean by “maturing in management, generally”? I noticed that people tend to have very different things in mind when they are talking about “Improving management in EA” so could be worth clarifying
I defer a lot to experts / well respected managers.
To me, EA has a bunch of young people optimized a lot for some specific non-management talents. It seems a lot like a startup in that way.
Many startups go through “growing up” periods. Some totally fail at this, but when it works well, the outcome can be very successful.
I imagine as we get good consultants here, they will recommend some fairly straightforward and correlated recommendations that I’d agree with.
I found the Personal MBA reading list to be interesting. There are really a lot of “serious organization” skills that are hard to get good at.
https://personalmba.com/best-business-books/
I think EA has a particular problem where the emphasis on getting people who are “value aligned” means they don’t get in experienced people from outside. Software startups have at least learned to bring in an experienced COO to help run day to day things
In the early days, it was hard to get people who were sufficiently “value aligned” with experience. I don’t think that’s the case anymore. For example, there are many value-aligned highly-experienced people on the EA Good Governance Project’s Trustee Directory. The issue is that many people don’t want to give up partial control of something to somebody outside their clique and/or prior to October didn’t have an easy way to find them
I think good startups often do this, but lots of startups have trouble about this stage. Many do have their own cultures that are difficult to retain as they grow.
I think EA is more intense as there’s more required material to understand, but it’s a similar idea.
Thanks for clarifying! I think Training for Good looked into “scalable management trainings”, but had a hard time identifying a common theme, which they could work on (This is my understanding based on a few informal chats. This might be outdated and I am sure they have a more nuanced take). Based on my experience, different managers seem to have quite different struggles which change over time and good coaching and peer support seemed to be the most time-effective interventions for the managers (This is based on me chatting occasionally to people and not based on proper research or deep thinking about the topic)
I think there’s no substitute for role models and experience. Whenever I advise people in EA on careers, I always suggest spending some time in ‘normal’ organisations first
Hmm. Obviously, career advice depends a lot on the individual and the specific context, all things equal, I tentatively agree that there is some value in having seen a large “functioning” org. I think many of these orgs have also dysfunctional aspects (e.g., I think most orgs are struggling with sexual harassment and concentration of formal and informal power) and that working at normal orgs has quite high opportunity costs. I also think that many of my former employers were net negative for some silly which I think are highly relevant, e.g., high-quality decision making
Re: “I think many of these orgs have also dysfunctional aspects (e.g., I think most orgs are struggling with sexual harassment and concentration of formal and informal power)”
I agree with that, but I also think there’s something to be learned from dysfunctional orgs. Why are they dysfunction? How did they become dysfunctional? Why have attempts to make them less dysfunctional failed?
There is just as much — possibly more — to be learned from failures as there is to learn from successes.
On the whole I think there’s less to learn from failures than successes because there are many different ways to get things wrong, so ruling out just one doesn’t help you that much. The lesson is only valuable if you would have been dysfunctional in that specific way, and are now able to avoid it.
Working in dysfunctional workplaces can still be educational if:
their dysfunction is the biggest / most obvious trap to fall into, and learning to avoid it really does affect your chances of success,
they’re not dysfunctional in all ways, and have enough other good examples to learn from.
I suspect that in reality a lot of organizations are dysfunctional in ways that aren’t useful, e.g. incentive problems. While incentive problems do exist in EA, the incentive landscape is pretty different and I’d guess a lot of the traps don’t straightforwardly translate.
Great post, thanks.
I might elaborate on your last category to include a) well-intentioned high competence people accidentally creating bad systems; and b) well-intentioned high competence people put into bad systems by leadership (so not just leaders but e.g. a community health team trying to deal with sexual harassment by one of their own Board members).
I think your section header covers this, but the body focuses specifically on CEOs and Boards. Lots of people in EA, not just leadership, can end up making mistakes because the systems/policies they work within aren’t fit for purpose.