Rowing and Steering the Effective Altruism Movement
Acknowledgements
I am grateful to Jessica McCurdy, Carla Zoe Cremer, Janvi Ahuja, Jonas Sandbrink, Kaleem Ahmid, and one anonymous reviewer for conversations and feedback that significantly improved this post. Any errors, of fact or judgment, remain my entirely own.
Introduction
In a recent blog post, Open Philanthropy Co-CEO Holden Karnofsky introduces the analogy of “the-world-as-a-ship” as a framework for thinking about efforts to make the world better.
Under this analogy, if you are ‘rowing’ the ship, you are trying to “help the ship reach its current destination faster.” By contrast, if you are ‘steering’ the ship, you aim to “navigate to a better destination than the current one.” Alternative options include anchoring, equity, and mutiny. It’s a terrific post introducing a practical framework for thinking about world-improving efforts such as those undertaken by the effective altruism and longtermism communities.
I think we can apply the analogy of the ship in a narrower sense to effective altruism and longtermist movement-building efforts.[1] In particular, I find that the notions of rowing and steering can elucidate some key differences in beliefs that I’ve recently observed in the community.
In this post, I briefly contrast what I see as the rowing and steering approaches to building the effective altruism movement and discuss success and failure from each perspective.
I present three arguments for why ‘steering’ is more important than is currently recognised: i) Given the momentum of the effective altruism and longtermism movements, steering is currently more neglected than rowing; ii) steering is more tractable if implemented sooner rather than later; and, iii) steering is valuable for community health.
I then offer concrete proposals for one form of steering that I’d like to see more of, namely good-faith critiques of the effective altruism philosophy and movement. Specifically, I suggest commissioning research on effective altruism and longtermism failure modes, offering Effective Altruism Forum prizes for good-faith critiques, funding, creating, and managing effective altruism ‘red teams’, and committing to reviewing and acting on valid critiques.
Defining rowing and steering
I’ll define ‘rowing for the effective altruism community’ roughly as ‘efforts to increase the power of the effective altruism movement.’ This can be achieved by recruiting or converting more people to support the cause, increasing the engagement of existing community members, obtaining more funds committed to the movement, creating powerful allies in government and industry, and so on.
By contrast, I’ll define ‘steering for the effective altruism community’ roughly as ‘efforts to improve the movement’s aims and/or to ensure that the movement executes on those specific aims.’ This can be achieved by researching global priorities, evaluating competing causes and interventions, critically examining and improving current cause-prioritisation methodologies, critiquing the direction of existing efforts, creating institutions and norms to enhance procedures and decision-making, and so on. Crucially, on my definition, steering isn’t just about pulling the ship in the direction you currently think is best. It’s also about questioning whether your navigational approach, crew composition, and equipment are adequate for the task. It’s about ensuring that, in the future, one can steer even better than the current crew, and the future crew is not hindered from pulling the ship in the direction they think is best. For example, it’s not just about doing global priorities research; it’s also about continually and rigorously asking whether your way of doing such research has any blind spots or drawbacks.
Rowers seek to accelerate the effective altruism ship; steerers seek to question and adjust its current direction.
What failure means for effective altruism under rowing and steering
Failure for rowers
My motivation for writing this post is a recurring observation that rowers and steerers tend to be concerned by different problems and challenges for effective altruism, which appear to arise from distinct (albeit mutually compatible) conceptions of what failure means for the movement.
According to the rowing perspective, effective altruism fails by not growing powerful enough, fast enough. For example, Nuno Sempere’s Forum post “Why do social movements fail: Two concrete examples” focuses on two social movements (the Spanish Enlightenment, 1750-1850, and the General Semantics movement,1938-2003) that seemingly failed in the sense of failing to persist. A more detailed version of the post’s title could be “Why do social movements fail [to be as powerful and lasting as they could be]” more so than “Why do social movements fail [to maintain good aims and careful execution of those aims]”. In other words, this analysis implicitly suggests that the effective altruism movement should learn from failures to grow in power – we should heed the lessons of failed rowers of the past. An alternative, steering version of this post would perhaps have considered how certain social movements (say, advocacy movements for socialism or capitalism) failed in the sense of leading to inadvertent negative consequences or in the sense of being co-opted by powerful interests.
One specific way that effective altruism might fail is by failing to avert an irreversible and extremely undesirable outcome, such as an existential catastrophe. In light of this, I can imagine that people’s views on things like AI timelines highly influence whether they lean towards a rowing or steering perspective. For example, suppose you believe that existential risks from artificial intelligence (or other sources) are imminent. In that case, this may lead to a strong emphasis on rowing: the movement needs to become very influential and very fast, lest it misses a slim window of opportunity to avert impending civilisational catastrophe.
According to the rowing perspective, critiques of effective altruism and longtermism can be highly concerning and frustrating, as they could impede movement growth. For example, such critiques may dampen the motivation of current and prospective effective altruism rowers – as far as I can tell, some Forum commenters have expressed concerns that a particular criticism of effective altruism or longtermism could have adverse effects on movement building. This issue could be particularly pressing in the early stages of a movement when its proponents have not yet fully defined terms like ‘longtermism’ in the minds of the broader public so that widely read critiques are more likely to shape initial impressions. Naturally, this concern will be especially salient when a critique is perceived as poor or even unfair. However, it is notable that the concern could also arise even for valid and reasonable critiques, insofar as such critiques may impose considerable costs by slowing down rowing efforts, notwithstanding whatever veracity and potential benefits. Consequently, the rowing mindset is particularly sympathetic to the idea that critiques such as those offered by Phil Torres warrant vigorous responses, such as the detailed discussions offered by Avital Balwit and HaydnBelfield.
According to the rowing perspective, ‘pessimism about effective altruism’ typically means something like ‘pessimism about the movement’s pace of growth’ rather than ‘pessimism about its direction.’ For example, in a blogpost from 2020 titled Optimism about EA Longtermism, Stefan Schubert writes that while effective altruists are “sometimes pessimistic about how the movement is developing,” he thinks that there’s some reason for optimism “if we look at how EA long-termism has developed in the last years.” As the basis for this optimism, Schubert cites the following observations:
“Several new organisations that are aligned with or sympathetic to EA long-termism have been set up”
“Key EA ideas, such as AI safety and long-termism, have received increased support.”
“The community is maturing in many ways.”
“The community is attracting lots of very talented and reasonable people.”
“The overall trajectory seems pretty stable and robust. Success isn’t dependent on some daring moves. Instead, most big initiatives seem pretty safe.”
The emphasis of (1)[3], (2), and (4), seems to fit pretty squarely within rowing, while (3) feels more like steering and (5) depends on how you interpret “stable and robust.” The rowing perspective of the blogpost is clearest when Schubert wraps it up by writing: “One thing that does worry me a bit, however, is that we might not grow fast enough. EA long-termists may be able to change the long-term direction of the world, if we become sufficiently influential. So failing to do so could constitute a great opportunity cost. Hence, it may be important that EA long-termism grows at a good pace.” In other words, this seems like an example where a ship’s pace at least partly determines optimism or pessimism about it.
According to the rowing perspective, effective altruism community builders should focus on reaching as many people as possible who are the most “aligned” with current conceptions of effective altruism and encourage them to execute the movement’s current aims. Given the importance of achieving as many of those current aims as fast as possible, community builders focused on rowing may be concerned about including “unaligned” members of the movement, lest they slow down progress by concentrating on relatively unimportant goals with significant opportunity costs.
Failure for steerers
According to the steering perspective, failure modes for effective altruism include, at least, the three following categories:
Object-level failures concerning cause prioritisation, evaluation of global priorities, or interventions with unintended consequences, etc.
Community failures concerning culture, (epistemic) norms, homogeneity of worldviews or demographics, whether the movement should be large or narrow, etc.
Organisational failures concerning hierarchies, decision-procedures, transparency, degree of democracy, accountability of those with power, etc.
According to the steering perspective, failure arises not (only) when effective altruism fails to grow but when it grows in the wrong direction. Steerers listen with keen ears when the American poet J. Cole cautions: “the good news is (…) you came a long way // the bad news is (…), you went the wrong way.”
According to the steering perspective, “pessimism about effective altruism” is more likely to mean “pessimism about what the movement is doing.” Accordingly, it would take more than growth to become optimistic about effective altruism; it would also require signs that norms and institutions for careful navigation are being implemented.
According to the steering perspective, it is profoundly concerning if the movement prematurely converges on a narrow view of the best direction for the effective altruism ship. For example, my Future of Humanity Institute colleague Zoe Cremer makes an exceptionally compelling case against value-alignment between effective altruists and highlights a need to steer clear of certain pitfalls.
According to the steering perspective, the effective altruism and longtermism communities must be diligent about procedural errors with potentially grave consequences. For example, Linch Zhang (of Rethink Priorities) highlights the threat of motivated reasoning to effective altruism efforts.
According to the steering perspective, the growing influence of the effective altruism movement could become a source of concern insofar as the movement is judged to be erring in serious ways. For example, when Phil Torres notes that “longtermism might be one of the most influential ideologies that few people outside of elite universities and Silicon Valley have ever heard about,” it seems that this should be interpreted as a word of caution rather than a celebration of success; that growth and influence only are successes insofar as one is convinced about the ship’s direction and navigation.
Because the steering perspective emphasises direction over speed, critiques such as these can be valuable in light of their potential to nudge the ship in the right direction. And just like the rower might be frustrated with critiques despite their merits, the steerer might welcome some critiques despite their flaws.
According to the steering perspective, community builders should focus on what the effective altruism community might be getting wrong and how their community-building efforts might remedy or exacerbate errors.
A brief case in favour of steering
My current vantage point likely shapes my inclinations towards rowing versus steering from the ship (see Addendum below). Having noted that, I would like to argue that, on the current margin, the effective altruism – and perhaps particularly the longtermism – community would benefit more from steering than from rowing. I recognise that the significant expansion of effective altruism and longtermism is not inevitable and that the opportunity costs of suboptimal growth rates are real. Still, I see at least three salient arguments for focusing more on steering in years to come:
1) Steering feels relatively neglected
It’s not feasible to quantify and directly compare the relative neglectedness of steering and rowing efforts. I also appreciate that many people are working hard on what I would categorise as steering. Those caveats aside, I think that one reason to focus more on steering is that, in many ways, progress for the effective altruism and longtermism movements seems to be going really quite well.
Over the coming decade, I expect philanthropists to commit and donate hundreds of millions of dollars to effective altruism causes, dozens of new organisations to be founded, thousands of individuals recruited to the movement, and numerous highly influential global decision-makers to become persuaded of its tenets. Moreover, the impression I get at lunchtime conversations around the Future of Humanity Institute, Global Priorities, Forethought Foundation, and Centre for Effective Altruism offices is that the focus much more often seems to be on “how can we get our vision implemented, quickly and powerfully?” rather than “how can we ensure that we have the right vision and that we implement it correctly before we push for it?” Perhaps one reason rowing is unlikely to be neglected is that it seems essential: being powerful is necessary to execute good aims. But the flip side is that steering can become neglected, as power can be obtained without having good aims.
In light of this, I would love to see more focus on the latter question, especially (though not exclusively) from people who influence the strategic decision-making in these communities. Again, I wish to recognise that many community leaders strongly support steering – e.g., by promoting ideas like ‘moral uncertainty’ and ‘the long reflection’ or via specific community-building activities. So, my argument here is not that steering currently doesn’t occur; rather, it doesn’t occur enough and should occur in more transparent and democratic ways.
2) Steering is more tractable earlier rather than later
I would argue that it will be easier to steer effective altruism in the right direction now, relative to how difficult it will be when the movement is, say, ten times as large and influential as it is now.
I think this applies strongly to steering aimed at improving community norms, dynamics, and health. For example, problems like intellectual or demographic homogeneity seem extremely self-reinforcing in many ways, meaning that they will only become harder to address once the movement has grown further. If so, this would make efforts to increase the currently dismal diversity of worldviews, styles of thinking, and demographics within the community highly urgent.
I also think something similar can apply for direct interventions and policy advocacy. Once you put something on the UN Secretary General’s desk or in the general public’s awareness, it can become hard to subsequently steer in a different direction than the one you initially laid out. I think we’ve seen this, in particular, with some of the early effective altruism ideas that originally were the objective of much rowing but no longer reflect the movement as well, such as earning to give or a stricter focus on interventions informed randomised controlled trials.
Perhaps there’s a lesson to be learned here from the private sector, where I get the impression that it is widely believed that organisational cultures must be carefully defined early on, as they are both highly influential and highly challenging to alter later on.
3) Steering is good for community health
Another reason to steer carefully is that some people, who could otherwise have been powerful rowers, will jump ship if they believe that the ship is going in the wrong direction and see too few efforts at correcting course.
I know several people who would fit any reasonable description of a “highly engaged EA” who privately and publicly question the extent to which they are comfortable identifying as members of the effective altruism community for this exact reason. I count myself in that category.
I also know about a handful of people who have ‘jumped ship’; who, after spending many hundreds of hours working on effective altruism causes, have concluded that they are too disillusioned with the ship’s direction to stay on board and no longer wish to associate with the movement. These are not people who were never going to become highly engaged in the community in the first place. They are people who have contributed enormously to the effective altruism project and could have continued if they had been more confident in its ability to move in the right direction.
Of course, different people are likely to have different visions for where to go, so it is somewhat unavoidable that any community would lose some people this way. Nevertheless, it is worth taking seriously that this happens and that a very concrete benefit of steering efforts is the retention of a wide range of highly capable community members.
Two steps for successful steering
Continually and critically evaluating the current course
Often, steering involves two steps: first, identifying what could be wrong with the current direction or the systems that produce navigational decisions, and second, acting to alleviate any identified problems. In practice, I think that the first step looks a lot like critiquing, or at least thinking critically about, the beliefs, values, norms, strategies, and tactics that predominate in the movement at any point in time. Accordingly, while I recognise that critiques may be unwelcome from a rowing perspective given their risk of being used as part of bad-faith opposition, I think that critique of effective altruism serves an essential role in steering.
Here’s a paradox I’ve sometimes felt in the effective altruism community: despite being exceptionally and impressively ardent about the importance of changing one’s position in response to novel evidence, and despite having unusually transparent and rigorous internal debates about even the smallest details, it sometimes seems that the community is not always great at taking seriously critiques coming from outsiders. [2] This is particularly the case if the critique is presented in different terminology or based on different methodologies than those that dominate the community (i.e., economics, computer science, evolutionary and behavioural psychology, analytic Western philosophy, physics, and so on).
More than once, I’ve seen peers outright dismiss a critique based on the thinnest of possible slices, such as the use of specific terminology or language associated with a certain ill-regarded academic discipline or social movement. In particular, I’ve more than once observed something being summarily dismissed based on using language associated with social justice movements, but I think the point holds more generally, too.[3]
Failing to heed critiques properly means missing opportunities to correct course and improve the movement’s direction. Ideally, we would become even better at taking outside critiques seriously, as they’re likely to raise considerations that could be missed by people already deeply involved. But in the meantime, I argue that proponents of effective altruism and longtermism should undertake much more concerted efforts to produce strong, good-faith critiques of their own worldviews, communities, and activities.
One benefit of such self-directed critiques is that they are more likely to be taken seriously if presented in a language and style of reasoning that community members tend to heed.
Another reason to produce good-faith critiques is to crowd out poor-quality critiques, allowing outside observers to form more accurate beliefs about what effective altruism and longtermism is and isn’t about.
Concrete interventions for improving effective altruism through critique
I would love to see more resources dedicated to professional steering. Here are some concrete interventions that I think could be great – but, importantly, this is not at all an exhaustive list, and I’m confident that better ideas exist:
Conduct and commission research on questions to critically guide the direction of the effective altruism and longtermism movement, such as:
What are current problems in the effective altruism funding landscape, and how might they be mitigated?
What are current problems in the effective altruism community culture, and how might they be mitigated?
What are current problems in the effective altruism strategic decision-making, and how might they be mitigated?
If longtermism went really wrong and caused a lot of harm, how might that happen? (Beware infohazards, of course)
How is the philosophy of longtermism most likely to be misunderstood by policymakers in the next three decades? How is it most likely to be misapplied?
What social movements have failed, not in the sense of not having an impact, but in the sense of having the wrong kind of impact? Why did that occur, and what can we learn?
Offer several Effective Altruism Forum prizes for the best posts on topics like these.
Passively offer funding to support someone dedicated to critiquing effective altruism on their terms. Here are two promising first steps, which I’d love to see publicised much more widely.
Actively set up a 2-4 person “Effective Altruism Red Team”, for instance, as an official part of the Centre for Effective Altruism. In contrast to the above proposal, this would involve active line management, creating a red team research agenda, and actively managing common red teaming challenges (e.g., pressures to hold back particular critiques or adverse social dynamics). For such a team, two things seem crucial:
Actively supporting the implementation of interventions based on identified problems.
Creating mechanisms for soliciting and integrating wider community views, including surveys, referenda, and interviews.
Have a dedicated session at Effective Altruism Global conferences focusing on identifying and broadcasting how the movement could improve in terms of community culture and norms, decision-making processes, and overall strategy.
Go further in using the annual ‘Coordination Forum’ to improve community norms and decision-making processes, including consideration of crowd-sourced input. Improve the forum’s transparency by producing a summary output from this session for the community to digest and scrutinise.
Conduct or commission an annual review involving a concrete evaluation of whether sufficient action has been taken in response to any of the above, or other, sources of good-faith criticism.
One reason I’m excited about interventions like these is that I suspect (and hope) that there’s a lot more untapped ideas out in the community than you’d expect just based on the work that people produce. For example, my daytime job is to work on global catastrophic biological risks. This is what the Future of Humanity Institute pays for, and it’s what my career is shaping up to be all about. While I’m very excited by this, it means that thinking critically about how the effective altruism community could improve is relegated to the margins of my days. I know several colleagues who would love to see more time spent on steering our community but who themselves can only focus on it as a moonlighting activity. [4] Case-in-point: I wrote this forum post on my holidays rather than launching a fully-fledged project to nudge the community in one of the directions that I’d like to see it go. The upshot is that creating the right financial, social and institutional conditions (i.e., by providing funding, changing norms, and reforming events and organisations) could potentially leverage a large stock of existing thinking on how to improve effective altruism.
Taking action to correct course
As noted, the second step of steering is to act on identified navigational failure or failure modes. Accordingly, for all of these suggestions, I really must stress that even the best critique is worthless without action. Funding and praising someone for pointing out a harmful community norm or inadequate institution is only valuable alongside concrete steps to remedy the issues. A brilliant analysis of how policymakers might misunderstand or misapply longtermism is only useful insofar as longtermist policy advocates take these risks seriously and proactively mitigate them. And so on.
Conclusion
I believe there are solid reasons for wanting effective altruism to grow in influence, reach, and power. These include my belief that the movement has had an extraordinarily positive impact on this world and very likely will continue to and the risk that existential catastrophes could cause irreversible damage in the near future without the right interventions.
I also believe that it is extremely important to focus on improving the movement beyond increasing its influence, reach, and power. This includes:
refining methodologies for evaluating causes;
examining crucial considerations for current tenets of effective altruism;
improving the epistemic and social norms in the community;
facilitating intellectual, moral, and demographic diversity in the community; establishing new mechanisms for transparency and distributed decision-making;
worrying more about how longtermism could increase risks or go wrong;
and more.
I believe that good-faith critiques play a vital role in achieving these ends and that the movement should use more of its financial and human capital to facilitate such critical examination of effective altruism. I also believe that critical evaluation is only the first step of effective steering; actual implementation, reevaluation, and maintenance must follow.
Finally, I choose to remain optimistic. The effective altruism movement is based on inspiring and pro-social tenets and the effective altruism community includes some of the most thoughtful and caring people I have the privilege of knowing. While I see much room for improvement for both the movement and the community and am often deeply concerned about the prospects of realising those improvements, I maintain hope that this ship will find a good course and reach a safe harbour in time.
Addendum: Vantage points from the ship
As noted above, the rowing and steering perspectives are entirely mutually compatible. You can want the ship to go faster and want it to go in the right direction. Accordingly, very few people are pure rowers or steerers in practice and I genuinely believe that almost all members of this community are excited about progress in both steering and rowing. But, as with many other things, there can be a trade-off in how much to emphasise one perspective over the other, and focusing on one can be at the direct expense of the other. People differ quite a lot and I suspect that one’s vantage point can sometimes be relevant. For example, my first-hand experiences of the movement’s current momentum have increased my interest in steering.
I currently work at the Future of Humanity Institute, which shares offices with the Global Priorities Institute and the Centre for Effective Altruism. Here, my friend and colleagues work on activities you could categorise as both steering (such as global priorities research) and rowing (such as outreach, whether to academia, policymakers, or university students). Working and living in an environment where most of the people I am around – including those I live with – are heavily embedded in effective altruism and/or longtermism has given me the visceral sense that the ship is moving very fast. Every day, I talk with capable, hard-working people who are keen to dedicate their entire professional, and most of their personal, lives to the effective altruism cause and community. Every week, I hear conversations about the enormous overhang of funds committed to effective altruism and how easily the community could channel dizzying amounts of money towards any cause or project deemed valuable. I hear about the Secretary-General of the United Nations supporting longtermism, the Prime Minister of the United Kingdom quoting Toby Ord on the floor of the United Nations, and ambitions and concrete projects to get members of the movement into seats of power across the world. From this vantage point, steering is relatively more neglected than rowing: it sometimes seems that the ship inevitably will make progress, that its pace is both impressive and gaining, and that more people thus should think longer and harder about its direction as well as its navigational equipment. Consequently, I now spend much more of my time thinking about steering as opposed to rowing, and I’m much more likely to bring the former to the lunch table and water-cooler conversations than the latter.
But I am also mindful that, from some vantage points, the movement feels in dire need of some heavy rowing. I experience this in my professional work on global catastrophic biological risks. Here, so many stakeholders in biosecurity and public health seem either unaware of – or indifferent to – the vital insights around scope sensitivity, information hazards, and tail risks that effective altruism-oriented members of the field bring to the table. In that context, I saliently feel the need to row hard and swiftly; to bring more people on board, and proliferate existing ideas and strategies widely. I also remember what it was like to do community building on a college campus, to encounter hundreds of people who were either unsympathetic or indifferent to the tenets of effective altruism, to work tirelessly for months to recruit just one more highly engaged group member. This is probably why my previous posts on the Forum (an effective altruism fellowship guide, an interview with Ben Garfinkel of GovAI, and a syllabus on longtermism) were motivated mainly by rowing inclinations. From that vantage point, rowing should be prioritised heavily. Progress for this ship is not inevitable, the opportunity cost of anything less than the fastest pace is enormous – particularly if existential catastrophe could be imminent – and we’re already on such a solid course compared to any other political or social movement that we should unequivocally pick up the pace even further.
Addendum, March 27, 2022: Relevant links
Thanks to everyone who attended my talk on this subject at EAGx this weekend! As promised, here’s a collection of the examples I highlighted in the talk, and a few others:
Announcements
Pre-announcing a contest for critiques and red teaming
Apply for Red Team Challenge [May 7 - June 4]
Highlighted critiques
Normative assumptions & moral values:
AI Alignment, Philosophical Pluralism, and the Relevance of Non-Western Philosophy
Constraining longtermism (Working paper, likely available upon request)
The only ethical argument for positive 𝛿?
On two arguments for Fanaticism
Empirical assumptions & theories of change
How sure are we about this AI stuff?
Why I am probably not a longtermist
The case of the missing cause prioritisation research
Institutions & organisations in the effective altruism movement
Why I’m concerned about Giving Green
The EA Community and Long-Term Future Funds Lack Transparency and Accountability
Social norms & practices in the effective altruism community
Objections to value-alignment between effective altruists
In defence of epistemic modesty
The motivated reasoning critique of effective altruism
- ^
I may not be using the terms exactly in the same way as Karnofsky does – in particular, I think my conception of steering often relates close to what he calls equity – and any conceptual errors or confusions are naturally entirely my own. For prospective commenters, I should note that I’m less interested in the semantics of whether my use of the terms maps perfectly onto Karnofsky’s and more interested in the substantive implications of rowing and steering as I define them.
- ^
Or even vigorous critiques from insiders, as highlighted in a recent post by Zoe Cremer and Luke Kemp.
- ^
I’m not sure why proponents of effective altruism would turn out to be especially poor at listening to critiques, when open-mindedness indeed seems so essential to its philosophy. If I had to guess, I suspect it has to do with the fact that effective altruism often involves thinking that you’ve spotted something true that the rest of the world hasn’t, such as more cost-effective interventions or more compelling moral views. To be clear, I profoundly do think that the community has done this – but the tricky part is to separate when one is right about an unusual view (and so can be more comfortable dismissing outgroup disagreement) and when one hasn’t (and so should heed outgroup advice). I think another contributing factor is the disciplinary homogeneity of the community, wherein most people come from a handful of academic backgrounds, leading some disciplines to be poorly understood, and thus undervalued, within the community.
- ^
Here is another recent example of someone who seems interested in such topics but doesn’t have the time to pursue them more systematically
- Pre-announcing a contest for critiques and red teaming by (25 Mar 2022 11:52 UTC; 173 points)
- The privilege of native English speakers in reaching high-status, influential positions in EA by (20 Dec 2023 21:07 UTC; 140 points)
- To what extent & how did EA indirectly contribute to financial crime—and what can be done now? One attempt at a review by (13 Apr 2024 5:55 UTC; 63 points)
- EA Updates for February 2022 by (28 Jan 2022 11:33 UTC; 28 points)
- Disagreements about Alignment: Why, and how, we should try to solve them by (8 Aug 2022 22:32 UTC; 16 points)
- 's comment on aogara’s Quick takes by (31 Mar 2022 20:59 UTC; 15 points)
- 's comment on The Culture of Fear in Effective Altruism Is Much Worse than Commonly Recognized by (7 Feb 2022 16:54 UTC; 15 points)
- Proposal for Scaling Community Building by (26 Feb 2024 1:13 UTC; 13 points)
- Disagreements about Alignment: Why, and how, we should try to solve them by (LessWrong; 9 Aug 2022 18:49 UTC; 11 points)
- 's comment on (Self-)Criticism Doesn’t Solve Problems. We Do. by (6 Aug 2022 16:28 UTC; 1 point)
Thanks for this post! I think it’s interesting, I’m very glad you wrote it, and I’m inclined to agree with a narrow version of it. I imagine many senior EAs will agree as well with a statement like “often to do much more good we’re on the margin more bottlenecked by conceptual and empirical clarity than we are by lack of resources” (eg I’d usually be happier for a researcher-year to be spent helping the movement become less confused than for them to acquire resources via eg writing white papers that policymakers will agree with or via public comms or via making money), though of course I imagine degree of agreement varies a lot depending on the operationalizations in question.
One thread of thought I had that’s somewhere between an extension and a critique:
I’d be interested in seeing someone develop into what anchoring, equity, and mutiny looks like for the EA and LT movements.
This is particularly relevant here because in practice. attempts of steering cannot easily be differentiated from attempts at anchoring, equity, or mutiny.
For example, “maybe we should stop what we’re doing and critically re-evaluate whether longtermism is good” might be an attempt at steering, but it might also be an attempt at anchoring.
Steering options that look like taking into account more voices within effective altruism, or considering a broader range of perspectives and causes, with an (implicit) inference that equity and diversity leads to better decisions, might likewise be hard to differentiate in practice from an intrinsic preference for within-effective altruism equity, whether among individual EA people or EA cause areas. Hadyn’s post of alternatives to donor lotteries comes to mind here.
Finally, some critiques of the EA orthodoxy by heterodox folks, particularly ones that are a) highly prominent, b) seek external influence as their primary readers, and c) read as personal attacks, verge on mutiny (albeit poorly executed). Dylan Matthews 2015 article on EA Global arguably reads this way to me, as does attempts by Torres et.al of painting longtermism as an instantiation of white supremacy.
I’m interested in seeing people developing and clarifying this ontology further.
If I were to be critical of your post (and to be clear, this is stronger than what I actually believe), I’d argue that steering has overly positive associations in people’s heads (not least because Holden identifies EA as trying to do steering!). If people thought through the implications of how steering might happen in practice, when it may be hard to differentiate from the more negatively-connoted (and probably more negative in practice) anchoring, equity, or mutiny, then their emotional associations and conclusions w/r/t the rowing vs. steering debate might be somewhat more measured.
Hey Linch, thanks for this thoughtful comment!
Yeah, I agree that my examples of steering sometimes are closely related to other terms in Holden’s framework, particularly equity – indeed I have a comment about that buried deep in a footnote.
One reason I think this happens is because I think a super important concept for steering is the idea of moral uncertainty, and taking moral uncertainty seriously can imply putting a greater weight on equity than you otherwise might.
I guess another reason is that I tend to assume that effective steering is, as an empirical matter, more likely to be achieved if you incorporate a wide range of voices and perspectives. And this does in practice end up being similar to efforts to “amplify the voices and advance the interests of historically marginalized groups” that Holden puts under the category of equity. But yeah, like you say, it can be hard to differentiate whether people advocate for equity and diversity of perspectives for instrumental or intrinsic reasons (I’m keen on both).
I also think your last remark is a fair critique of my post – perhaps I did bring in some more controversial (though, to me, compelling!) perspectives under the less controversial heading of steering.
A very similar critique I’ve heard from two others is something like: “Is your argument purely that there isn’t enough steering going on, or is it also that you disagree with the current direction of steering?” And I think, to be fair, that it’s also partly the latter for me, at least on some very specific domains.
But one response to that is that, yes, I disagree with some of the current steering – but a necessary condition for changing direction is that people talk/care/focus more on steering, so I’m going to make the case for that first.
Thanks again for your comment!
Thanks for this post! I agree that the EA movement has a lot of momentum right now and “steering” is potentially relatively more important than “rowing”.
I think this is the biggest bottleneck I see in the EA community. Within EA, I see some concrete action based on core EA principles established over the last decade; I also see some criticism of EA or suggestions for how to do things differently. I’m most excited about the middle ground—”speeding up” efforts to move EA in a new direction.
For example, I think Phil Trammell did a great job outlining the idea of patient philanthropy [1], and I’m excited that Founder’s Pledge took the next step and created the Patient Philanthropy Fund [2]. WANBAM took concrete steps to address problems with diversity in the EA community. I’m not sure if these counts as steering or rowing.
[1] https://80000hours.org/podcast/episodes/phil-trammell-patient-philanthropy/
[2] https://founderspledge.com/funds/patient-philanthropy-fund
Thanks for your comment, much appreciated!
I wholeheartedly agree that taking action to do something is often the most important, and most desperately lacking, component. Why is it lacking?
One potential cause could be if many people agree with a critical take, but those people are not the ones who have a lot of influence, e.g. because decision-making power is concentrated.
Another explanation could be that there are actually many people who agree with a critical take on the direction of effective altruism and would have the ability to do something about it, but they just can’t/won’t dedicate time to it given their other professional commitments. (I count myself in that category. If I had a clone of myself that had to work on something other than biorisk, I might ask them to work full-time on ‘steering’ for this movement.)
Thankfully, we can expect a large influx of new, exciting members of the community (thanks to the awesome rowing of so many community builders!) looking for projects to take up. For that reason, I think it’s important that (1) there’s a culture of dissent that prompts people to think about new directions to pull the movement in, and (2) there are institutions in place that can facilitate the implementation of critical work, e.g. funding, positions within organisations, or mechanisms for distributed decision-making for the movement.
This recommendation appears briefly in the middle of a long post, but it seems like one of the most concrete things someone could do (and among the easiest).
Have you considered just doing this yourself by writing a separate post that highlights these two opportunities? (And maybe talking to Will/Jonas first to see if there are details they would add?)
Will’s comment might already be the most popular in the Forum’s history (not sure), but it wouldn’t hurt to have more eyeballs on his and Jonas’s comments! I’m very interested to see what projects people submit to them for funding.
Thanks, Aaron, this is a great suggestion! I’ll try to get around to writing a very brief post about it this weekend.
On a related note, I’d be curious to hear what you think of the idea of using EA Forum prizes for this sort of purpose? Of course, there’d have to be some more work on specifying what exactly the prize should be for, etc.
If you know who will be working on the Forum going forward, I’d love to get a sense of whether they’d be interested in doing some version of this. If so, I’d be more than happy to set up a meeting to discuss.
A competition for steering-type material seems like a reasonable contest theme (along the lines of the contests we had for creative writing and Wiki entries).
Now that I won’t be running any future events, I’m not sure what the best place to put ideas like this is. Perhaps a comment here (I imagine that future EA Forum leaders will check that thread when thinking about contests).
I’ve also added your idea to the document I’ll send our next Content Specialist, but that’s a really long document, so having the idea in more places seems good!
(Finally, when the next specialist is chosen, they’ll probably introduce themselves on the Forum, so you could set a reminder to contact them directly with this idea later on!)
Sounds good! I’ll post a comment and make sure to reach out to the next content specialist. Thanks!
<<I also know about a handful of people who have ‘jumped ship’; who, after spending many hundreds of hours working on effective altruism causes, have concluded that they are too disillusioned with the ship’s direction to stay on board and no longer wish to associate with the movement. These are not people who were never going to become highly engaged in the community in the first place. They are people who have contributed enormously to the effective altruism project and could have continued if they had been more confident in its ability to move in the right direction.>>
I’d love to hear more about this. What were these people’s concerns? What sorts of things are they doing now that seem better? These questions seem relevant both to rowing and steering.
(For context, I don’t know people like this. Maybe they know about important arguments I don’t. Should I jump ship too?)
First off, I think this is a really useful post that’s moved the discussion forward productively, and I agree with most of it.
I agree with the basic claim that steering is relatively neglected and that we should do more of it, so I’m much more curious about what current steering you disagree with/think we should do differently.
My view is closer to: most steering interventions are obvious, but they’ve ended up being most people’s second priority, and we should mostly just do much more of various things that are currently only occasionally done, or have been proposed but not carried out.
Most of the specific things you’ve suggested in this post I agree with. But you didn’t mention any specific current steering you thought was mistaken.
The way I naturally think of steering is in terms of making more sophisticated decisions: EA should be better at dealing with moral and empirical uncertainty in a rigorous and principled way. Here are some things that come to mind:
Talking more about moral uncertainty: I’d like to see more discussion of something like Ajeya’s concrete explicit world-view diversification framework, where you make sure you don’t go all-in and take actions that one worldview you’re considering would label catastrophic, even if you’re really confident in your preferred worldview—e.g. strong longtermism vs neartermism. I think taking this framework seriously would address a lot of the concerns people have with strong longtermism. From this perspective it’s natural to say that there’s a longtermist case for extinction risk mitigation based on total utilitarian potential and also a neartermist one based on a basket of moral views, and then we can say there’s clear and obvious interventions we can all get behind on either basis, along with speculative interventions that depend on your confidence in longtermism. Also, if we use a moral/‘worldview’ uncertainty framework, the justification for doing more research into how to prioritise different worldviews is easier to understand.
Better risk analysis: On the empirical uncertainty side, I very much agree with the specific criticism that longtermists should use more sophisticated risk/fault analysis methods when doing strategy work and forecasting (which was one of the improvements suggested in Carla’s paper). This is a good place to start on that. I think considering the potential backfire risks of particular interventions, along with how different X-risks and Risk factors interact, is a big part of this.
Soliciting external discussions and red-teaming: these seem like exactly the sorts of interventions that would throw up ways of better dealing with moral and empirical uncertainty, point out blindspots etc.
The part that makes me think we’re maybe thinking of different things is the focus on democratic feedback.
There are ways of reading this that make a lot of sense on the view of steering that I’m imagining here.
Under ‘more democratic feedback’: we might prefer to get elected governments and non-EA academics thinking about cause prioritisation and longtermism, without pushing our preferred interventions on them (because we expect this to help in pointing out mistakes, better interventions or things we’ve missed). I’ve also argued before that since common sense morality is a view we should care about, if we get to the point of recommending things that are massively at odds with CSM we should take that into account.
But if it’s something beyond all of these considerations, something like it’s intrinsically better when you’re doing things that lots of people agree with (and I realize this is a very fine distinction in practice!), arguing for more democratic feedback unconditionally looks more like Anchoring/Equity than Steering.
I think this would probably be cleared up a lot if we understood what specifically is being proposed by ‘democratic feedback’ - maybe it is just all the things I’ve listed, and I’d have no objections whatsoever!
Good post!
Some thoughts I had:
I noticed that part of me feels defensive of EA when there are criticisms. That part says something like “but EA is doing massively better than most social movements regarding steering”. But of course, that can be true while at the same time being true that “EA needs to become better at steering and invest marginal efforts into it rather than rowing”.
An interesting question is “how sensitive are outcomes to steering vs. rowing?” (Are we navigating towards a continent over open sea, or towards a treasure hidden between treacherous cliffs?) because that informs how obsessed we should be with either.
the absolute value of an outcome isn’t necessarily in proportion to the amount of power: it can be the case that if we don’t achieve a certain amount of power, key decision will go uninfluenced/wrong (e.g. if AGI decisions will be made centrally, we need A LOT of power to influence that)
I’d be curious to hear examples of movements/institutions that are/have been really good at steering, and how EA compares
I’m very interested in seeing more work and debate around what the optimal distribution of power in EA should be. I can see arguments both for narrow concentration and wide concentration being better for steering.
If I can expand the metaphor, I think EA is better thought of as a fleet than a single boat. This allows better for concepts such as chain of command, localized decision making, spreading out vs. staying together. One failure mode this could identify is “spreading out so much that inevitably some boats will do tremendous harm”
Formatting point—your link for ‘the long reflection’ seems to be broken here:
This was probably the intended link.
James, thanks for pointing this out, and thanks, Pablo, that was indeed the link I intended to use! Fixed it now.