I am grateful to Jessica McCurdy, Carla Zoe Cremer, Janvi Ahuja, Jonas Sandbrink, Kaleem Ahmid, and one anonymous reviewer for conversations and feedback that significantly improved this post. Any errors, of fact or judgment, remain my entirely own.
Introduction
In a recent blog post, Open Philanthropy Co-CEO Holden Karnofsky introduces the analogy of “the-world-as-a-ship” as a framework for thinking about efforts to make the world better.
Under this analogy, if you are ‘rowing’ the ship, you are trying to “help the ship reach its current destination faster.” By contrast, if you are ‘steering’ the ship, you aim to “navigate to a better destination than the current one.” Alternative options include anchoring, equity, and mutiny. It’s a terrific post introducing a practical framework for thinking about world-improving efforts such as those undertaken by the effective altruism and longtermism communities.
I think we can apply the analogy of the ship in a narrower sense to effective altruism and longtermist movement-building efforts.[1]In particular, I find that the notions of rowing and steering can elucidate some key differences in beliefs that I’ve recently observed in the community.
In this post, I briefly contrast what I see as the rowing and steering approaches to building the effective altruism movement and discuss success and failure from each perspective.
I present three arguments for why ‘steering’ is more important than is currently recognised: i) Given the momentum of the effective altruism and longtermism movements, steering is currently more neglected than rowing; ii) steering is more tractable if implemented sooner rather than later; and, iii) steering is valuable for community health.
I then offer concrete proposals for one form of steering that I’d like to see more of, namely good-faith critiques of the effective altruism philosophy and movement. Specifically, I suggest commissioning research on effective altruism and longtermism failure modes, offering Effective Altruism Forum prizes for good-faith critiques, funding, creating, and managing effective altruism ‘red teams’, and committing to reviewing and acting on valid critiques.
Defining rowing and steering
I’ll define ‘rowing for the effective altruism community’ roughly as ‘efforts to increase the power of the effective altruism movement.’ This can be achieved by recruiting or converting more people to support the cause, increasing the engagement of existing community members, obtaining more funds committed to the movement, creating powerful allies in government and industry, and so on.
By contrast, I’ll define ‘steering for the effective altruism community’ roughly as ‘efforts to improve the movement’s aims and/or to ensure that the movement executes on those specific aims.’ This can be achieved by researching global priorities, evaluating competing causes and interventions, critically examining and improving current cause-prioritisation methodologies, critiquing the direction of existing efforts, creating institutions and norms to enhance procedures and decision-making, and so on. Crucially, on my definition, steering isn’t just about pulling the ship in the direction you currently think is best. It’s also about questioning whether your navigational approach, crew composition, and equipment are adequate for the task. It’s about ensuring that, in the future, one can steer even better than the current crew, and the future crew is not hindered from pulling the ship in the direction they think is best. For example, it’s not just about doing global priorities research; it’s also about continually and rigorously asking whether your way of doing such research has any blind spots or drawbacks.
Rowers seek to accelerate the effective altruism ship; steerers seek to question and adjust its current direction.
What failure means for effective altruism under rowing and steering
Failure for rowers
My motivation for writing this post is a recurring observation that rowers and steerers tend to be concerned by different problems and challenges for effective altruism, which appear to arise from distinct (albeit mutually compatible) conceptions of what failure means for the movement.
According to the rowing perspective, effective altruism fails by not growing powerful enough, fast enough. For example, Nuno Sempere’s Forum post “Why do social movements fail: Two concrete examples” focuses on two social movements (the Spanish Enlightenment, 1750-1850, and the General Semantics movement,1938-2003) that seemingly failed in the sense of failing to persist. A more detailed version of the post’s title could be “Why do social movements fail [to be as powerful and lasting as they could be]” more so than “Why do social movements fail [to maintain good aims and careful execution of those aims]”. In other words, this analysis implicitly suggests that the effective altruism movement should learn from failures to grow in power – we should heed the lessons of failed rowers of the past. An alternative, steering version of this post would perhaps have considered how certain social movements (say, advocacy movements for socialism or capitalism) failed in the sense of leading to inadvertent negative consequences or in the sense of being co-opted by powerful interests.
One specific way that effective altruism might fail is by failing to avert an irreversible and extremely undesirable outcome, such as an existential catastrophe. In light of this, I can imagine that people’s views on things like AI timelines highly influence whether they lean towards a rowing or steering perspective. For example, suppose you believe that existential risks from artificial intelligence (or other sources) are imminent. In that case, this may lead to a strong emphasis on rowing: the movement needs to become very influential and very fast, lest it misses a slim window of opportunity to avert impending civilisational catastrophe.
According to the rowing perspective, critiques of effective altruism and longtermism can be highly concerning and frustrating, as they could impede movement growth. For example, such critiques may dampen the motivation of current and prospective effective altruism rowers – as far as I can tell, some Forum commenters have expressed concerns that a particular criticism of effective altruism or longtermism could have adverse effects on movement building. This issue could be particularly pressing in the early stages of a movement when its proponents have not yet fully defined terms like ‘longtermism’ in the minds of the broader public so that widely read critiques are more likely to shape initial impressions. Naturally, this concern will be especially salient when a critique is perceived as poor or even unfair. However, it is notable that the concern could also arise even for valid and reasonable critiques, insofar as such critiques may impose considerable costs by slowing down rowing efforts, notwithstanding whatever veracity and potential benefits. Consequently, the rowing mindset is particularly sympathetic to the idea that critiques such as thoseoffered by Phil Torres warrant vigorous responses, such as the detailed discussions offered by Avital Balwit and HaydnBelfield.
According to the rowing perspective, ‘pessimism about effective altruism’ typically means something like ‘pessimism about the movement’s pace of growth’ rather than ‘pessimism about its direction.’ For example, in a blogpost from 2020 titled Optimism about EA Longtermism, Stefan Schubert writes that while effective altruists are “sometimes pessimistic about how the movement is developing,” he thinks that there’s some reason for optimism “if we look at how EA long-termism has developed in the last years.” As the basis for this optimism, Schubert cites the following observations:
“Several new organisations that are aligned with or sympathetic to EA long-termism have been set up”
“Key EA ideas, such as AI safety and long-termism, have received increased support.”
“The community is maturing in many ways.”
“The community is attracting lots of very talented and reasonable people.”
“The overall trajectory seems pretty stable and robust. Success isn’t dependent on some daring moves. Instead, most big initiatives seem pretty safe.”
The emphasis of (1)[3], (2), and (4), seems to fit pretty squarely within rowing, while (3) feels more like steering and (5) depends on how you interpret “stable and robust.” The rowing perspective of the blogpost is clearest when Schubert wraps it up by writing: “One thing that does worry me a bit, however, is that we might not grow fast enough. EA long-termists may be able to change the long-term direction of the world, if we become sufficiently influential. So failing to do so could constitute a great opportunity cost. Hence, it may be important that EA long-termism grows at a good pace.” In other words, this seems like an example where a ship’s pace at least partly determines optimism or pessimism about it.
According to the rowing perspective, effective altruism community builders should focus on reaching as many people as possible who are the most “aligned” with current conceptions of effective altruism and encourage them to execute the movement’s current aims. Given the importance of achieving as many of those current aims as fast as possible, community builders focused on rowing may be concerned about including “unaligned” members of the movement, lest they slow down progress by concentrating on relatively unimportant goals with significant opportunity costs.
Failure for steerers
According to the steering perspective, failure modes for effective altruism include, at least, the three following categories:
Object-level failures concerning cause prioritisation, evaluation of global priorities, or interventions with unintended consequences, etc.
Community failures concerning culture, (epistemic) norms, homogeneity of worldviews or demographics, whether the movement should be large or narrow, etc.
Organisational failures concerning hierarchies, decision-procedures, transparency, degree of democracy, accountability of those with power, etc.
According to the steering perspective, failure arises not (only) when effective altruism fails to grow but when it grows in the wrong direction. Steerers listen with keen ears when the American poet J. Cole cautions: “the good news is (…) you came a long way // the bad news is (…), you went the wrong way.”
According to the steering perspective, “pessimism about effective altruism” is more likely to mean “pessimism about what the movement is doing.” Accordingly, it would take more than growth to become optimistic about effective altruism; it would also require signs that norms and institutions for careful navigation are being implemented.
According to the steering perspective, it is profoundly concerning if the movement prematurely converges on a narrow view of the best direction for the effective altruism ship. For example, my Future of Humanity Institute colleague Zoe Cremer makes an exceptionally compelling case against value-alignment between effective altruists and highlights a need to steer clear of certain pitfalls.
According to the steering perspective, the growing influence of the effective altruism movement could become a source of concern insofar as the movement is judged to be erring in serious ways. For example, when Phil Torres notes that “longtermism might be one of the most influential ideologies that few people outside of elite universities and Silicon Valley have ever heard about,” it seems that this should be interpreted as a word of caution rather than a celebration of success; that growth and influence only are successes insofar as one is convinced about the ship’s direction and navigation.
Because the steering perspective emphasises direction over speed, critiques such as these can be valuable in light of their potential to nudge the ship in the right direction. And just like the rower might be frustrated with critiques despite their merits, the steerer might welcome some critiques despite their flaws.
According to the steering perspective, community builders should focus on what the effective altruism community might be getting wrong and how their community-building efforts might remedy or exacerbate errors.
A brief case in favour of steering
My current vantage point likely shapes my inclinations towards rowing versus steering from the ship (see Addendum below). Having noted that, I would like to argue that, on the current margin, the effective altruism – and perhaps particularly the longtermism – community would benefit more from steering than from rowing. I recognise that the significant expansion of effective altruism and longtermism is not inevitable and that the opportunity costs of suboptimal growth rates are real. Still, I see at least three salient arguments for focusing more on steering in years to come:
1) Steering feels relatively neglected
It’s not feasible to quantify and directly compare the relative neglectedness of steering and rowing efforts. I also appreciate that many people are working hard on what I would categorise as steering. Those caveats aside, I think that one reason to focus more on steering is that, in many ways, progress for the effective altruism and longtermism movements seems to be going really quite well.
Over the coming decade, I expect philanthropists to commit and donate hundreds of millions of dollars to effective altruism causes, dozens of new organisations to be founded, thousands of individuals recruited to the movement, and numerous highly influential global decision-makers to become persuaded of its tenets. Moreover, the impression I get at lunchtime conversations around the Future of Humanity Institute, Global Priorities, Forethought Foundation, and Centre for Effective Altruism offices is that the focus much more often seems to be on “how can we get our vision implemented, quickly and powerfully?” rather than “how can we ensure that we have the right vision and that we implement it correctly before we push for it?” Perhaps one reason rowing is unlikely to be neglected is that it seems essential: being powerful is necessary to execute good aims. But the flip side is that steering can become neglected, as power can be obtained without having good aims.
In light of this, I would love to see more focus on the latter question, especially (though not exclusively) from people who influence the strategic decision-making in these communities. Again, I wish to recognise that many community leaders strongly support steering – e.g., by promoting ideas like ‘moral uncertainty’ and ‘the long reflection’ or via specific community-building activities. So, my argument here is not that steering currently doesn’t occur; rather, it doesn’t occur enough and should occur in more transparent and democratic ways.
2) Steering is more tractable earlier rather than later
I would argue that it will be easier to steer effective altruism in the right direction now, relative to how difficult it will be when the movement is, say, ten times as large and influential as it is now.
I think this applies strongly to steering aimed at improving community norms, dynamics, and health. For example, problems like intellectual or demographic homogeneity seem extremely self-reinforcing in many ways, meaning that they will only become harder to address once the movement has grown further. If so, this would make efforts to increase the currently dismal diversity of worldviews, styles of thinking, and demographics within the community highly urgent.
I also think something similar can apply for direct interventions and policy advocacy. Once you put something on the UN Secretary General’s desk or in the general public’s awareness, it can become hard to subsequently steer in a different direction than the one you initially laid out. I think we’ve seen this, in particular, with some of the early effective altruism ideas that originally were the objective of much rowing but no longer reflect the movement as well, such as earning to give or a stricter focus on interventions informed randomised controlled trials.
Perhaps there’s a lesson to be learned here from the private sector, where I get the impression that it is widely believed that organisational cultures must be carefully defined early on, as they are both highly influential and highly challenging to alter later on.
3) Steering is good for community health
Another reason to steer carefully is that some people, who could otherwise have been powerful rowers, will jump ship if they believe that the ship is going in the wrong direction and see too few efforts at correcting course.
I know several people who would fit any reasonable description of a “highly engaged EA” who privately and publicly question the extent to which they are comfortable identifying as members of the effective altruism community for this exact reason. I count myself in that category.
I also know about a handful of people who have ‘jumped ship’; who, after spending many hundreds of hours working on effective altruism causes, have concluded that they are too disillusioned with the ship’s direction to stay on board and no longer wish to associate with the movement. These are not people who were never going to become highly engaged in the community in the first place. They are people who have contributed enormously to the effective altruism project and could have continued if they had been more confident in its ability to move in the right direction.
Of course, different people are likely to have different visions for where to go, so it is somewhat unavoidable that any community would lose some people this way. Nevertheless, it is worth taking seriously that this happens and that a very concrete benefit of steering efforts is the retention of a wide range of highly capable community members.
Two steps for successful steering
Continually and critically evaluating the current course
Often, steering involves two steps: first, identifying what could be wrong with the current direction or the systems that produce navigational decisions, and second, acting to alleviate any identified problems. In practice, I think that the first step looks a lot like critiquing, or at least thinking critically about, the beliefs, values, norms, strategies, and tactics that predominate in the movement at any point in time. Accordingly, while I recognise that critiques may be unwelcome from a rowing perspective given their risk of being used as part of bad-faith opposition, I think that critique of effective altruism serves an essential role in steering.
Here’s a paradox I’ve sometimes felt in the effective altruism community: despite being exceptionally and impressively ardent about the importance of changing one’s position in response to novel evidence, and despite having unusually transparent and rigorous internal debates about even the smallest details, it sometimes seems that the community is not always great at taking seriously critiques coming from outsiders. [2] This is particularly the case if the critique is presented in different terminology or based on different methodologies than those that dominate the community (i.e., economics, computer science, evolutionary and behavioural psychology, analytic Western philosophy, physics, and so on).
More than once, I’ve seen peers outright dismiss a critique based on the thinnest of possible slices, such as the use of specific terminology or language associated with a certain ill-regarded academic discipline or social movement. In particular, I’ve more than once observed something being summarily dismissed based on using language associated with social justice movements, but I think the point holds more generally, too.[3]
Failing to heed critiques properly means missing opportunities to correct course and improve the movement’s direction. Ideally, we would become even better at taking outside critiques seriously, as they’re likely to raise considerations that could be missed by people already deeply involved. But in the meantime, I argue that proponents of effective altruism and longtermism should undertake much more concerted efforts to produce strong, good-faith critiques of their own worldviews, communities, and activities.
One benefit of such self-directed critiques is that they are more likely to be taken seriously if presented in a language and style of reasoning that community members tend to heed.
Another reason to produce good-faith critiques is to crowd out poor-quality critiques, allowing outside observers to form more accurate beliefs about what effective altruism and longtermism is and isn’t about.
Concrete interventions for improving effective altruism through critique
I would love to see more resources dedicated to professional steering. Here are some concrete interventions that I think could be great – but, importantly, this is not at all an exhaustive list, and I’m confident that better ideas exist:
Conduct and commission research on questions to critically guide the direction of the effective altruism and longtermism movement, such as:
What are current problems in the effective altruism funding landscape, and how might they be mitigated?
What are current problems in the effective altruism community culture, and how might they be mitigated?
What are current problems in the effective altruism strategic decision-making, and how might they be mitigated?
If longtermism went really wrong and caused a lot of harm, how might that happen? (Beware infohazards, of course)
How is the philosophy of longtermism most likely to be misunderstood by policymakers in the next three decades? How is it most likely to be misapplied?
What social movements have failed, not in the sense of not having an impact, but in the sense of having the wrong kind of impact? Why did that occur, and what can we learn?
Offer severalEffective Altruism Forum prizes for the best posts on topics like these.
Passively offer funding to support someone dedicated to critiquing effective altruism on their terms. Here are two promising first steps, which I’d love to see publicised much more widely.
Actively set up a 2-4 person “Effective Altruism Red Team”, for instance, as an official part of the Centre for Effective Altruism. In contrast to the above proposal, this would involve active line management, creating a red team research agenda, and actively managing common red teaming challenges (e.g., pressures to hold back particular critiques or adverse social dynamics). For such a team, two things seem crucial:
Actively supporting the implementation of interventions based on identified problems.
Creating mechanisms for soliciting and integrating wider community views, including surveys, referenda, and interviews.
Have a dedicated session at Effective Altruism Global conferences focusing on identifying and broadcasting how the movement could improve in terms of community culture and norms, decision-making processes, and overall strategy.
Go further in using the annual ‘Coordination Forum’ to improve community norms and decision-making processes, including consideration of crowd-sourced input. Improve the forum’s transparency by producing a summary output from this session for the community to digest and scrutinise.
Conduct or commission an annual review involving a concrete evaluation of whether sufficient action has been taken in response to any of the above, or other, sources of good-faith criticism.
One reason I’m excited about interventions like these is that I suspect (and hope) that there’s a lot more untapped ideas out in the community than you’d expect just based on the work that people produce. For example, my daytime job is to work on global catastrophic biological risks. This is what the Future of Humanity Institute pays for, and it’s what my career is shaping up to be all about. While I’m very excited by this, it means that thinking critically about how the effective altruism community could improve is relegated to the margins of my days. I know several colleagues who would love to see more time spent on steering our community but who themselves can only focus on it as a moonlighting activity. [4] Case-in-point: I wrote this forum post on my holidays rather than launching a fully-fledged project to nudge the community in one of the directions that I’d like to see it go. The upshot is that creating the right financial, social and institutional conditions (i.e., by providing funding, changing norms, and reforming events and organisations) could potentially leverage a large stock of existing thinking on how to improve effective altruism.
Taking action to correct course
As noted, the second step of steering is to act on identified navigational failure or failure modes. Accordingly, for all of these suggestions, I really must stress that even the best critique is worthless without action. Funding and praising someone for pointing out a harmful community norm or inadequate institution is only valuable alongside concrete steps to remedy the issues. A brilliant analysis of how policymakers might misunderstand or misapply longtermism is only useful insofar as longtermist policy advocates take these risks seriously and proactively mitigate them. And so on.
Conclusion
I believe there are solid reasons for wanting effective altruism to grow in influence, reach, and power. These include my belief that the movement has had an extraordinarily positive impact on this world and very likely will continue to and the risk that existential catastrophes could cause irreversible damage in the near future without the right interventions.
I also believe that it is extremely important to focus on improving the movement beyond increasing its influence, reach, and power. This includes:
refining methodologies for evaluating causes;
examining crucial considerations for current tenets of effective altruism;
improving the epistemic and social norms in the community;
facilitating intellectual, moral, and demographic diversity in the community; establishing new mechanisms for transparency and distributed decision-making;
worrying more about how longtermism could increase risks or go wrong;
and more.
I believe that good-faith critiques play a vital role in achieving these ends and that the movement should use more of its financial and human capital to facilitate such critical examination of effective altruism. I also believe that critical evaluation is only the first step of effective steering; actual implementation, reevaluation, and maintenance must follow.
Finally, I choose to remain optimistic. The effective altruism movement is based on inspiring and pro-social tenets and the effective altruism community includes some of the most thoughtful and caring people I have the privilege of knowing. While I see much room for improvement for both the movement and the community and am often deeply concerned about the prospects of realising those improvements, I maintain hope that this ship will find a good course and reach a safe harbour in time.
Addendum: Vantage points from the ship
As noted above, the rowing and steering perspectives are entirely mutually compatible. You can want the ship to go faster and want it to go in the right direction. Accordingly, very few people are pure rowers or steerers in practice and I genuinely believe that almost all members of this community are excited about progress in both steering and rowing. But, as with many other things, there can be a trade-off in how much to emphasise one perspective over the other, and focusing on one can be at the direct expense of the other. People differ quite a lot and I suspect that one’s vantage point can sometimes be relevant. For example, my first-hand experiences of the movement’s current momentum have increased my interest in steering.
I currently work at the Future of Humanity Institute, which shares offices with the Global Priorities Institute and the Centre for Effective Altruism. Here, my friend and colleagues work on activities you could categorise as both steering (such as global priorities research) and rowing (such as outreach, whether to academia, policymakers, or university students). Working and living in an environment where most of the people I am around – including those I live with – are heavily embedded in effective altruism and/or longtermism has given me the visceral sense that the ship is moving very fast. Every day, I talk with capable, hard-working people who are keen to dedicate their entire professional, and most of their personal, lives to the effective altruism cause and community. Every week, I hear conversations about the enormous overhang of funds committed to effective altruism and how easily the community could channel dizzying amounts of money towards any cause or project deemed valuable. I hear about the Secretary-General of the United Nations supporting longtermism, the Prime Minister of the United Kingdom quoting Toby Ord on the floor of the United Nations, and ambitions and concrete projects to get members of the movement into seats of power across the world. From this vantage point, steering is relatively more neglected than rowing: it sometimes seems that the ship inevitably will make progress, that its pace is both impressive and gaining, and that more people thus should think longer and harder about its direction as well as its navigational equipment. Consequently, I now spend much more of my time thinking about steering as opposed to rowing, and I’m much more likely to bring the former to the lunch table and water-cooler conversations than the latter.
But I am also mindful that, from some vantage points, the movement feels in dire need of some heavy rowing. I experience this in my professional work on global catastrophic biological risks. Here, so many stakeholders in biosecurity and public health seem either unaware of – or indifferent to – the vital insights around scope sensitivity, information hazards, and tail risks that effective altruism-oriented members of the field bring to the table. In that context, I saliently feel the need to row hard and swiftly; to bring more people on board, and proliferate existing ideas and strategies widely. I also remember what it was like to do community building on a college campus, to encounter hundreds of people who were either unsympathetic or indifferent to the tenets of effective altruism, to work tirelessly for months to recruit just one more highly engaged group member. This is probably why my previous posts on the Forum (an effective altruism fellowship guide, an interview with Ben Garfinkel of GovAI, and a syllabus on longtermism) were motivated mainly by rowing inclinations. From that vantage point, rowing should be prioritised heavily. Progress for this ship is not inevitable, the opportunity cost of anything less than the fastest pace is enormous – particularly if existential catastrophe could be imminent – and we’re already on such a solid course compared to any other political or social movement that we should unequivocally pick up the pace even further.
Addendum, March 27, 2022: Relevant links
Thanks to everyone who attended my talk on this subject at EAGx this weekend! As promised, here’s a collection of the examples I highlighted in the talk, and a few others:
I may not be using the terms exactly in the same way as Karnofsky does – in particular, I think my conception of steering often relates close to what he calls equity – and any conceptual errors or confusions are naturally entirely my own. For prospective commenters, I should note that I’m less interested in the semantics of whether my use of the terms maps perfectly onto Karnofsky’s and more interested in the substantive implications of rowing and steering as I define them.
I’m not sure why proponents of effective altruism would turn out to be especially poor at listening to critiques, when open-mindedness indeed seems so essential to its philosophy. If I had to guess, I suspect it has to do with the fact that effective altruism often involves thinking that you’ve spotted something true that the rest of the world hasn’t, such as more cost-effective interventions or more compelling moral views. To be clear, I profoundly do think that the community has done this – but the tricky part is to separate when one is right about an unusual view (and so can be more comfortable dismissing outgroup disagreement) and when one hasn’t (and so should heed outgroup advice). I think another contributing factor is the disciplinary homogeneity of the community, wherein most people come from a handful of academic backgrounds, leading some disciplines to be poorly understood, and thus undervalued, within the community.
Rowing and Steering the Effective Altruism Movement
Acknowledgements
I am grateful to Jessica McCurdy, Carla Zoe Cremer, Janvi Ahuja, Jonas Sandbrink, Kaleem Ahmid, and one anonymous reviewer for conversations and feedback that significantly improved this post. Any errors, of fact or judgment, remain my entirely own.
Introduction
In a recent blog post, Open Philanthropy Co-CEO Holden Karnofsky introduces the analogy of “the-world-as-a-ship” as a framework for thinking about efforts to make the world better.
Under this analogy, if you are ‘rowing’ the ship, you are trying to “help the ship reach its current destination faster.” By contrast, if you are ‘steering’ the ship, you aim to “navigate to a better destination than the current one.” Alternative options include anchoring, equity, and mutiny. It’s a terrific post introducing a practical framework for thinking about world-improving efforts such as those undertaken by the effective altruism and longtermism communities.
I think we can apply the analogy of the ship in a narrower sense to effective altruism and longtermist movement-building efforts.[1] In particular, I find that the notions of rowing and steering can elucidate some key differences in beliefs that I’ve recently observed in the community.
In this post, I briefly contrast what I see as the rowing and steering approaches to building the effective altruism movement and discuss success and failure from each perspective.
I present three arguments for why ‘steering’ is more important than is currently recognised: i) Given the momentum of the effective altruism and longtermism movements, steering is currently more neglected than rowing; ii) steering is more tractable if implemented sooner rather than later; and, iii) steering is valuable for community health.
I then offer concrete proposals for one form of steering that I’d like to see more of, namely good-faith critiques of the effective altruism philosophy and movement. Specifically, I suggest commissioning research on effective altruism and longtermism failure modes, offering Effective Altruism Forum prizes for good-faith critiques, funding, creating, and managing effective altruism ‘red teams’, and committing to reviewing and acting on valid critiques.
Defining rowing and steering
I’ll define ‘rowing for the effective altruism community’ roughly as ‘efforts to increase the power of the effective altruism movement.’ This can be achieved by recruiting or converting more people to support the cause, increasing the engagement of existing community members, obtaining more funds committed to the movement, creating powerful allies in government and industry, and so on.
By contrast, I’ll define ‘steering for the effective altruism community’ roughly as ‘efforts to improve the movement’s aims and/or to ensure that the movement executes on those specific aims.’ This can be achieved by researching global priorities, evaluating competing causes and interventions, critically examining and improving current cause-prioritisation methodologies, critiquing the direction of existing efforts, creating institutions and norms to enhance procedures and decision-making, and so on. Crucially, on my definition, steering isn’t just about pulling the ship in the direction you currently think is best. It’s also about questioning whether your navigational approach, crew composition, and equipment are adequate for the task. It’s about ensuring that, in the future, one can steer even better than the current crew, and the future crew is not hindered from pulling the ship in the direction they think is best. For example, it’s not just about doing global priorities research; it’s also about continually and rigorously asking whether your way of doing such research has any blind spots or drawbacks.
Rowers seek to accelerate the effective altruism ship; steerers seek to question and adjust its current direction.
What failure means for effective altruism under rowing and steering
Failure for rowers
My motivation for writing this post is a recurring observation that rowers and steerers tend to be concerned by different problems and challenges for effective altruism, which appear to arise from distinct (albeit mutually compatible) conceptions of what failure means for the movement.
According to the rowing perspective, effective altruism fails by not growing powerful enough, fast enough. For example, Nuno Sempere’s Forum post “Why do social movements fail: Two concrete examples” focuses on two social movements (the Spanish Enlightenment, 1750-1850, and the General Semantics movement,1938-2003) that seemingly failed in the sense of failing to persist. A more detailed version of the post’s title could be “Why do social movements fail [to be as powerful and lasting as they could be]” more so than “Why do social movements fail [to maintain good aims and careful execution of those aims]”. In other words, this analysis implicitly suggests that the effective altruism movement should learn from failures to grow in power – we should heed the lessons of failed rowers of the past. An alternative, steering version of this post would perhaps have considered how certain social movements (say, advocacy movements for socialism or capitalism) failed in the sense of leading to inadvertent negative consequences or in the sense of being co-opted by powerful interests.
One specific way that effective altruism might fail is by failing to avert an irreversible and extremely undesirable outcome, such as an existential catastrophe. In light of this, I can imagine that people’s views on things like AI timelines highly influence whether they lean towards a rowing or steering perspective. For example, suppose you believe that existential risks from artificial intelligence (or other sources) are imminent. In that case, this may lead to a strong emphasis on rowing: the movement needs to become very influential and very fast, lest it misses a slim window of opportunity to avert impending civilisational catastrophe.
According to the rowing perspective, critiques of effective altruism and longtermism can be highly concerning and frustrating, as they could impede movement growth. For example, such critiques may dampen the motivation of current and prospective effective altruism rowers – as far as I can tell, some Forum commenters have expressed concerns that a particular criticism of effective altruism or longtermism could have adverse effects on movement building. This issue could be particularly pressing in the early stages of a movement when its proponents have not yet fully defined terms like ‘longtermism’ in the minds of the broader public so that widely read critiques are more likely to shape initial impressions. Naturally, this concern will be especially salient when a critique is perceived as poor or even unfair. However, it is notable that the concern could also arise even for valid and reasonable critiques, insofar as such critiques may impose considerable costs by slowing down rowing efforts, notwithstanding whatever veracity and potential benefits. Consequently, the rowing mindset is particularly sympathetic to the idea that critiques such as those offered by Phil Torres warrant vigorous responses, such as the detailed discussions offered by Avital Balwit and HaydnBelfield.
According to the rowing perspective, ‘pessimism about effective altruism’ typically means something like ‘pessimism about the movement’s pace of growth’ rather than ‘pessimism about its direction.’ For example, in a blogpost from 2020 titled Optimism about EA Longtermism, Stefan Schubert writes that while effective altruists are “sometimes pessimistic about how the movement is developing,” he thinks that there’s some reason for optimism “if we look at how EA long-termism has developed in the last years.” As the basis for this optimism, Schubert cites the following observations:
“Several new organisations that are aligned with or sympathetic to EA long-termism have been set up”
“Key EA ideas, such as AI safety and long-termism, have received increased support.”
“The community is maturing in many ways.”
“The community is attracting lots of very talented and reasonable people.”
“The overall trajectory seems pretty stable and robust. Success isn’t dependent on some daring moves. Instead, most big initiatives seem pretty safe.”
The emphasis of (1)[3], (2), and (4), seems to fit pretty squarely within rowing, while (3) feels more like steering and (5) depends on how you interpret “stable and robust.” The rowing perspective of the blogpost is clearest when Schubert wraps it up by writing: “One thing that does worry me a bit, however, is that we might not grow fast enough. EA long-termists may be able to change the long-term direction of the world, if we become sufficiently influential. So failing to do so could constitute a great opportunity cost. Hence, it may be important that EA long-termism grows at a good pace.” In other words, this seems like an example where a ship’s pace at least partly determines optimism or pessimism about it.
According to the rowing perspective, effective altruism community builders should focus on reaching as many people as possible who are the most “aligned” with current conceptions of effective altruism and encourage them to execute the movement’s current aims. Given the importance of achieving as many of those current aims as fast as possible, community builders focused on rowing may be concerned about including “unaligned” members of the movement, lest they slow down progress by concentrating on relatively unimportant goals with significant opportunity costs.
Failure for steerers
According to the steering perspective, failure modes for effective altruism include, at least, the three following categories:
Object-level failures concerning cause prioritisation, evaluation of global priorities, or interventions with unintended consequences, etc.
Community failures concerning culture, (epistemic) norms, homogeneity of worldviews or demographics, whether the movement should be large or narrow, etc.
Organisational failures concerning hierarchies, decision-procedures, transparency, degree of democracy, accountability of those with power, etc.
According to the steering perspective, failure arises not (only) when effective altruism fails to grow but when it grows in the wrong direction. Steerers listen with keen ears when the American poet J. Cole cautions: “the good news is (…) you came a long way // the bad news is (…), you went the wrong way.”
According to the steering perspective, “pessimism about effective altruism” is more likely to mean “pessimism about what the movement is doing.” Accordingly, it would take more than growth to become optimistic about effective altruism; it would also require signs that norms and institutions for careful navigation are being implemented.
According to the steering perspective, it is profoundly concerning if the movement prematurely converges on a narrow view of the best direction for the effective altruism ship. For example, my Future of Humanity Institute colleague Zoe Cremer makes an exceptionally compelling case against value-alignment between effective altruists and highlights a need to steer clear of certain pitfalls.
According to the steering perspective, the effective altruism and longtermism communities must be diligent about procedural errors with potentially grave consequences. For example, Linch Zhang (of Rethink Priorities) highlights the threat of motivated reasoning to effective altruism efforts.
According to the steering perspective, the growing influence of the effective altruism movement could become a source of concern insofar as the movement is judged to be erring in serious ways. For example, when Phil Torres notes that “longtermism might be one of the most influential ideologies that few people outside of elite universities and Silicon Valley have ever heard about,” it seems that this should be interpreted as a word of caution rather than a celebration of success; that growth and influence only are successes insofar as one is convinced about the ship’s direction and navigation.
Because the steering perspective emphasises direction over speed, critiques such as these can be valuable in light of their potential to nudge the ship in the right direction. And just like the rower might be frustrated with critiques despite their merits, the steerer might welcome some critiques despite their flaws.
According to the steering perspective, community builders should focus on what the effective altruism community might be getting wrong and how their community-building efforts might remedy or exacerbate errors.
A brief case in favour of steering
My current vantage point likely shapes my inclinations towards rowing versus steering from the ship (see Addendum below). Having noted that, I would like to argue that, on the current margin, the effective altruism – and perhaps particularly the longtermism – community would benefit more from steering than from rowing. I recognise that the significant expansion of effective altruism and longtermism is not inevitable and that the opportunity costs of suboptimal growth rates are real. Still, I see at least three salient arguments for focusing more on steering in years to come:
1) Steering feels relatively neglected
It’s not feasible to quantify and directly compare the relative neglectedness of steering and rowing efforts. I also appreciate that many people are working hard on what I would categorise as steering. Those caveats aside, I think that one reason to focus more on steering is that, in many ways, progress for the effective altruism and longtermism movements seems to be going really quite well.
Over the coming decade, I expect philanthropists to commit and donate hundreds of millions of dollars to effective altruism causes, dozens of new organisations to be founded, thousands of individuals recruited to the movement, and numerous highly influential global decision-makers to become persuaded of its tenets. Moreover, the impression I get at lunchtime conversations around the Future of Humanity Institute, Global Priorities, Forethought Foundation, and Centre for Effective Altruism offices is that the focus much more often seems to be on “how can we get our vision implemented, quickly and powerfully?” rather than “how can we ensure that we have the right vision and that we implement it correctly before we push for it?” Perhaps one reason rowing is unlikely to be neglected is that it seems essential: being powerful is necessary to execute good aims. But the flip side is that steering can become neglected, as power can be obtained without having good aims.
In light of this, I would love to see more focus on the latter question, especially (though not exclusively) from people who influence the strategic decision-making in these communities. Again, I wish to recognise that many community leaders strongly support steering – e.g., by promoting ideas like ‘moral uncertainty’ and ‘the long reflection’ or via specific community-building activities. So, my argument here is not that steering currently doesn’t occur; rather, it doesn’t occur enough and should occur in more transparent and democratic ways.
2) Steering is more tractable earlier rather than later
I would argue that it will be easier to steer effective altruism in the right direction now, relative to how difficult it will be when the movement is, say, ten times as large and influential as it is now.
I think this applies strongly to steering aimed at improving community norms, dynamics, and health. For example, problems like intellectual or demographic homogeneity seem extremely self-reinforcing in many ways, meaning that they will only become harder to address once the movement has grown further. If so, this would make efforts to increase the currently dismal diversity of worldviews, styles of thinking, and demographics within the community highly urgent.
I also think something similar can apply for direct interventions and policy advocacy. Once you put something on the UN Secretary General’s desk or in the general public’s awareness, it can become hard to subsequently steer in a different direction than the one you initially laid out. I think we’ve seen this, in particular, with some of the early effective altruism ideas that originally were the objective of much rowing but no longer reflect the movement as well, such as earning to give or a stricter focus on interventions informed randomised controlled trials.
Perhaps there’s a lesson to be learned here from the private sector, where I get the impression that it is widely believed that organisational cultures must be carefully defined early on, as they are both highly influential and highly challenging to alter later on.
3) Steering is good for community health
Another reason to steer carefully is that some people, who could otherwise have been powerful rowers, will jump ship if they believe that the ship is going in the wrong direction and see too few efforts at correcting course.
I know several people who would fit any reasonable description of a “highly engaged EA” who privately and publicly question the extent to which they are comfortable identifying as members of the effective altruism community for this exact reason. I count myself in that category.
I also know about a handful of people who have ‘jumped ship’; who, after spending many hundreds of hours working on effective altruism causes, have concluded that they are too disillusioned with the ship’s direction to stay on board and no longer wish to associate with the movement. These are not people who were never going to become highly engaged in the community in the first place. They are people who have contributed enormously to the effective altruism project and could have continued if they had been more confident in its ability to move in the right direction.
Of course, different people are likely to have different visions for where to go, so it is somewhat unavoidable that any community would lose some people this way. Nevertheless, it is worth taking seriously that this happens and that a very concrete benefit of steering efforts is the retention of a wide range of highly capable community members.
Two steps for successful steering
Continually and critically evaluating the current course
Often, steering involves two steps: first, identifying what could be wrong with the current direction or the systems that produce navigational decisions, and second, acting to alleviate any identified problems. In practice, I think that the first step looks a lot like critiquing, or at least thinking critically about, the beliefs, values, norms, strategies, and tactics that predominate in the movement at any point in time. Accordingly, while I recognise that critiques may be unwelcome from a rowing perspective given their risk of being used as part of bad-faith opposition, I think that critique of effective altruism serves an essential role in steering.
Here’s a paradox I’ve sometimes felt in the effective altruism community: despite being exceptionally and impressively ardent about the importance of changing one’s position in response to novel evidence, and despite having unusually transparent and rigorous internal debates about even the smallest details, it sometimes seems that the community is not always great at taking seriously critiques coming from outsiders. [2] This is particularly the case if the critique is presented in different terminology or based on different methodologies than those that dominate the community (i.e., economics, computer science, evolutionary and behavioural psychology, analytic Western philosophy, physics, and so on).
More than once, I’ve seen peers outright dismiss a critique based on the thinnest of possible slices, such as the use of specific terminology or language associated with a certain ill-regarded academic discipline or social movement. In particular, I’ve more than once observed something being summarily dismissed based on using language associated with social justice movements, but I think the point holds more generally, too.[3]
Failing to heed critiques properly means missing opportunities to correct course and improve the movement’s direction. Ideally, we would become even better at taking outside critiques seriously, as they’re likely to raise considerations that could be missed by people already deeply involved. But in the meantime, I argue that proponents of effective altruism and longtermism should undertake much more concerted efforts to produce strong, good-faith critiques of their own worldviews, communities, and activities.
One benefit of such self-directed critiques is that they are more likely to be taken seriously if presented in a language and style of reasoning that community members tend to heed.
Another reason to produce good-faith critiques is to crowd out poor-quality critiques, allowing outside observers to form more accurate beliefs about what effective altruism and longtermism is and isn’t about.
Concrete interventions for improving effective altruism through critique
I would love to see more resources dedicated to professional steering. Here are some concrete interventions that I think could be great – but, importantly, this is not at all an exhaustive list, and I’m confident that better ideas exist:
Conduct and commission research on questions to critically guide the direction of the effective altruism and longtermism movement, such as:
What are current problems in the effective altruism funding landscape, and how might they be mitigated?
What are current problems in the effective altruism community culture, and how might they be mitigated?
What are current problems in the effective altruism strategic decision-making, and how might they be mitigated?
If longtermism went really wrong and caused a lot of harm, how might that happen? (Beware infohazards, of course)
How is the philosophy of longtermism most likely to be misunderstood by policymakers in the next three decades? How is it most likely to be misapplied?
What social movements have failed, not in the sense of not having an impact, but in the sense of having the wrong kind of impact? Why did that occur, and what can we learn?
Offer several Effective Altruism Forum prizes for the best posts on topics like these.
Passively offer funding to support someone dedicated to critiquing effective altruism on their terms. Here are two promising first steps, which I’d love to see publicised much more widely.
Actively set up a 2-4 person “Effective Altruism Red Team”, for instance, as an official part of the Centre for Effective Altruism. In contrast to the above proposal, this would involve active line management, creating a red team research agenda, and actively managing common red teaming challenges (e.g., pressures to hold back particular critiques or adverse social dynamics). For such a team, two things seem crucial:
Actively supporting the implementation of interventions based on identified problems.
Creating mechanisms for soliciting and integrating wider community views, including surveys, referenda, and interviews.
Have a dedicated session at Effective Altruism Global conferences focusing on identifying and broadcasting how the movement could improve in terms of community culture and norms, decision-making processes, and overall strategy.
Go further in using the annual ‘Coordination Forum’ to improve community norms and decision-making processes, including consideration of crowd-sourced input. Improve the forum’s transparency by producing a summary output from this session for the community to digest and scrutinise.
Conduct or commission an annual review involving a concrete evaluation of whether sufficient action has been taken in response to any of the above, or other, sources of good-faith criticism.
One reason I’m excited about interventions like these is that I suspect (and hope) that there’s a lot more untapped ideas out in the community than you’d expect just based on the work that people produce. For example, my daytime job is to work on global catastrophic biological risks. This is what the Future of Humanity Institute pays for, and it’s what my career is shaping up to be all about. While I’m very excited by this, it means that thinking critically about how the effective altruism community could improve is relegated to the margins of my days. I know several colleagues who would love to see more time spent on steering our community but who themselves can only focus on it as a moonlighting activity. [4] Case-in-point: I wrote this forum post on my holidays rather than launching a fully-fledged project to nudge the community in one of the directions that I’d like to see it go. The upshot is that creating the right financial, social and institutional conditions (i.e., by providing funding, changing norms, and reforming events and organisations) could potentially leverage a large stock of existing thinking on how to improve effective altruism.
Taking action to correct course
As noted, the second step of steering is to act on identified navigational failure or failure modes. Accordingly, for all of these suggestions, I really must stress that even the best critique is worthless without action. Funding and praising someone for pointing out a harmful community norm or inadequate institution is only valuable alongside concrete steps to remedy the issues. A brilliant analysis of how policymakers might misunderstand or misapply longtermism is only useful insofar as longtermist policy advocates take these risks seriously and proactively mitigate them. And so on.
Conclusion
I believe there are solid reasons for wanting effective altruism to grow in influence, reach, and power. These include my belief that the movement has had an extraordinarily positive impact on this world and very likely will continue to and the risk that existential catastrophes could cause irreversible damage in the near future without the right interventions.
I also believe that it is extremely important to focus on improving the movement beyond increasing its influence, reach, and power. This includes:
refining methodologies for evaluating causes;
examining crucial considerations for current tenets of effective altruism;
improving the epistemic and social norms in the community;
facilitating intellectual, moral, and demographic diversity in the community; establishing new mechanisms for transparency and distributed decision-making;
worrying more about how longtermism could increase risks or go wrong;
and more.
I believe that good-faith critiques play a vital role in achieving these ends and that the movement should use more of its financial and human capital to facilitate such critical examination of effective altruism. I also believe that critical evaluation is only the first step of effective steering; actual implementation, reevaluation, and maintenance must follow.
Finally, I choose to remain optimistic. The effective altruism movement is based on inspiring and pro-social tenets and the effective altruism community includes some of the most thoughtful and caring people I have the privilege of knowing. While I see much room for improvement for both the movement and the community and am often deeply concerned about the prospects of realising those improvements, I maintain hope that this ship will find a good course and reach a safe harbour in time.
Addendum: Vantage points from the ship
As noted above, the rowing and steering perspectives are entirely mutually compatible. You can want the ship to go faster and want it to go in the right direction. Accordingly, very few people are pure rowers or steerers in practice and I genuinely believe that almost all members of this community are excited about progress in both steering and rowing. But, as with many other things, there can be a trade-off in how much to emphasise one perspective over the other, and focusing on one can be at the direct expense of the other. People differ quite a lot and I suspect that one’s vantage point can sometimes be relevant. For example, my first-hand experiences of the movement’s current momentum have increased my interest in steering.
I currently work at the Future of Humanity Institute, which shares offices with the Global Priorities Institute and the Centre for Effective Altruism. Here, my friend and colleagues work on activities you could categorise as both steering (such as global priorities research) and rowing (such as outreach, whether to academia, policymakers, or university students). Working and living in an environment where most of the people I am around – including those I live with – are heavily embedded in effective altruism and/or longtermism has given me the visceral sense that the ship is moving very fast. Every day, I talk with capable, hard-working people who are keen to dedicate their entire professional, and most of their personal, lives to the effective altruism cause and community. Every week, I hear conversations about the enormous overhang of funds committed to effective altruism and how easily the community could channel dizzying amounts of money towards any cause or project deemed valuable. I hear about the Secretary-General of the United Nations supporting longtermism, the Prime Minister of the United Kingdom quoting Toby Ord on the floor of the United Nations, and ambitions and concrete projects to get members of the movement into seats of power across the world. From this vantage point, steering is relatively more neglected than rowing: it sometimes seems that the ship inevitably will make progress, that its pace is both impressive and gaining, and that more people thus should think longer and harder about its direction as well as its navigational equipment. Consequently, I now spend much more of my time thinking about steering as opposed to rowing, and I’m much more likely to bring the former to the lunch table and water-cooler conversations than the latter.
But I am also mindful that, from some vantage points, the movement feels in dire need of some heavy rowing. I experience this in my professional work on global catastrophic biological risks. Here, so many stakeholders in biosecurity and public health seem either unaware of – or indifferent to – the vital insights around scope sensitivity, information hazards, and tail risks that effective altruism-oriented members of the field bring to the table. In that context, I saliently feel the need to row hard and swiftly; to bring more people on board, and proliferate existing ideas and strategies widely. I also remember what it was like to do community building on a college campus, to encounter hundreds of people who were either unsympathetic or indifferent to the tenets of effective altruism, to work tirelessly for months to recruit just one more highly engaged group member. This is probably why my previous posts on the Forum (an effective altruism fellowship guide, an interview with Ben Garfinkel of GovAI, and a syllabus on longtermism) were motivated mainly by rowing inclinations. From that vantage point, rowing should be prioritised heavily. Progress for this ship is not inevitable, the opportunity cost of anything less than the fastest pace is enormous – particularly if existential catastrophe could be imminent – and we’re already on such a solid course compared to any other political or social movement that we should unequivocally pick up the pace even further.
Addendum, March 27, 2022: Relevant links
Thanks to everyone who attended my talk on this subject at EAGx this weekend! As promised, here’s a collection of the examples I highlighted in the talk, and a few others:
Announcements
Pre-announcing a contest for critiques and red teaming
Apply for Red Team Challenge [May 7 - June 4]
Highlighted critiques
Normative assumptions & moral values:
AI Alignment, Philosophical Pluralism, and the Relevance of Non-Western Philosophy
Constraining longtermism (Working paper, likely available upon request)
The only ethical argument for positive 𝛿?
On two arguments for Fanaticism
Empirical assumptions & theories of change
How sure are we about this AI stuff?
Stop the Robot Apocalypse
Why I am probably not a longtermist
The case of the missing cause prioritisation research
Institutions & organisations in the effective altruism movement
Why I’m concerned about Giving Green
The EA Community and Long-Term Future Funds Lack Transparency and Accountability
Social norms & practices in the effective altruism community
Objections to value-alignment between effective altruists
In defence of epistemic modesty
The motivated reasoning critique of effective altruism
I may not be using the terms exactly in the same way as Karnofsky does – in particular, I think my conception of steering often relates close to what he calls equity – and any conceptual errors or confusions are naturally entirely my own. For prospective commenters, I should note that I’m less interested in the semantics of whether my use of the terms maps perfectly onto Karnofsky’s and more interested in the substantive implications of rowing and steering as I define them.
Or even vigorous critiques from insiders, as highlighted in a recent post by Zoe Cremer and Luke Kemp.
I’m not sure why proponents of effective altruism would turn out to be especially poor at listening to critiques, when open-mindedness indeed seems so essential to its philosophy. If I had to guess, I suspect it has to do with the fact that effective altruism often involves thinking that you’ve spotted something true that the rest of the world hasn’t, such as more cost-effective interventions or more compelling moral views. To be clear, I profoundly do think that the community has done this – but the tricky part is to separate when one is right about an unusual view (and so can be more comfortable dismissing outgroup disagreement) and when one hasn’t (and so should heed outgroup advice). I think another contributing factor is the disciplinary homogeneity of the community, wherein most people come from a handful of academic backgrounds, leading some disciplines to be poorly understood, and thus undervalued, within the community.
Here is another recent example of someone who seems interested in such topics but doesn’t have the time to pursue them more systematically