Doing good while clueless

[Milan_Griffes]

This is the fourth (and final) post in a series exploring consequentialist cluelessness and its implications for effective altruism:

• The first post describes cluelessness & its relevance to EA; arguing that for many popular EA interventions we don’t have a clue about the intervention’s overall net impact.

• The second post considers a potential reply to concerns about cluelessness.

• The third post examines how tractable cluelessness is – to what extent we can grow more clueful about an intervention through intentional effort?

• This post discusses how we might do good while being clueless to an important extent.

Consider reading the previous posts (1, 2, 3) first.

---

The last post looked at whether we could grow more clueful by intentional effort. It concluded that, for the foreseeable future, we will probably remain clueless about the long-run impacts of our actions to a meaningful extent, even after taking measures to improve our understanding and foresight.

Given this state of affairs, we should act cautiously when trying to do good. This post outlines a framework for doing good while being clueless, then looks at what this framework implies about current EA cause prioritization.

The following only make sense if you already believe that the far future matters a lot; this argument has been made elegantly elsewhere so we won’t rehash it here.[1]

An analogy: interstellar travel

Consider a spacecraft, journeying out into space. The occupants of the craft are searching for a star system to settle. Promising destination systems are all very far away, and the voyagers don’t have a complete map of how to get to any of them. Indeed, they know very little about the space they will travel through.

To have a good journey, the voyagers will have to successfully steer their ship (both literally & metaphorically). Let’s use “steering capacity” as an umbrella term that refers to the capacity needed to have a successful journey.[2] “Steering capacity” can be broken down into the following five attributes:[3]

• The voyagers must have a clear idea of what they are looking for. (Intent)

• The voyagers must be able to reach agreement about where to go. (Coordination)

• The voyagers must be discerning enough to identify promising systems as promising, when they encounter them. Similarly, they must be discerning enough to accurately identify threats & obstacles. (Wisdom)

• Their craft must be powerful enough to reach the destinations they choose. (Capability)

• Because the voyagers travel through unmapped territory, they must be able to see far enough ahead to avoid obstacles they encounter. (Predictive power)

This spacecraft is a useful analogy for thinking about our civilization’s trajectory. Like us, the space voyagers are somewhat clueless – they don’t know quite where they should go (though they can make guesses), and they don’t know how to get there (though they can plot a course and make adjustments along the way).

The five attributes given above – intent, coordination, wisdom, capability, and predictive power – determine how successful the space voyagers will be in arriving at a suitable destination system. These same attributes can also serve as a useful framework for considering which altruistic interventions we should prioritize, given our present situation.

The basic point

The basic point here is that interventions whose main known effects do not improve our steering capacity (i.e. our intent, wisdom, coordination, capability, and predictive power) are not as important as interventions whose main known effects do improve these attributes.

An implication of this is that interventions whose effectiveness is driven mainly by their proximate impacts are less important than interventions whose effectiveness is driven mainly by increasing our steering capacity.

This is because any action we take is going to have indirect & long-run consequences that bear on our civilization’s trajectory. Many of the long-run consequences of our actions are unknown, so the future is unpredictable. Therefore, we ought to prioritize interventions that improve the wisdom, capability, and coordination of future actors, so that they are better positioned to address future problems that we did not foresee.

What being clueless means for altruistic prioritization

I think the steering capacity framework implies a portfolio approach to doing good – simultaneously pursuing a large number of diverse hypotheses about how to do good, provided that each approach maintains reversibility.[4]

This approach is similar to the Open Philanthropy Project’s hits-based giving framework – invest in many promising initiatives with the expectation that most will fail.

Below, I look at how this framework interacts with focus areas that effective altruists are already working on. Other causes that EA has not looked into closely (e.g. improving education) may also perform well under this framework; assessing causes of this sort is beyond the scope of this essay.

My thinking here is preliminary, and very probably contains errors & oversights.

EA focus areas to prioritize

Broadly speaking, the steering capacity framework suggests prioritizing interventions that:[5]

• Further our understanding of what matters

• Improve governance

• Improve prediction-making & foresight

• Reduce existential risk

• Increase the number of well-intentioned, highly capable people

To prioritize – better understanding what matters

Increasing our understanding of what’s worth caring about is important for clarifying our intentions about what trajectories to aim for. For many moral questions, there is already broad agreement in the EA community (e.g. the view that all currently existing human lives matter is uncontroversial within EA). On other questions, further thinking would be valuable (e.g. how best to compare human lives to the lives of animals).

Myriad thinkers have done valuable work on this question. Particularly worth mentioning is the work of the Foundational Research Institute, the Global Priorities Project, the Qualia Research Institute, as well the Open Philanthropy Project’s work on consciousness & moral patienthood.

To prioritize – improving governance

Improving governance is largely aimed at improving coordination – our ability to mediate diverse preferences, decide on collectively held goals, and work together towards those goals.

Efficient governance institutions are robustly useful in that they keep focus oriented on solving important problems & minimize resource expenditure on zero-sum competitive signaling.

Two routes towards improved governance seem promising: (1) improving the functioning of existing institutions, and (2) experimenting with alternative institutional structures (Robin Hanson’s futarchy proposal and seasteading initiatives are examples here).

To prioritize – improving foresight

Improving foresight & prediction-making ability is important for informing our decisions. The further we can see down the path, the more information we can incorporate into our decision-making, which in turn leads to higher quality outcomes with fewer surprises.

Forecasting ability can definitely be improved from baseline, but there are probably hard limits on how far into the future we can extend our predictions while remaining believable.

Philip Tetlock’s Good Judgment Project is a promising forecasting intervention, as are prediction markets like PredictIt and polling aggregators like 538.

To prioritize – reducing existential risk

Reducing existential risk can be framed as “avoiding large obstacles that lie ahead.” Avoiding extinction and “lock-in” of suboptimal states is necessary for realizing the full potential benefit of the future.

Many initiatives are underway in the x-risk reduction cause area. Larks’ annual review of AI safety work is excellent; Open Phil has good material about projects focused on other x-risks.

To prioritize – increase the number of well-intentioned, highly capable people

Well-intentioned, highly capable people are a scarce resource, and will almost certainly continue to be highly useful going forward. Increasing the number of well-intentioned, highly capable people seems robustly good, as such people are able to diagnosis & coordinate together on future problems as they arise.

Projects like CFAR and SPARC are in this category.

In a different vein, psychedelic experiences hold promise as a treatment for treatment-resistant depression, and may also improve the intentions of highly capable people who have not reflected much about what matters (“the betterment of well people”).

EA focus areas to deprioritize, maybe

The steering capacity framework suggests deprioritizing animal welfare & global health interventions, to the extent that these interventions’ effectiveness is driven by their proximate impacts.

Under this framework, prioritizing animal welfare & global health interventions may be justified, but only on the basis of improving our intent, wisdom, coordination, capability, or predictive power.

To deprioritize, maybe – animal welfare

To the extent that animal welfare interventions expand our civilization’s moral circle, they may hold promise as interventions that improve our intentions & understanding of what matters (the Sentience Institute is doing work along this line).

However, following this framework, the case for animal welfare interventions has to be made on these grounds, not on the basis of cost-effectively reducing animal suffering in the present.

This is because the animals that are helped in such interventions cannot help “steer the ship” – they cannot contribute to making sure that our civilization’s trajectory is headed in a good direction.

To deprioritize, maybe – global health

To the extent that global health interventions improve coordination, or reduce x-risk by increasing socio-political stability, they may hold promise under the steering capacity framework.

However, the case for global health interventions would have to be made on the grounds of increasing coordination, reducing x-risk, or improving another steering capacity attribute. Arguments for global health interventions on the grounds that they cost-effectively help people in the present day (without consideration of how this bears on our future trajectory) are not competitive under this framework.

Conclusion

In sum, I think the fact that we are intractably clueless implies a portfolio approach to doing good – pursuing, in parallel, a large number of diverse hypotheses about how to do good.

Interventions that improve our understanding of what matters, improve governance, improve prediction-making ability, reduce existential risk, and increase the number of well-intentioned, highly capable people are all promising. Global health & animal welfare interventions may hold promise as well, but the case for these cause areas needs to be made on the basis of improving our steering capacity, not on the basis of their proximate impacts.

Thanks to members of the Mather essay discussion group and an anonymous collaborator for thoughtful feedback on drafts of this post. Views expressed above are my own. Cross-posted to LessWrong & my personal blog.

---

Footnotes

[1]: Nick Beckstead has done the best work I know of on the topic of why the far future matters. This post is a good introduction; for a more in-depth treatment see his PhD thesis, On the Overwhelming Importance of Shaping the Far Future.

[2]: I’m grateful to Ben Hoffman for discussion that fleshed out the “steering capacity” concept; see this comment thread.

[3]: Note that this list of attributes is not exhaustive & this metaphor isn’t perfect. I’ve found the space travel metaphor useful for thinking about cause prioritization given our uncertainty about the far future, so am deploying it here.

[4]: Maintaining reversibility is important because given our cluelessness, we are unsure of the net impact of any action. When uncertain about overall impact, it’s important to be able to walk back actions that we come to view as net negative.

[5]: I’m not sure of how to prioritize these things amongst themselves. Probably improving our understanding of what matters & our predictive power are highest priority, but that’s a very weakly held view.

[John_Maxwell]

Metaculus is an EA project worth a mention in the “improving foresight” area. I’m also excited by what the Less Wrong 2 team is doing. And Clearer Thinking is cool.

I think steering capacity is valuable, but there has to be a balance between building steering capacity and taking object-level action. In many cases, object-level actions are likely to be time-sensitive. Delaying object-level action only makes sense insofar as we can usefully resolve our cluelessness. (But as you say, we tend to become less cluelessness about things as they move from the far future to the near future. So object-level actions which destroy option value can be bad.)

Remember also that acting in the world is sometimes the best way to gather information (which can help resolve cluelessness).

[SiebeRozendal]

I like this post Milan, I think it’s the best of your series. I think that you rightly picked a very important topic to write about (cluelessness) that should receive more attention than it currently does. I do have some comments:

Although I admire new ways to think about prioritisation, I have two worries: Conceptual distinction. Wisdom and predictive power seem not conceptually distinct. Both are about our ability to identifying and predicting the probability of good and bad outcomes. Intent also seems a little tangled up in wisdom, although I can see that we want to seperate those. Furthermore, intent influences coordination capability: the more different the intentions are of a population, the more difficult coordination becomes.

This creates the second worry that this model adds only one dimension (Intent) to the 3-dimensional model of Bostrom’s Technology [Capacity] - Insight [Wisdom] - Coordination. Do you think this increases to usefulness of the model enough? The advantage of Bostrom’s model is that it allows for differential progress (wisdom > coordination > capacity), while you don’t specify the interplay of attributes. Are they supposed to be multiplied, or are some combinations better than others, or do we want differential progress?

I was a bit confused that you write about things to prioritise, but don’t refer back to the 5 attributes of the steering capacity. Some relate more strongly to specific attributes, and some attributes are not discussed much (coordination) or at all (capability).

Further our understanding of what matters

This seems to be Intent in your framework. I totally agree that this is valuable. I would call this moral (or more precisely: axiological) uncertainty, and people work on this outside of EA as well. By the way, besides resolving uncertainty, another pathway is to improve our methods to deal with moral uncertainty. (Like MacAskill argues for)

Improve governance

I am not sure to which this concept this relates to, though I suppose it is Coordination. I find the discussion a bit shallow here as it discusses only institutions, and not the coordination of individuals in e.g. the EA community, or the coordination between nation states.

Improve prediction-making & foresight

This seems to be the attribute predictive power. I agree with you that this is very important. To a large extent, this is also what science in general is aiming to do: improving our understanding so that we can better predict and alter the future. However, straight up forecasting seems more neglected. I think this could also just be called “reducing empirical uncertainty”? If we call it that, we can also consider other approaches, such as researching effects in complex systems.

Reduce existential risk

I’m not sure this was intended to relate to a specific attribute. Guess not.

Increase the number of well-intentioned, highly capable people

This seems to relate mostly to “Intent”as well. I wanted to remark that this can either be done by increasing capability and knowledge of well-intentioned people, or by improving intentions of capable (and knowledgeable) people. My observation is that so far, the focus has been on the latter in term of growth and outreach, and only some effort has been expended to develop the skills of effective altruists. (Although this is noted as a comparative advantage for EA Groups)

Lastly, I wanted to remark that hits-based giving does not imply a portfolio approach in my opinion. It just implies being more or less risk-neutral in altruistic efforts. What drives the diversification in OPP’s grants seems to be worldview diversification, option value, and the possibility that high-value opportunities are spread over cause areas, rather than concentrated in one cause area. I think what would support the conclusion that we need to diversify could be that we need to hit a certain value on each of the attributes otherwise the project fails (a bit like that power-laws arise from success needing ABC instead of A+B+C).

All in all, an important project, but I’m not sure how much novel insight it has brought (yet). This is quite similar to my own experience in that I wrote a philosophy essay about cluelessness and arrived at not-so-novel conclusion. Let me know if you’d like to read the essay :)

[Milan_Griffes]

Wisdom and predictive power seem not conceptually distinct.

I’m using “predictive power” as something like “ability to see what’s coming down the pipe” and “wisdom” as something like “ability to assess whether what’s coming down the pipe is good or bad, according to one’s value system.”

On your broader point, I agree that these attributes are all tangled up in each other. I don’t think there’s a useful way to draw clean distinctions here.

I was a bit confused that you write about things to prioritise, but don’t refer back to the 5 attributes of the steering capacity.

This is a good point, I’ll think about this more & get back to you.

quite similar to my own experience in that I wrote a philosophy essay about cluelessness

I’d like to read this. Could you link to it here, or (if private) send it to the email address on this page? https://​​flightfromperfection.com/​​pages/​​about.html

[SiebeRozendal]

Sure! Here it is.

[Eevee🔹]

For what it’s worth, Bostrom (2013) does distinguish between insight and good values:

We thus want to reach a state in which we have (a) far greater intelligence, knowledge, and sounder judgment than we currently do; (b) far greater ability to solve global-coordination problems; (c) far greater technological capabilities and physical resources; and such that (d) our values and preferences are not corrupted in the process of getting there (but rather, if possible, improved). Factors b and c expand the option set available to humanity. Factor a increases humanity’s ability to predict the outcomes of the available options and understand what each outcome would entail in terms of the realization of human values. Factor d, finally, makes humanity more likely to want to realize human values.

[Eevee🔹]

This post helped clarify to me which causes ought to be prioritized from a longtermist standpoint. Although we don’t know the long-term consequences of our actions (and hence are clueless), we can take steps to reduce our uncertainties and reliably do good over the long term. These include:

• Global priorities research—to improve our understanding of what’s morally important and of the risks and opportunities facing humanity.

• Improving institutional decision-making at the global level—to improve humanity’s “ability to mediate diverse preferences, decide on collectively held goals, and work together towards those goals.”

• Improving foresight—“The further we can see [into the future], the more information we can incorporate into our decision-making, which in turn leads to higher quality outcomes with fewer surprises.”

• Reducing existential risks—“Avoiding extinction and “lock-in” of suboptimal states is necessary for realizing the full potential benefit of the future.”

• “Increasing the number of well-intentioned, highly capable people,” including through building effective altruism and training people to think more clearly.

Although we don’t necessarily know where humanity will end up in the very long term, these interventions help us increase our steering capacity—humanity’s ability to navigate risks and opportunities along the way.

I recommend this post to anyone interested in longtermism, as it’s one of the few systematic attempts at longtermist cause prioritization that I’ve seen. There are things I’d add: Perhaps economic growth would augment humanity’s steering capacity by increasing the amount of resources available to us to avoid risks and pursue opportunities (see also “Existential Risk and Growth”). And perhaps promoting effective altruism to a culturally and intellectually diverse audience would help us make more robust decisions through exposure to more ideas on what matters and how to do good.

I’m heartened to have seen progress in the areas identified in this post. For example, the Effective Institutions Project was created in 2020 to work systematically on IIDM. Also, I’ve seen posts calling attention to the inadequacy of existing cause prioritization research.

Going forward, I’d like to see more systematic attempts at cause prioritization from a longtermist perspective, perhaps building on this post. 80,000 Hours’ list of problem profiles currently includes 17 problems that they claim might be as pressing as their current priority problems (artificial intelligence, biosecurity, etc.). I’d like to see more research clarifying and evaluating these problems, and drawing quantitative comparisons between them.

[Eli Rose]

Therefore, we ought to prioritize interventions that improve the wisdom, capability, and coordination of future actors.

If we operate under the “ethical precautionary principle” you laid out in the previous post (always behave as if there was another crucial consideration yet to discover), how do we do this? We might think that some intervention will increase the wisdom of future actors, based on our best analysis of the situation. But we fear a lurking crucial consideration that will someday pounce and reveal that actually the intervention did nothing, or did the opposite.

In other words, don’t we need to be *somewhat* clueful already in order to bootstrap our way into more cluefulness?

[WillPearson]

I would add something likes “Sensitivity” to the list of attributes needed to navigate the world.

This is different from Predictive Power. You can imagine two ships, with the exact same compute power and Predictive Power. One with cameras on the outside and long range sensors, one blind without. You’d expect the first to do a lot better moving about the world

In Effective Altruism’s case I suspect this would be things like the basic empirical research about the state of the world and the things important to their goals.