This competition entry has been selected for publication by the Forum team.

Introduction and the challenge from cluelessness

This essay is written in response to Anthony DiGiovanni’s sequence “The challenge of unawareness for impartial altruist action guidance” (henceforth referred to as “the sequence”), as an entry to the “Cluelessness Critiques Competition”.

If we want to be impartially altruistic, we care about all the morally relevant consequences of our actions, no matter how distant. Thus, we need to account for all of them when making decisions. I’m introducing the term total expected value (TEV) for this, meaning the expected value or something like it that our choices are going to have, considering all their consequences everywhere.^[1] Though we cannot know the consequences of our actions for certain, to be impartially altruistic, it seems we must use some decision procedure — such as maximising expected moral value or using heuristics likely to bring about better results — that is based on trying to maximise TEV.

The challenge from cluelessness^[2] is that there are contexts, such as effects on the very distant future, where we can’t even say some actions are better in expectation than others because we are unaware of so much of what could affect those consequences.^[3] (This is what will be referred to as “cluelessness”.) This negates all the decision procedures we could use. Assigning expected values would be uselessly arbitrary, heuristics that have worked before might not apply at all, and our intuitions are not calibrated to give us hints about something that we haven’t encountered before.

This seems to lead to the conclusion that since impartial altruistic motivations would require making decisions based on TEV, and we can’t do that because of cluelessness, we have no reason to choose one action over another based on impartial altruist motivations.

As can be seen in the above example, my approach here is not to engage with every specific detail or concept used in examining the question that is used in the sequence, but to distil the question down to just the level of detail that is needed to engage with it at a given time. Thus, bringing in just as much detail and context as is needed. To some extent, this also means bringing in a usefully different perspective.

This essay will aim to prove that even though we cannot make rational decisions based on TEV, we nevertheless can make rational decisions based on impartial altruistic motivations.^[4] Because it is convenient to have a label, I will call the argument I present the cluelessness horizon argument, CHA for short.

CHA is so close to the reply that DiGiovanni calls “Rejecting UEV” or “Option 3” that it could be seen as a variant of it. Contrary to his suggestion in the text, I’m not here formulating that idea into a full-fledged decision theory. Instead, I’m explaining why the right version of the reasoning negates the idea that cluelessness stops us from being able to choose based on impartial altruism. I will separately answer the challenges DiGiovanni poses for “Option 3” from the point of view of the CHA in one of the sections below. In any case, I will motivate CHA on its own with a whole essay, as it is the kind of thing that I suspect takes as much an exercise in seeing a perspective to understand as it does logical argumentation.

Radical scepticism leaks in

In the challenge from cluelessness, we effectively end up in a state of radical scepticism about things involving cluelessness. It’s generally easy to arrive at radical scepticism and hard to show a real way out of it. There could always be something that we are mistaken about that undermines our beliefs. If nothing else, we might be deceived by Descartes’ evil demon fabricating everything we think we observe.

One perspective on how to view radical scepticism is by considering the problem of induction. This is usually presented by saying that we’re not justified in using inductive inference because we can’t know the future will be like the past, but the underlying logic is really about knowing whether the universe is regular enough to know about.

As Nicholas Maxwell^[5] points out, the problem of induction extends to space just as much as time; we can’t know that the same laws of nature or any other generalisation we make apply right now outside the areas of space we have observed. I will add that it can be extended into other conceptual directions as well. For example, we aren’t actually safe assuming we know what we have observed, because that requires assuming the immutability of memories or other records of the past.

Another perspective to scepticism is to think about how we must assume there are no unknown factors. Suppose induction works and we have used it correctly to formulate general if—then statements. So, I know that a rock will fall to the ground if I drop it. Well, what I would actually know is that that is what rocks do under the right conditions. If I drop the rock, but then an alien swoops down in a tiny flying saucer and catches it, then it won’t fall after all. Even assuming we have induced the correct laws for the universe, any individual prediction also relies on assuming we know enough about the current state of the universe to say which laws will apply. We cannot strictly speaking ever know this, hence universal scepticism from this direction as well.

Fundamentally, the problem of induction and the problem of unknown factors are instances of the same problem. An exception to the rules of how things work is a kind of unknown factor, and a failure to predict due to an unknown factor is a failure of induction.

The sequence starts out implicitly assuming that we can make decent inductions and we don’t need to take into account merely imaginable unknown unknowns. In the cases where we have some kind of reasons to think we can know by normal (or perhaps more rigorous but still not impossible) standards, we can go with these.

The problem that leads to the challenge of cluelessness is that even if we assume this on a commonsensical level, there are cases, the ones where we are clueless, where we can’t know things even based on this normal standard of knowing. The things we assume are trustworthy fall away.

The cluelessness event horizon

Still, there is no principled way to completely separate cluelessness from the scepticism that could be applied to anything we think we know. We always have to make some assumptions that we don’t question, even though it’s possible that we could be wrong. We can only set our assumptions to some reasonable level.

We can see cluelessness as the limit where we can no longer even make meaningful guesses about the outcomes or their likelihoods. It’s a kind of metaphorical event horizon that we can epistemically approach but never cross, and if we did, we would fall into an epistemic black hole.

This analogy to a physical event horizon is rather awkward and potentially works in two opposite directions, so even though it makes sense in my mind and I could milk it for more, that would probably only make things more confusing. The point needed to be made here is to introduce the idea of the cluelessness horizon: the line beyond which we are clearly clueless. Inside this horizon are consequences we can have some idea about, so we could make decisions if we only focused on those. Taking into account also what is beyond the horizon is what becomes impossible.

A solution to the problem of induction

Since the challenge from cluelessness is related to the problem of induction, I will now present a solution to the problem of induction to build up towards my own answer, first formulated by Nicholas Maxwell.^[6]

We cannot epistemically overcome the problem of induction. The solution lies in a different direction, specifically, what it is rational for us to assume when trying to understand the world. In order for the world to be comprehensible, and for us to know or understand anything about it (except arguably what experiences we are currently having), the world needs to have some kind of structure. It may be that the world is not entirely structured and ordered, but we can understand it only to the extent that it is.

Thus, either the world has some structure that we could understand by induction, or it does not, and in this latter case, we cannot know anything about it anyway. The only case where we can possibly gain knowledge of the world is the one where the world has enough structure and we act on the assumption that it does. Therefore, it is rationally justified to assume that the world has structure, even though we cannot know for sure that it does. If we are wrong about that, then our attempts are doomed regardless. We must bet on the world having enough structure, because it is the only bet we have.

Cluelessness and choice

As the sequence points out, we always have to make a choice, even if we choose inaction. If we are uncertain or even clueless about the consequences of our choice, we can’t avoid making that choice by refraining from action. Something will still follow in the future from the choice of not having acted.

Someone might object that inaction is better than doing something that risks causing great harm. This is probably a good heuristic to follow in many situations. However, if we can justifiably say that inaction is a safer bet in some particular case, then that case is not one about which we are completely clueless; we must be able to know enough to have reason to justify the use of the heuristic. The problem raised by the challenge from cluelessness is that, about some kinds of things, we do not. There are also cases in which seeing some of the shape of the problem speaks against the safety of inaction.^[7]

Cluelessness is definitely a problem for making reasoned choices. In order to make a choice that is meaningful and not random, we need to have information about the consequences.^[8] Otherwise, the choice is made for no reason, and it is meaningless which choice we make^[9], even though we still unavoidably have to make a choice. The basis for choosing one thing rather than another falls off.

A solution to the problem of cluelessness and choice: The cluelessness horizon argument

Because we need to have knowable reasons in order to make meaningful choices, consequences that are beyond the horizon can’t give us reasons to make decisions based on. We can only have reasons based on those possible consequences that we are not clueless about. From this, the claim of the cluelessness horizon argument is that, even while we are impartial in our motivations, we are rationally justified and indeed required to make our decisions based on and based only on reasons on the knowable side of the horizon.

It’s still true that the utterly unforeseeable consequences of our choices are consequences of our choices. However, as far as our reasons to make choices go, the unforeseeable consequences are as disconnected from the rational reasons for our choice-making as something that we can’t causally affect. Hence, they are irrelevant for our decision-making.

Conversely, consequences we are not clueless about are relevant to our decision-making. This leads to the idea that we could make decisions based on reasons only on this side of the cluelessness horizon. But this means we can’t try to maximise TEV — does that mean, in turn, that we still can’t have reasons to choose one action over another based on impartial altruistic motivations?

I argue that it does not,^[10] because the reason that we exclude consequences beyond the horizon is not because we value more immediate consequences more — it is because we can only do anything about knowable consequences. Any altruistic consequences that we are able to take into account, we do take into account. We disregard those consequences we can’t possibly anticipate only because we, practically, have to. There is no arbitrary or value-based privileging of that which is more immediate, only a rationally required privileging of it in the context of choice.

This solution has something in common with the solution to the problem of induction above. We can’t possibly eliminate the epistemic uncertainty, but we can still rationally justify making certain choices because they are the only ones that make sense as choices for us to make given our limitations.

Wrapping our heads around what this means

It’s not normal to think of making choices while considering cluelessness. That’s rather the whole point being made here, but it also means intuitions from more normal contexts might make the conclusion of the cluelessness horizon argument seem wrong. There are also other ways in which it seems the wrong ideas might be triggered by what I’m saying here.

It may sound wrong to ignore possible consequences of our actions that could be much larger than those we are making our decisions based on. However, it is not wrong to do so, but unavoidable. It is bad that we can’t take them into account. It’s worrying that we can’t be sure that something terrible doesn’t happen in the future. But it’s bad and worrying the same way as the possibility of bad things happening in the future that are beyond our causal power to affect. In terms of our making choices based on reasons, consequences of our actions that we are clueless about are no different from future things that may happen that are not even causally affected by our choices.

As I already pointed out above, we are not ignoring the possibility of greater harm in the same sense as if we were taking avoidable risks, and thus intuitions applicable to such a situation, though they might be activated, don’t apply here. The risk from cluelessness is unavoidable, and functionally impossible to do anything about, insofar as the risk really lies beyond the cluelessness horizon.

A second point: we may need to recalibrate our heads at this point if we have been thinking in terms of total expected value, lest we think that which we can affect is not significant. Just because we have here been thinking in terms of much bigger consequences than those we can affect on this side of the horizon doesn’t mean the things we can affect are not hugely valuable in themselves. Just as being able to, say, save thousands of people from disease is a great thing even if there are millions more that are still sick, it would be great to be able to (say) help humanity in general flourish for the next one hundred years even if that pales in comparison with the value and disvalue that will happen within the next gazillion years in the rest of the universe.

Suppose we somehow knew, for sure, that our actions could only affect things happening on our own planet — or our own hometown, or in our hometown during the next century. There is still much value that could be affected (or effected) in such a case. We could potentially affect billions or thousands of lives in such scenarios. This would be important — regardless of there being vastly larger amounts of value outside our scope. It would also be the most important thing we could care about in our choices, despite not being nearly the biggest scope we know exists or might exist. Furthermore, it would be much larger in scale than the standard consequences on the scale of our own life that our actions could have without impartial considerations.

Finally, we need to be careful not to slip from the idea that we can and should justifiably exclude consequences that we are clueless about from our decision-making to the idea that we can be cavalier about those consequences. They still matter, from our impartial perspective, and our choices still potentially affect them. Therefore, we should continue to do whatever we can to account for consequences we can account for, including trying to reduce our cluelessness. Only beyond the horizon where we are truly clueless about something (and can’t stop being so) do the unknown consequences become something we should discount. We would take them into account if we could.

The cluelessness horizon argument and “Option 3”

As stated at the beginning, there’s an answer already discussed in the sequence that resembles the cluelessness horizon argument so much that they can be considered variants of the same idea. Taking this argument from Jesse Clifton, who calls is “Option 3” in the original context, DiGiovanni paraphrases it as follows:

I suspend judgment on whether A results in higher expected total welfare than B. But on one hand, A is better than B with respect to some subset of their overall effects (e.g., donating to AMF saves lives in the near term), which gives me a reason in favor of A. On the other hand, I have no clue how to compare A to B with respect to all the other effects (e.g., donating to AMF might increase or decrease x-risk; Mogensen 2020), which therefore don’t give me any reasons in favor of B. So, overall, I have a reason to choose A but no reason to choose B, so I should choose A.

The similarity to the CHA should be obvious. (A and B are simply some two options between which we have to choose, hopefully on impartial altruistic grounds.)

The major problem DiGiovanni sees with this is the following:

There are many different ways of carving up the set of “effects” according to the reasoning above, which favor different strategies. For example: I might say that I’m confident that an AMF donation saves lives, and I’m clueless about its long-term effects overall. Yet I could just as well say I’m confident that there’s some nontrivially likely possible world containing an astronomical number of happy lives, which the donation makes less likely via potentially increasing x-risk, and I’m clueless about all the other effects overall. So, at least without an argument that some decomposition of the effects is normatively privileged over others, Option 3 won’t give us much action guidance.

In terms of the CHA, for this example, we need to ask whether this “some nontrivially likely possible world” is one we could include while honestly still calculating expected values (or the like) and not having to concede cluelessness.

Let’s look at this in terms of the problems that lead to cluelessness. Can we make EV estimates that are not too arbitrary, or trust our intuitions, or take confidence from the record of superforecasters, and so on, with respect to the outcomes we’re considering? Are the outcomes that we’re considering ones that we can describe with enough precision that we can speak their expected values (or their signs), as opposed to ones that are too coarsely described? If not, then these outcomes are ones that we can’t take into account. The nontrivially likely bad outcome singled out in the example above is clearly one that we can’t. (It’s nontrivially likely, you say? Can you give a meaningful likelihood that can be used in decision-making to the outcome or to how much more likely the intervention makes it?)

We know the proposed additional considerations (outcomes) have gone over the cluelessness horizon and have to be disregarded as a basis for making decisions when we run into the problem that we can’t make decisions based on them. Thus, we do have a test for which outcomes to consider, one that is not arbitrary to the extent that the recognition of cluelessness is not arbitrary.^[11]

There’s something about how “Option 3” is presented and discussed that makes me uncertain about whether it is saying something subtly different from CHA, or not. I will just flag that here. “Option 3” refers to “some subset” of the overall effects of a choice that is tractable. No wonder it sounds as though it’s not clear how that subset should be defined. CHA refers to the subset that is all the effects that are not beyond the cluelessness horizon. I don’t know whether Option 3 should be said to be referring specifically to this same subset — the wording (also in the original source) gives this some support, but it’s not made as clear.

The next objection DiGiovanni discusses is about how the other reasons still matter, even if we can’t properly reason about what choice they suggest. This goes back to one of the distinctions emphasized in the previous section: they matter in terms of our values, but they don’t matter to our decision-making because (again) they can’t guide it. They would give us reasons, but those reasons are inaccessible.

Perhaps also worth emphasizing for the CHA, something that may not be so clear with “Option 3,” is how we are not merely enabled to make decisions based on the reasons on this side of the horizon, but in a sense forced into it, because the only other option is to make choices without even the accessible amount of rationality. This gives CHA a certain urgency: we are not so much given a possibility to be (kind of) rational in our choices as forced to apply the limited degree of rationality that’s available. There is no option to avoid making choices, so complaining about having to do it imperfectly would go nowhere.

DiGiovanni writes, not as a criticism this time, that “insofar as Option 3 is action-guiding, it seems to support a focus on near-term welfare, which would already suggest significant changes to current EA prioritization.” Does this apply to the CHA? Only insofar as we are clueless about how to have the desired long-term impact. Insofar as we are, how were we going to plan for long-term impact anyway? If we needed to be told we can’t do this, well, it’s just as well that we were. The problem, if this is the case, is about how we don’t know what we should do about it, and this could only be fixed by figuring out a better way to know our impact, not by studying different theories of decision-making.

Practical takeaways?

There are some general practical implications that I can tentatively draw from this solution to the challenge of cluelessness. None of them are properly novel, and all of them could have been (and surely have been, previously) arrived at by means of other kinds of reasoning. Then again, different ways of reasoning converging on the same conclusions gives some evidence of the soundness of the reasoning, so the soundness of the sequence and my answer put together gains support from convergence with the simpler reasoning, and vice versa. It would also be possible to arrive at other conclusions, so it may be significant to see these particular conclusions reinforced.

We should make efforts to reduce our areas of cluelessness, to push back the horizon, because beyond it lie consequences that matter but that we can’t even try to affect with our choices.
On the other hand, we shouldn’t try to account for consequences we can’t account for, especially at the expense of ones that we are not clueless about and can reasonably try to affect. Huge expected values shouldn’t sway our priorities if we’re clueless about how to calculate them — and we need to acknowledge we genuinely can be so clueless. More local increases of value can be of supreme value to us in terms of making choices, even if something unknowable would dwarf them, because they can be the maximum amount of value we can add (other than by random luck that is meaningless to our choices).
While we can’t prove even that things like slowing down dangerous developments, building capacities or increasing knowledge have better consequences in expectation with respect to outcomes we are truly clueless about, it may be that they are to be recommended because they are a better way of making the bet that there is something we can do, since they can be a way of betting on the possibility that we can push back the horizon or keep stalling until it gets pushed back. It may be that we have to make choices about a given thing that go beyond the horizon and can’t be made rationally, but it may be that we can skirt around that until we are able to push back the horizon enough that we can make informed choices. While we can’t know which is the case, we might bet on the latter option on the basis that it’s the only one where we can do something, and thus we are justified in betting on it. (I am not quite sure how this point works out. Maybe cluelessness simply negates the possibility of meaningfully betting like this, unlike generally betting on things on this side of the horizon. It would need further thinking about.)
In some cases, we are forced to approach the cluelessness horizon very closely, as it impinges even on those areas of life and choice where we can normally operate with more confidence. The obvious example is dealing with the possibility that superintelligent AI might emerge relatively soon and lead to great impact in ways that we cannot anticipate based on past knowledge. In such cases, we have to think in terms of things we might be clueless about. We need to make less certain bets about not being clueless in given places, and we need to make a great deal of efforts to find ways to expand our horizon of non-cluelessness. Of course, different though this is as an experience and a practical challenge, in principle, this is the same situation as regarding more distant consequences, since we always have to do as much as we can and no more.
Once we really get to the point where something is beyond our ability to make meaningful choices about, we should stop worrying about it — perhaps with a Zen-like attitude of acceptance. At the same time, we still need to keep our minds open to the possibility that we were wrong about this given thing being beyond the horizon or beyond our ability to move to the knowable side of the horizon.
There is a sense in which things we are merely uncertain about are not entirely different from those we are clueless about. Just because we can justifiably assign some degree of probability to some consequence doesn’t mean it’s not possible we are radically mistaken. Thus, the perspectives and lessons described here apply not only all the way across the horizon but increasingly as we approach it.

Conclusion

The challenge from cluelessness is that it seems that there is no reason to make choices based on impartial altruistic motivations, since those should take into account total expected value, and due to cluelessness, it’s impossible to do that. My argument here, the cluelessness horizon argument, is that there can be rational reasons to choose one action over another from impartial altruistic motivations precisely because things we are clueless about are inaccessible for motivated choice; though there is no way to account for them, to the extent that we really are clueless, it is rational to discount them for just that reason. This leads to impartial altruistic motivation being guided by those impartial altruistic considerations that we can be guided by. Nothing else limits the impartiality than our limits of being able to take things into account at all.

The practical takeaways from CHA are not properly novel. They could hardly be so when the argument only takes a different perspective on something that has been explored from many others before, both theoretically and based on practical experience. Of course, I might be wrong about this if the challenge from cluelessness itself has been having effects on people’s choices instead of merely making them feel confused about their justifications.

Though theoretical arguments like the challenge from cluelessness and CHA can strictly prove some conclusions on the level on which they operate, I think that their value in the bigger picture (of altruistic action, or other domains) is not in proving anything once and for all. Instead, the value lies in getting more perspectives on how to understand our situation as we make choices, and thus to build wisdom and better models that enable us to make better choices within the realm of things that we can meaningfully affect — and hopefully, to expand that realm and the reach of our positive impact ever outward.

^
The sequence only speaks of expected value, as, of course, everything is part of the expected value. My current argument benefits from highlighting the difference between total expected value and a more bounded version. Speaking of total expected value here also sometimes has the same extension as, in the sequence, speaking of uncertainty-inclusive expected value (UEV), though the meanings of the concepts are different. The overlap is in that trying to maximise TEV in this context amounts to trying to maximise UEV, because uncertainty needs to be taken into account with TEV.
^
Based on the title of the sequence, this could have been called “challenge of unawareness,” but I find this phrase slightly better for my purposes.
^
The wording used here generalises over two versions of the problem described in the sequence: our being unable to model the resulting worlds with enough precision to tell what the morally relevant outcomes will be like, and our being completely unaware of the possibility of some factors that might come into play and affect things significantly.
^
Thus, in terms of the competition aims, I am “challenging the inference,” i.e. questioning its conclusion that “we have no impartial-altruistic reason to prefer any action over another.” There are three other ways of challenging the argument in which this answer could be framed:
Rejecting premise 1, “that a justified impartial-altruistic preference for A over B requires that A has higher ‘expected value’ (broadly construed, not necessarily literal EV), not merely (e.g.) that A seems heuristically good.” This premise is being rejected if we consider EV (broadly construed) in the premise as necessary meaning TEV rather than any more bounded version of EV.
Defending a specific counterargument (“Option 3”) against objections (see next in main text).
Accepting the conclusion but “offering a constructive response” by using a different interpretation of what count as “impartial-altruistic reasons”.
These distinctions are unimportant for the point itself; they depend on ambiguities of wording in the original argument that don’t change the argument itself.
^
In his From Knowledge to Wisdom: A Revolution for Science and the Humanities.
^
Ibid.
^
The risk of misaligned AI is an obvious example: if there is a threat that future AI systems might harm us simply because of the way things are already going, then just because we cannot be sure our interventions will not do more harm than good does not mean that it is obviously safer not to intervene. The risk already exists, regardless, as far as we can tell. Not working on building better AI in the first place would have been a safer way in the mundane sense, though it would still raise the possibility of someone else doing it in a worse way.
^
Or at least how the choice conforms to some standards of action, like deontological principles. In the current context, we are of course interested in ethical consequences.
^
Free will libertarians (not to be confused with the political view by the same name) think that some amount of randomness in some or all choices is not only acceptable but required — that indeterminism is required to avoid the supposed problems of determinism. Few will admit that they want “randomness”, imagining that the indeterminism can somehow be meaningful choice, but they can’t escape this problem, and even though libertarianism has different forms and motivations, I have not found any self-proclaimed “libertarians” who do not at least implicitly want indeterminism. I have argued this point elsewhere at a great deal of detail (see here and here), but here, it suffices to say that there is nothing in the idea that indeterminism is part of choice that can help with the current problem of lack of rational reasons.
^
Recall the footnote near the beginning saying that I would be challenging the inference in this sense, but also that it doesn’t matter whether this is seen as accepting the conclusion but offering a constructive response to it. If one wanted to insist that impartial altruistic motivation has to be based on maximising total expected value regardless of what is said here — not that I would agree with that interpretation — then we could say that even if this is so, I’m still offering a constructive way forward for impartial altruists.
^
Remember from before that drawing the line for cluelessness is somewhat arbitrary, but only if you don’t block universal radical scepticism; by the logic of the sequence, we are already allowed to take more mundane things as a given. If we now worry that we can’t draw the line in a principled manner, it should already have led us into scepticism about everything before. If we are allowed to say which things we are clueless about and which not, the action-guiding line can be drawn in terms of CHA.

The Cluelessness Horizon Argument: Limited Rationality from Impartial Motivations