I like this post Milan, I think it’s the best of your series. I think that you rightly picked a very important topic to write about (cluelessness) that should receive more attention than it currently does. I do have some comments:
Although I admire new ways to think about prioritisation, I have two worries:
Conceptual distinction. Wisdom and predictive power seem not conceptually distinct. Both are about our ability to identifying and predicting the probability of good and bad outcomes. Intent also seems a little tangled up in wisdom, although I can see that we want to seperate those. Furthermore, intent influences coordination capability: the more different the intentions are of a population, the more difficult coordination becomes.
This creates the second worry that this model adds only one dimension (Intent) to the 3-dimensional model of Bostrom’s Technology [Capacity] - Insight [Wisdom] - Coordination. Do you think this increases to usefulness of the model enough? The advantage of Bostrom’s model is that it allows for differential progress (wisdom > coordination > capacity), while you don’t specify the interplay of attributes. Are they supposed to be multiplied, or are some combinations better than others, or do we want differential progress?
I was a bit confused that you write about things to prioritise, but don’t refer back to the 5 attributes of the steering capacity. Some relate more strongly to specific attributes, and some attributes are not discussed much (coordination) or at all (capability).
Further our understanding of what matters
This seems to be Intent in your framework. I totally agree that this is valuable. I would call this moral (or more precisely: axiological) uncertainty, and people work on this outside of EA as well. By the way, besides resolving uncertainty, another pathway is to improve our methods to deal with moral uncertainty. (Like MacAskill argues for)
Improve governance
I am not sure to which this concept this relates to, though I suppose it is Coordination. I find the discussion a bit shallow here as it discusses only institutions, and not the coordination of individuals in e.g. the EA community, or the coordination between nation states.
Improve prediction-making & foresight
This seems to be the attribute predictive power. I agree with you that this is very important. To a large extent, this is also what science in general is aiming to do: improving our understanding so that we can better predict and alter the future. However, straight up forecasting seems more neglected. I think this could also just be called “reducing empirical uncertainty”? If we call it that, we can also consider other approaches, such as researching effects in complex systems.
Reduce existential risk
I’m not sure this was intended to relate to a specific attribute. Guess not.
Increase the number of well-intentioned, highly capable people
This seems to relate mostly to “Intent”as well. I wanted to remark that this can either be done by increasing capability and knowledge of well-intentioned people, or by improving intentions of capable (and knowledgeable) people. My observation is that so far, the focus has been on the latter in term of growth and outreach, and only some effort has been expended to develop the skills of effective altruists. (Although this is noted as a comparative advantage for EA Groups)
Lastly, I wanted to remark that hits-based giving does not imply a portfolio approach in my opinion. It just implies being more or less risk-neutral in altruistic efforts. What drives the diversification in OPP’s grants seems to be worldview diversification, option value, and the possibility that high-value opportunities are spread over cause areas, rather than concentrated in one cause area. I think what would support the conclusion that we need to diversify could be that we need to hit a certain value on each of the attributes otherwise the project fails (a bit like that power-laws arise from success needing ABC instead of A+B+C).
All in all, an important project, but I’m not sure how much novel insight it has brought (yet). This is quite similar to my own experience in that I wrote a philosophy essay about cluelessness and arrived at not-so-novel conclusion. Let me know if you’d like to read the essay :)
Wisdom and predictive power seem not conceptually distinct.
I’m using “predictive power” as something like “ability to see what’s coming down the pipe” and “wisdom” as something like “ability to assess whether what’s coming down the pipe is good or bad, according to one’s value system.”
On your broader point, I agree that these attributes are all tangled up in each other. I don’t think there’s a useful way to draw clean distinctions here.
I was a bit confused that you write about things to prioritise, but don’t refer back to the 5 attributes of the steering capacity.
This is a good point, I’ll think about this more & get back to you.
quite similar to my own experience in that I wrote a philosophy essay about cluelessness
For what it’s worth, Bostrom (2013) does distinguish between insight and good values:
We thus want to reach a state in which we have (a) far greater intelligence, knowledge, and sounder judgment than we currently do; (b) far greater ability to solve global-coordination problems; (c) far greater technological capabilities and physical resources; and such that (d) our values and preferences are not corrupted in the process of getting there (but rather, if possible, improved). Factors b and c expand the option set available to humanity. Factor a increases humanity’s ability to predict the outcomes of the available options and understand what each outcome would entail in terms of the realization of human values. Factor d, finally, makes humanity more likely to want to realize human values.
I like this post Milan, I think it’s the best of your series. I think that you rightly picked a very important topic to write about (cluelessness) that should receive more attention than it currently does. I do have some comments:
Although I admire new ways to think about prioritisation, I have two worries: Conceptual distinction. Wisdom and predictive power seem not conceptually distinct. Both are about our ability to identifying and predicting the probability of good and bad outcomes. Intent also seems a little tangled up in wisdom, although I can see that we want to seperate those. Furthermore, intent influences coordination capability: the more different the intentions are of a population, the more difficult coordination becomes.
This creates the second worry that this model adds only one dimension (Intent) to the 3-dimensional model of Bostrom’s Technology [Capacity] - Insight [Wisdom] - Coordination. Do you think this increases to usefulness of the model enough? The advantage of Bostrom’s model is that it allows for differential progress (wisdom > coordination > capacity), while you don’t specify the interplay of attributes. Are they supposed to be multiplied, or are some combinations better than others, or do we want differential progress?
I was a bit confused that you write about things to prioritise, but don’t refer back to the 5 attributes of the steering capacity. Some relate more strongly to specific attributes, and some attributes are not discussed much (coordination) or at all (capability).
This seems to be Intent in your framework. I totally agree that this is valuable. I would call this moral (or more precisely: axiological) uncertainty, and people work on this outside of EA as well. By the way, besides resolving uncertainty, another pathway is to improve our methods to deal with moral uncertainty. (Like MacAskill argues for)
I am not sure to which this concept this relates to, though I suppose it is Coordination. I find the discussion a bit shallow here as it discusses only institutions, and not the coordination of individuals in e.g. the EA community, or the coordination between nation states.
This seems to be the attribute predictive power. I agree with you that this is very important. To a large extent, this is also what science in general is aiming to do: improving our understanding so that we can better predict and alter the future. However, straight up forecasting seems more neglected. I think this could also just be called “reducing empirical uncertainty”? If we call it that, we can also consider other approaches, such as researching effects in complex systems.
I’m not sure this was intended to relate to a specific attribute. Guess not.
This seems to relate mostly to “Intent”as well. I wanted to remark that this can either be done by increasing capability and knowledge of well-intentioned people, or by improving intentions of capable (and knowledgeable) people. My observation is that so far, the focus has been on the latter in term of growth and outreach, and only some effort has been expended to develop the skills of effective altruists. (Although this is noted as a comparative advantage for EA Groups)
Lastly, I wanted to remark that hits-based giving does not imply a portfolio approach in my opinion. It just implies being more or less risk-neutral in altruistic efforts. What drives the diversification in OPP’s grants seems to be worldview diversification, option value, and the possibility that high-value opportunities are spread over cause areas, rather than concentrated in one cause area. I think what would support the conclusion that we need to diversify could be that we need to hit a certain value on each of the attributes otherwise the project fails (a bit like that power-laws arise from success needing ABC instead of A+B+C).
All in all, an important project, but I’m not sure how much novel insight it has brought (yet). This is quite similar to my own experience in that I wrote a philosophy essay about cluelessness and arrived at not-so-novel conclusion. Let me know if you’d like to read the essay :)
I’m using “predictive power” as something like “ability to see what’s coming down the pipe” and “wisdom” as something like “ability to assess whether what’s coming down the pipe is good or bad, according to one’s value system.”
On your broader point, I agree that these attributes are all tangled up in each other. I don’t think there’s a useful way to draw clean distinctions here.
This is a good point, I’ll think about this more & get back to you.
I’d like to read this. Could you link to it here, or (if private) send it to the email address on this page? https://flightfromperfection.com/pages/about.html
Sure! Here it is.
For what it’s worth, Bostrom (2013) does distinguish between insight and good values: