First two points sound reasonable (and helpfully clarifying) to me!
I suppose it is the nature of being scope-sensitive and prioritiationist though that something being very important and neglected and moderately tractable (like x-risk work) isn’t always enough for it to be the ‘best’
I share the guess that scope sensitivity and prioritarianism could be relevant here, as you clearly (I think) endorse these more strongly and more consistently than I do; but having thought about it for only 5-10 minutes, I’m not sure I’m able to exactly point at how these notions play into our intuitions and views on the topic—maybe it’s something about me ignoring the [(super-high payoff of larger future)*(super-low probability of affecting whether there is a larger future) = (there is good reason to take this action)] calculation/conclusion more readily?
That said, I fully agree that “something being very important and neglected and moderately tractable (like x-risk work) isn’t always enough for it to be the ‘best’ ”. To figure out which option is best, we’d need to somehow compare their respective scores on importance, neglectedness, and tractability… I’m not sure actually figuring that out is possible in practice, but I think it’s fair to challenge the claim that “action X is best because it is very important and neglected and moderately tractable” regardless. In spite of that, I continue to feel relatively confident in claiming that efforts to reduce x-risks are better (more desirable) than efforts to increase the probable size of the future, because the former is an unstable precondition for the latter (and because I strongly doubt the tractability and am at least confused about the desirability of the latter).
Another, perhaps tortured, analogy: you have founded a company, and could spend all your time trying to avoid going bankrupt and mitigating risks, but maybe some employee should spend some fraction of their time thinking about best-case scenarios and how you could massively expand and improve the company 5 years down the line if everything else falls into place nicely.
I think my stance on this example would depend on the present state of the company. If the company is in really dire straits, I’m resource-constrained, and there are more things that need fixing now than I feel able to easily handle, I would seriously question whether one of my employees should go thinking about making best-case future scenarios the best they can be[1]. I would question this even more strongly if I thought that the world and my company (if it survives) will change so drastically in the next 5 years that the employee in question has very little chance of imaging and planning for the eventuality.
(I also notice while writing that a part of my disagreement here is motivated by values rather than logic/empirics: part of my brain just rejects the objective of massively expanding and improving a company/situation that is already perfectly acceptable and satisfying. I don’t know if I endorse this intuition for states of the world (I do endorse it pretty strongly for private life choices), but can imagine that the intuitive preference for satisficing informs/shapes/directs my thinking on the topic at least a bit—something for myself to think about more, since this may or may not be a concerning bias.)
I expected this would not be a take particularly to your liking, but your pushback is stronger than I thought, this is useful to hear. [...] As a process note, I think these discussions are a lot easier and better to have when we are (I think) both confident the other person is well-meaning and thoughtful and altruistic, I think otherwise it would be a lot easier to dismiss prematurely ideas I disagree with or find uncomfortable.
(This is not to say that it might not make sense for one or a few individuals to think about the company’s mid- to long-term success; I imagine that type of resource allocation will be quite sensible in most cases, because it’s not sustainable to preserve the company in a day-to-day survival strategy forever; but I think that’s different from asking these individuals to paint a best-case future to be prepared to make a good outcome even better.)
That makes sense, yes perhaps there are some fanaticism worries re my make-the-future large approach even more so than x-risk work, and maybe I am less resistant to fanaticism-flavoured conclusions than you. That said I think not all work like this need be fanatical—e.g. improving international cooperation and treaties for space exploration could be good in more frames (and bad is some frames you brought up, granted).
I don’t know lots about it, but I wonder if you prefer more of a satisficing decision theory where we want to focus on getting a decent outcome rather than necessarily the best (e.g. Bostrom’s ‘Maxipok’ rule). So I think not wholeheartedly going for maximum expected value isn’t a sign of irrationality, and could reflect different, sound, decision approaches.
First two points sound reasonable (and helpfully clarifying) to me!
I share the guess that scope sensitivity and prioritarianism could be relevant here, as you clearly (I think) endorse these more strongly and more consistently than I do; but having thought about it for only 5-10 minutes, I’m not sure I’m able to exactly point at how these notions play into our intuitions and views on the topic—maybe it’s something about me ignoring the [(super-high payoff of larger future)*(super-low probability of affecting whether there is a larger future) = (there is good reason to take this action)] calculation/conclusion more readily?
That said, I fully agree that “something being very important and neglected and moderately tractable (like x-risk work) isn’t always enough for it to be the ‘best’ ”. To figure out which option is best, we’d need to somehow compare their respective scores on importance, neglectedness, and tractability… I’m not sure actually figuring that out is possible in practice, but I think it’s fair to challenge the claim that “action X is best because it is very important and neglected and moderately tractable” regardless. In spite of that, I continue to feel relatively confident in claiming that efforts to reduce x-risks are better (more desirable) than efforts to increase the probable size of the future, because the former is an unstable precondition for the latter (and because I strongly doubt the tractability and am at least confused about the desirability of the latter).
I think my stance on this example would depend on the present state of the company. If the company is in really dire straits, I’m resource-constrained, and there are more things that need fixing now than I feel able to easily handle, I would seriously question whether one of my employees should go thinking about making best-case future scenarios the best they can be[1]. I would question this even more strongly if I thought that the world and my company (if it survives) will change so drastically in the next 5 years that the employee in question has very little chance of imaging and planning for the eventuality.
(I also notice while writing that a part of my disagreement here is motivated by values rather than logic/empirics: part of my brain just rejects the objective of massively expanding and improving a company/situation that is already perfectly acceptable and satisfying. I don’t know if I endorse this intuition for states of the world (I do endorse it pretty strongly for private life choices), but can imagine that the intuitive preference for satisficing informs/shapes/directs my thinking on the topic at least a bit—something for myself to think about more, since this may or may not be a concerning bias.)
+100 :)
(This is not to say that it might not make sense for one or a few individuals to think about the company’s mid- to long-term success; I imagine that type of resource allocation will be quite sensible in most cases, because it’s not sustainable to preserve the company in a day-to-day survival strategy forever; but I think that’s different from asking these individuals to paint a best-case future to be prepared to make a good outcome even better.)
That makes sense, yes perhaps there are some fanaticism worries re my make-the-future large approach even more so than x-risk work, and maybe I am less resistant to fanaticism-flavoured conclusions than you. That said I think not all work like this need be fanatical—e.g. improving international cooperation and treaties for space exploration could be good in more frames (and bad is some frames you brought up, granted).
I don’t know lots about it, but I wonder if you prefer more of a satisficing decision theory where we want to focus on getting a decent outcome rather than necessarily the best (e.g. Bostrom’s ‘Maxipok’ rule). So I think not wholeheartedly going for maximum expected value isn’t a sign of irrationality, and could reflect different, sound, decision approaches.