It’s great to know where your specific weights differ! I agree that each of the arguments you put forth are important. Some specifics:
I agree that differences in the future (especially the weird possibilities like digital minds and acausal trade) is a big reason to discount historical evidence. Also, by these lights, some historical evidence (e.g., relations across huge gulfs of understanding and ability like from humans to insects) seems a lot more important than others (e.g., the fact that animal muscle and fat happens to be an evolutionarily advantageous food source).
I’m not sure if I’d agree that historical harms have occurred largely through divergence; there are many historical counterfactuals that could have prevented many harms: the nonexistence of humans, an expansion of the moral circle, better cooperation, discovery of a moral reality, etc.. In many cases, a positive leap in any of these would have prevented the atrocity. What makes divergence more important? I would make the case based on something like “maximum value impact from one standard deviation change” or “number of cases where harm seemed likely but this factor prevented it.” You could write an EA Forum post going into more detail on that. I would be especially excited for you to go through specific historical events and do some reading to estimate the role of (small changes in) each of these forces.
As I mention in the post, reasons to put negative weight on DMPS include the vulnerability of digital minds to intrusion, copying, etc., the likelihood of their instrumental usefulness in various interstellar projects, and the possibility of many nested minds who may be ignored or neglected.
I agree moral trade is an important mechanism of reasoned cooperation.
I’m really glad you put your own numbers in the spreadsheet! That’s super useful. The ease of flipping the estimates from negative to positive and positive to negative is one reason I only make the conclusion “not highly positive” or “close to zero” and not going with the mean estimate from myself and others (which would probably be best described as moderately negative, e.g., the average at an EA meetup where I presented this work was around −10).
I think your analysis is on the right track to getting us better answers to these crucial questions :)
Thanks! Responding on the points where we may have different intuitions:
Regarding your second bullet point, I agree there are a bunch of things that we can imagine having gone differently historically, where each would have been enough to make things go better. These other factors are all already accounted for, so putting the weight on historical harms/progress again still seems to be double-counting (even if which thing it’s double-counting isn’t well-defined).
Regarding your third bullet point, thanks for flagging those points—I don’t think I buy that any of them are reasons for negative weight.
Intrusions could be harmful, but there could also be positive analogues.
Duplication, instrumental usefulness, and nested minds are just reasons to think there might be more of these minds, so these considerations only seem net negative if we already have other reasons to assume these minds’ well-being would be net negative (we may have such reasons, but I think these are already covered by other factors, so counting them here seems like double-counting)
(As long as we’re speculating about nested minds: should we expect them to be especially vulnerable because others wouldn’t recognize them as minds? I’m skeptical; it seems odd to assume we’ll be at that level of scientific progress without having learned how experiences work.)
On interpretation of the spreadsheet:
I think (as you might agree) that results should be taken as suggestive but far from definitive. Adding things up fails to capture many important dynamics of how these things work (e.g., cooperation might not just create good things but also separately counteract bad things).
Still, insofar as we’re looking at these results, I think we should mostly look at the logarithmic sum (because some dynamics of the future could easily be far more important than others).
As I suggested, I have a few smaller quibbles, so these aren’t quite my numbers (although these quibbles don’t really matter if we’re looking at the logarithmic sum).
Thanks for going into the methodological details here.
I think we view “double-counting” differently, or I may not be sufficiently clear in how I handle it. If we take a particular war as a piece of evidence, which we think fits into both “Historical Harms” and “Disvalue Through Intent,” and it is overall −8 evidence on the EV of the far future, but it seems 75% explained through “Historical Harms” and 25% explained through “Disvalue Through Intent,” then I would put −6 weight on the former and −2 weight on the latter. I agree this isn’t very precise, and I’d love future work to go into more analytical detail (though as I say in the post, I expect more knowledge per effort from empirical research).
I also think we view “reasons for negative weight” differently. To me, the existence of analogues to intrusion does not make intrusion a non-reason. It just means we should also weigh those analogues. Perhaps they are equally likely and equal in absolute value if they obtain, in which case they would cancel, but usually there is some asymmetry. Similarly, duplication and nesting are factors that are more negative than positive to me, such as because we may discount and neglect the interests of these minds because they are more different and more separated from the mainstream (e.g., the nested minds are probably not out in society campaigning for their own interests because they would need to do so through the nest mind—I think you allude to this, but I wouldn’t dismiss it merely because we’ll learn how experiences work, such as because we have very good neuroscientific and behavioral evidence of animal consciousness in 2022 but still exploit animals).
Your points on interaction effects and nonlinear variation are well-taken and good things to account for in future analyses. In a back-of-the-envelope estimate, I think we should just assign values numerically and remember to feel free to widely vary those numbers, but of course there are hard-to-account-for biases in such assignment, and I think the work of GJP, QURI, etc. can lead to better estimation methods.
I think we’re on a similar page regarding double-counting—the approach you describe seems like roughly what I was going for. (My last comment was admittedly phrased in an overly all-or-nothing way, but I think the numbers I attached suggest that I wasn’t totally eliminating the weight on history.)
On whether we see “reasons for negative weight” differently, I think that might be semantic—I had in mind the net weight, as you suggest (I was claiming this net weight was 0). The suggestion that digital minds might be affected just by their being different is a good point that I hadn’t been thinking about. (I could imagine some people speculating that this won’t be much of a problem because influential minds will also eventually tend to be digital.) I tentatively think that does justify a mildly negative weight on digital minds, with the other factors you mention seeming to be fully accounted for in other weights.
In my head, I’ve tended to simplify the picture into essentially the “Value Through Intent” argument vs the “Historical Harms” argument, since these seem liked the strongest arguments in either direction to me. In that framing, I lean towards the future being weakly positive.
But this post is a helpful reminder that there are various other arguments pointing in either direction (which, in my case, overall push me towards a less optimistic view). My overall view still seems pretty close to zero at the moment though.
Also interesting how wildly different each of our scores are. Partly I think this might be because I was quite confused/worried about double-counting. Also maybe just not fully grasping some of the points listed in the post.
It’s great to know where your specific weights differ! I agree that each of the arguments you put forth are important. Some specifics:
I agree that differences in the future (especially the weird possibilities like digital minds and acausal trade) is a big reason to discount historical evidence. Also, by these lights, some historical evidence (e.g., relations across huge gulfs of understanding and ability like from humans to insects) seems a lot more important than others (e.g., the fact that animal muscle and fat happens to be an evolutionarily advantageous food source).
I’m not sure if I’d agree that historical harms have occurred largely through divergence; there are many historical counterfactuals that could have prevented many harms: the nonexistence of humans, an expansion of the moral circle, better cooperation, discovery of a moral reality, etc.. In many cases, a positive leap in any of these would have prevented the atrocity. What makes divergence more important? I would make the case based on something like “maximum value impact from one standard deviation change” or “number of cases where harm seemed likely but this factor prevented it.” You could write an EA Forum post going into more detail on that. I would be especially excited for you to go through specific historical events and do some reading to estimate the role of (small changes in) each of these forces.
As I mention in the post, reasons to put negative weight on DMPS include the vulnerability of digital minds to intrusion, copying, etc., the likelihood of their instrumental usefulness in various interstellar projects, and the possibility of many nested minds who may be ignored or neglected.
I agree moral trade is an important mechanism of reasoned cooperation.
I’m really glad you put your own numbers in the spreadsheet! That’s super useful. The ease of flipping the estimates from negative to positive and positive to negative is one reason I only make the conclusion “not highly positive” or “close to zero” and not going with the mean estimate from myself and others (which would probably be best described as moderately negative, e.g., the average at an EA meetup where I presented this work was around −10).
I think your analysis is on the right track to getting us better answers to these crucial questions :)
Thanks! Responding on the points where we may have different intuitions:
Regarding your second bullet point, I agree there are a bunch of things that we can imagine having gone differently historically, where each would have been enough to make things go better. These other factors are all already accounted for, so putting the weight on historical harms/progress again still seems to be double-counting (even if which thing it’s double-counting isn’t well-defined).
Regarding your third bullet point, thanks for flagging those points—I don’t think I buy that any of them are reasons for negative weight.
Intrusions could be harmful, but there could also be positive analogues.
Duplication, instrumental usefulness, and nested minds are just reasons to think there might be more of these minds, so these considerations only seem net negative if we already have other reasons to assume these minds’ well-being would be net negative (we may have such reasons, but I think these are already covered by other factors, so counting them here seems like double-counting)
(As long as we’re speculating about nested minds: should we expect them to be especially vulnerable because others wouldn’t recognize them as minds? I’m skeptical; it seems odd to assume we’ll be at that level of scientific progress without having learned how experiences work.)
On interpretation of the spreadsheet:
I think (as you might agree) that results should be taken as suggestive but far from definitive. Adding things up fails to capture many important dynamics of how these things work (e.g., cooperation might not just create good things but also separately counteract bad things).
Still, insofar as we’re looking at these results, I think we should mostly look at the logarithmic sum (because some dynamics of the future could easily be far more important than others).
As I suggested, I have a few smaller quibbles, so these aren’t quite my numbers (although these quibbles don’t really matter if we’re looking at the logarithmic sum).
Thanks for going into the methodological details here.
I think we view “double-counting” differently, or I may not be sufficiently clear in how I handle it. If we take a particular war as a piece of evidence, which we think fits into both “Historical Harms” and “Disvalue Through Intent,” and it is overall −8 evidence on the EV of the far future, but it seems 75% explained through “Historical Harms” and 25% explained through “Disvalue Through Intent,” then I would put −6 weight on the former and −2 weight on the latter. I agree this isn’t very precise, and I’d love future work to go into more analytical detail (though as I say in the post, I expect more knowledge per effort from empirical research).
I also think we view “reasons for negative weight” differently. To me, the existence of analogues to intrusion does not make intrusion a non-reason. It just means we should also weigh those analogues. Perhaps they are equally likely and equal in absolute value if they obtain, in which case they would cancel, but usually there is some asymmetry. Similarly, duplication and nesting are factors that are more negative than positive to me, such as because we may discount and neglect the interests of these minds because they are more different and more separated from the mainstream (e.g., the nested minds are probably not out in society campaigning for their own interests because they would need to do so through the nest mind—I think you allude to this, but I wouldn’t dismiss it merely because we’ll learn how experiences work, such as because we have very good neuroscientific and behavioral evidence of animal consciousness in 2022 but still exploit animals).
Your points on interaction effects and nonlinear variation are well-taken and good things to account for in future analyses. In a back-of-the-envelope estimate, I think we should just assign values numerically and remember to feel free to widely vary those numbers, but of course there are hard-to-account-for biases in such assignment, and I think the work of GJP, QURI, etc. can lead to better estimation methods.
I think we’re on a similar page regarding double-counting—the approach you describe seems like roughly what I was going for. (My last comment was admittedly phrased in an overly all-or-nothing way, but I think the numbers I attached suggest that I wasn’t totally eliminating the weight on history.)
On whether we see “reasons for negative weight” differently, I think that might be semantic—I had in mind the net weight, as you suggest (I was claiming this net weight was 0). The suggestion that digital minds might be affected just by their being different is a good point that I hadn’t been thinking about. (I could imagine some people speculating that this won’t be much of a problem because influential minds will also eventually tend to be digital.) I tentatively think that does justify a mildly negative weight on digital minds, with the other factors you mention seeming to be fully accounted for in other weights.
I also put my intuitive scores into a copy of your spreadsheet.
In my head, I’ve tended to simplify the picture into essentially the “Value Through Intent” argument vs the “Historical Harms” argument, since these seem liked the strongest arguments in either direction to me. In that framing, I lean towards the future being weakly positive.
But this post is a helpful reminder that there are various other arguments pointing in either direction (which, in my case, overall push me towards a less optimistic view). My overall view still seems pretty close to zero at the moment though.
Also interesting how wildly different each of our scores are. Partly I think this might be because I was quite confused/worried about double-counting. Also maybe just not fully grasping some of the points listed in the post.