Sorry Iâm probably missing something, but Iâm not understanding why real world examples from EA would be particularly relevant given how young a movement it is. I think someone could grant that we have the ability to be justified in assigning probabilities to things that are likely to happen soon, and agree that the risk of things weâre totally unaware of happening in the next ~ 10-50 years might be (at least in some circumstances) sufficiently small to not have unawareness problems.
But once you start trying to be an impartial altruist about far future beings, that seems to me where you really canât get away from unawareness problems. And so I guess if you wanted to convince me I was wrong about that, we should be looking at things that people thought 1000 years ago, and how things they caused today were bad even though they were trying to do good for reasons they werenât only poorly calibrated on but in fact totally unaware ofâand it just seems likely to me there would be tons of examples of that?
Maybe the development of gunpowder stands out here as something being pursued in the hopes of achieving eternal life (ostensibly an altruistic motivation) and presumably the possibility of guns was not on peopleâs radar. I guess it would eventually have been figured out anyway, but how much harm did having gunpowder X years earlier cause?
Maybe an objection here is that an âidealâ agent would have of course considered the possibility of any chemical work being misused, but IDKâthey werenât even trying to make something explosive. I donât see how even a perfectly rational being could have predicted all the harms gunpowder would cause given that they were aiming to do alchemy. What probability could they have possibly been justified, given their epistemic position, in assigning to âsuper bad outcomes from pursuing eternal life chemistryâ given that they probably could not have imagined the scale of modern warfare?
I do get a little mixed up on this between âpeople are not ideal and so regularly make large mistakes that look like cluelessnessâ vs âeven an ideal agent could not be justified in their probability assignments given what is theoretically knowableâ so maybe Iâm misunderstanding something.
Anthony cites Greaves and MacAskill giving an example similar to your gunpowder one:
Consider, for example, would-be longtermists in the Middle Ages. It is plausible that the considerations most relevant to their decision â such as the benefits of science, and therefore the enormous value of efforts to help make the scientific and industrial revolutions happen sooner â would not have been on their radar. Rather, they might instead have backed attempts to spread Christianity, perhaps by violence: a putative route to value that, by our more enlightened lights today, looks wildly off the mark. The suggestion, then, is that our current predicament is relevantly similar to that of our medieval would-be longtermists.
I personally think these examples are less compelling than they first appear (e.g. the persistence literature generally finds weaker effects than what you might imagine), but I agree that a failure of EAs to find examples of sign flips doesnât mean that future ones wonât exist.
Sorry Iâm probably missing something, but Iâm not understanding why real world examples from EA would be particularly relevant given how young a movement it is. I think someone could grant that we have the ability to be justified in assigning probabilities to things that are likely to happen soon, and agree that the risk of things weâre totally unaware of happening in the next ~ 10-50 years might be (at least in some circumstances) sufficiently small to not have unawareness problems.
But once you start trying to be an impartial altruist about far future beings, that seems to me where you really canât get away from unawareness problems. And so I guess if you wanted to convince me I was wrong about that, we should be looking at things that people thought 1000 years ago, and how things they caused today were bad even though they were trying to do good for reasons they werenât only poorly calibrated on but in fact totally unaware ofâand it just seems likely to me there would be tons of examples of that?
Maybe the development of gunpowder stands out here as something being pursued in the hopes of achieving eternal life (ostensibly an altruistic motivation) and presumably the possibility of guns was not on peopleâs radar. I guess it would eventually have been figured out anyway, but how much harm did having gunpowder X years earlier cause?
Maybe an objection here is that an âidealâ agent would have of course considered the possibility of any chemical work being misused, but IDKâthey werenât even trying to make something explosive. I donât see how even a perfectly rational being could have predicted all the harms gunpowder would cause given that they were aiming to do alchemy. What probability could they have possibly been justified, given their epistemic position, in assigning to âsuper bad outcomes from pursuing eternal life chemistryâ given that they probably could not have imagined the scale of modern warfare?
I do get a little mixed up on this between âpeople are not ideal and so regularly make large mistakes that look like cluelessnessâ vs âeven an ideal agent could not be justified in their probability assignments given what is theoretically knowableâ so maybe Iâm misunderstanding something.
Anthony cites Greaves and MacAskill giving an example similar to your gunpowder one:
I personally think these examples are less compelling than they first appear (e.g. the persistence literature generally finds weaker effects than what you might imagine), but I agree that a failure of EAs to find examples of sign flips doesnât mean that future ones wonât exist.