Something that provides some amount of protection is the idea of ethical injunctions.
”Ethical injunctions are rules not to do something even when it’s the right thing to do. (That is, you refrain “even when your brain has computed it’s the right thing to do”, but this will just seem like “the right thing to do”.)”
The problem is, this doesn’t help with some of these examples. Ethical injunctions would protect us against the first item. A 1940′s EA might say “Yes, eugenics is a good thing, see the literature, but we still shouldn’t kill or sterilise people without their consent to make it happen.”
Lobotomies are a bit harder, but I think a suitably strong ethical injunction of “Don’t perform surgeries without people’s consent” would protect us there too. What it wouldn’t protect us from would be thinking lobotomies were a good thing and managing to convince patients to let us do them.
Finally, ethical injunctions gives us no protection against recovered memory therapy. If you grant as a prior that RMT works, there is no harm being done—you are simply bringing old memories to light and bringing guilty people to justice.
So there’s basically two separate problems here—how should EA avoid being on the wrong side of history morally, and how should EA avoid being on the wrong side of history scientifically?
I think our best bet for the first is to hold to universal moral principles like “Thou shalt not kill” and avoid allowing clever arguments to convince us to do things for the greater good. I think EA already does this pretty well—there’s a strong norm against, say, lying about our top charities’ effectiveness to solicit donations.
For the second one...I don’t think there’s a way out of that one. Science is humanity’s best guess at the time, so we can’t do better except in areas we might happen to have a comparative advantage in—psychology isn’t one of them. All we can do is be open to changing our minds when the evidence stacks up against $WRONG_THEORY, so we should continue to promote epistemic norms of reasoning transparency and soliciting criticism of the movement.
Something that provides some amount of protection is the idea of ethical injunctions.
”Ethical injunctions are rules not to do something even when it’s the right thing to do. (That is, you refrain “even when your brain has computed it’s the right thing to do”, but this will just seem like “the right thing to do”.)”
The problem is, this doesn’t help with some of these examples. Ethical injunctions would protect us against the first item. A 1940′s EA might say “Yes, eugenics is a good thing, see the literature, but we still shouldn’t kill or sterilise people without their consent to make it happen.”
Lobotomies are a bit harder, but I think a suitably strong ethical injunction of “Don’t perform surgeries without people’s consent” would protect us there too. What it wouldn’t protect us from would be thinking lobotomies were a good thing and managing to convince patients to let us do them.
Finally, ethical injunctions gives us no protection against recovered memory therapy. If you grant as a prior that RMT works, there is no harm being done—you are simply bringing old memories to light and bringing guilty people to justice.
So there’s basically two separate problems here—how should EA avoid being on the wrong side of history morally, and how should EA avoid being on the wrong side of history scientifically?
I think our best bet for the first is to hold to universal moral principles like “Thou shalt not kill” and avoid allowing clever arguments to convince us to do things for the greater good. I think EA already does this pretty well—there’s a strong norm against, say, lying about our top charities’ effectiveness to solicit donations.
For the second one...I don’t think there’s a way out of that one. Science is humanity’s best guess at the time, so we can’t do better except in areas we might happen to have a comparative advantage in—psychology isn’t one of them. All we can do is be open to changing our minds when the evidence stacks up against $WRONG_THEORY, so we should continue to promote epistemic norms of reasoning transparency and soliciting criticism of the movement.