Not trying to take this out on you, but I’m annoyed by how much all this advocacy of deontology all of sudden overlaps with covering our own asses. I don’t buy it as a massive update about morality or psychology from the events themselves but a massive update about optics.
Reposting from twitter: It’s a moderate update on the prevalence of naive utilitarians among EAs.
Expanded:
Classical problem with this debate on utilitarianism is the vocabulary used makes motte-and-bailey defense of utilitarianism too easy. 1. Someone points to a bunch problems with a act consequentialist decision procedure / cases where naive consequentialism tells you to do bad things 2. The default response is “but this is naive consequentialism, no one actually does that” 3. You may wonder that while people don’t advocate for or self-identify as naive utilitarians … they actually make the mistakes
The case provides some evidence that the problems can actually happen in practice in important enough situations to care. [*]
Also, you have the problem that sophisticated naive consequentialists could be tempted to lie to you about their morality (“no worries, you can trust me, I’m following the sensible deontic constraints!”). Personally, before the recent FTX happenings, I would be more of the opinion “nah, this sounds too much like an example from a philosophical paper, unlikely with typical human psychology ”. Now I take it as more real problem.
[*] What I’m actually worried about …
Effective altruism motivated thousands of people to move into highly leveraged domains, with large and potentially deadly consequences—powerful AI stuff, pandemics, epistemic tech. I think that if just 15% of them believe in some form of hardcore utilitarianism where you drop integrity constrains and trust your human brain ability to evaluate when to be constrained and when not, it’s … actually a problem?
I’d agree with this statement more if it acknowledged the extent to which most human minds have the kind of propositional separation between “morality” and “optics” that obtained financially between FTX and Alameda.
I don’t buy it as a massive update about morality or psychology from the events themselves but a massive update about optics.
This will be a relief if true. I am much more worried about people not having principles (or their principles guided by something other than morality) than people being overly concerned about optics. The latter is a tactical concern (albeit a big one) and hopefully fixable, the former is evidence that people in our movement is too conformist or otherwise too weak or too evil to confront moral catastrophes.
Not trying to take this out on you, but I’m annoyed by how much all this advocacy of deontology all of sudden overlaps with covering our own asses. I don’t buy it as a massive update about morality or psychology from the events themselves but a massive update about optics.
Reposting from twitter: It’s a moderate update on the prevalence of naive utilitarians among EAs.
Expanded:
Classical problem with this debate on utilitarianism is the vocabulary used makes motte-and-bailey defense of utilitarianism too easy.
1. Someone points to a bunch problems with a act consequentialist decision procedure / cases where naive consequentialism tells you to do bad things
2. The default response is “but this is naive consequentialism, no one actually does that”
3. You may wonder that while people don’t advocate for or self-identify as naive utilitarians … they actually make the mistakes
The case provides some evidence that the problems can actually happen in practice in important enough situations to care. [*]
Also, you have the problem that sophisticated naive consequentialists could be tempted to lie to you about their morality (“no worries, you can trust me, I’m following the sensible deontic constraints!”). Personally, before the recent FTX happenings, I would be more of the opinion “nah, this sounds too much like an example from a philosophical paper, unlikely with typical human psychology ”. Now I take it as more real problem.
[*] What I’m actually worried about …
Effective altruism motivated thousands of people to move into highly leveraged domains, with large and potentially deadly consequences—powerful AI stuff, pandemics, epistemic tech. I think that if just 15% of them believe in some form of hardcore utilitarianism where you drop integrity constrains and trust your human brain ability to evaluate when to be constrained and when not, it’s … actually a problem?
I’d agree with this statement more if it acknowledged the extent to which most human minds have the kind of propositional separation between “morality” and “optics” that obtained financially between FTX and Alameda.
This will be a relief if true. I am much more worried about people not having principles (or their principles guided by something other than morality) than people being overly concerned about optics. The latter is a tactical concern (albeit a big one) and hopefully fixable, the former is evidence that people in our movement is too conformist or otherwise too weak or too evil to confront moral catastrophes.
I don’t think they know they are concerned about optics. My suspicion was that the bad optics suddenly made utilitarian ideas seem false or reckless.