Make it clearer that EA is not just classical utilitarianism and that EA abides by moral rules like not breaking the law, not lying and not carrying out violence.
(Especially with short AI timelines and the close to infinite EV of x-risk mitigation, classical utilitarianism will sometimes endorse breaking some moral rules, but EA shouldn’t!)
You keep saying that classical utilitarianism combined with short timelines condones crime, but I don’t think this is the case at all.
The standard utilitarian argument for adhering to various commonsense moral norms, such as norms against lying, stealing and killing, is that violating these norms would have disastrous consequences (much worse than you naively think), damaging your reputation and, in turn, your future ability to do good in the world. A moral perspective, such as the total view, for which the value at stake is much higher than previously believed, doesn’t increase the utilitarian incentives for breaking such moral norms. Although the goodness you can realize by violating these norms is now much greater, the reputational costs are correspondingly large. As Hubinger reminds us in a recent post, “credibly pre-committing to follow certain principles—such as never engaging in fraud—is extremely advantageous, as it makes clear to other agents that you are a trustworthy actor who can be relied upon.” Thinking that you have a license to disregard these principles because the long-term future has astronomical value fails to appreciate that endangering your perceived trustworthiness will seriously compromise your ability to protect that valuable future.
But the short timelines mean the chances of getting caught are a lot smaller because the world might end in 20-30 years. As we get closer to AGI, the chances of getting caught will be smaller still.
FWIW obviously people with the same utilitarian views will disagree on what is positive EV / a good action under the same set of premises / beliefs.
But even then, I think the chances you condone fraud to fund AI safety is very high under the combo of pure / extreme classical total utilitarianism + longtermism + short AI timelines, even if some people who share these beliefs might not condone fraud.
classical utilitarianism will sometimes endorse breaking some moral rules
Classical utilitarianism endorses maximising total wellbeing. Whether this involves breaking “moral rules” or not is, in my view, an empirical question. For the vast majority of cases, not breaking “moral rules” leads to more total wellbeing than breaking them, but we should be open to the contrary. There are many examples of people breaking “moral rules” in some sense (e.g. lying) which are widely recognised as good deeds (widespread fraud is obviously not one such example).
I think classical total utilitarianism + short-ish AI timelines + longtermism unavoidably endorses widespread fraud to fund AI safety research.
Agree that there are examples of when common-sense morality endorses breaking moral rules and even breaking the law, eg—illegal protests against authoritarian leaders, nationalist violence against European colonialists, violence against Nazi soldiers
I think classical total utilitarianism + short-ish AI timelines + longtermism unavoidably endorses widespread fraud to fund AI safety research.
Yes, I can see this being true for some cases, especially if the people whose money is lost are pretty wealthy (e.g. billionaires), and the likelihood of the fraud being detected is super low (e.g. 10^-6). For these cases, I think we should be open to fraud being good. However, this does not mean at all endorsing fraud, because that would have pretty bad effects, and would be incompatible with the probability of fraud being detected being super low.
Make it clearer that EA is not just classical utilitarianism and that EA abides by moral rules like not breaking the law, not lying and not carrying out violence.
(Especially with short AI timelines and the close to infinite EV of x-risk mitigation, classical utilitarianism will sometimes endorse breaking some moral rules, but EA shouldn’t!)
You keep saying that classical utilitarianism combined with short timelines condones crime, but I don’t think this is the case at all.
The standard utilitarian argument for adhering to various commonsense moral norms, such as norms against lying, stealing and killing, is that violating these norms would have disastrous consequences (much worse than you naively think), damaging your reputation and, in turn, your future ability to do good in the world. A moral perspective, such as the total view, for which the value at stake is much higher than previously believed, doesn’t increase the utilitarian incentives for breaking such moral norms. Although the goodness you can realize by violating these norms is now much greater, the reputational costs are correspondingly large. As Hubinger reminds us in a recent post, “credibly pre-committing to follow certain principles—such as never engaging in fraud—is extremely advantageous, as it makes clear to other agents that you are a trustworthy actor who can be relied upon.” Thinking that you have a license to disregard these principles because the long-term future has astronomical value fails to appreciate that endangering your perceived trustworthiness will seriously compromise your ability to protect that valuable future.
But the short timelines mean the chances of getting caught are a lot smaller because the world might end in 20-30 years. As we get closer to AGI, the chances of getting caught will be smaller still.
FWIW obviously people with the same utilitarian views will disagree on what is positive EV / a good action under the same set of premises / beliefs.
But even then, I think the chances you condone fraud to fund AI safety is very high under the combo of pure / extreme classical total utilitarianism + longtermism + short AI timelines, even if some people who share these beliefs might not condone fraud.
I agree.
Classical utilitarianism endorses maximising total wellbeing. Whether this involves breaking “moral rules” or not is, in my view, an empirical question. For the vast majority of cases, not breaking “moral rules” leads to more total wellbeing than breaking them, but we should be open to the contrary. There are many examples of people breaking “moral rules” in some sense (e.g. lying) which are widely recognised as good deeds (widespread fraud is obviously not one such example).
I think classical total utilitarianism + short-ish AI timelines + longtermism unavoidably endorses widespread fraud to fund AI safety research.
Agree that there are examples of when common-sense morality endorses breaking moral rules and even breaking the law, eg—illegal protests against authoritarian leaders, nationalist violence against European colonialists, violence against Nazi soldiers
Yes, I can see this being true for some cases, especially if the people whose money is lost are pretty wealthy (e.g. billionaires), and the likelihood of the fraud being detected is super low (e.g. 10^-6). For these cases, I think we should be open to fraud being good. However, this does not mean at all endorsing fraud, because that would have pretty bad effects, and would be incompatible with the probability of fraud being detected being super low.