I agree with the distinction between naive and prudent utilitarianism, but I also think it all breaks down when you factor in the infinite expected value of mitigating extinction risks (assuming a potential techno utopian future, as some longtermists do) and the risk of AGI driven extinction in our lifetimes.
I’m pretty sure lots of pure prudent utilitarians would still endorse stealing money to fund AI safety research, especially as we get closer in time to the emergence of AGI.
A lot of people are disagreeing with you. But what you are saying is just uncontroversially correct, as anyone who studies philosophy outside of EA would tell you.
To those disagree-downvoting: I would like to suggest the possibility that the problem, here, is with utilitarianism. Not with anything freedomandutility said. Because what they said is literally just true as a matter of math. If you don’t like that, then you don’t like utilitarianism.
Are you assuming that “stealing money” wouldn’t (or couldn’t possibly?) prove counterproductive to the cause of AI safety research and funding? Because I’m pretty sure there’s no mathematical theorem that rules out the possibility of a criminal action turning out to be counterproductive in practice! And that’s the issue here, not some pristine thought experiment with frictionless planes.
I am using the math that the CEA has pushed. Here’s a quote from The Case For Strong Longtermism by Will MacAskill and Hilary Greaves (page 15):
That would mean that every $100 spent had, on average, an impact as valuable as saving one trillion (resp., one million, 100) lives on our main (resp. low, restricted) estimate – far more than the near-future benefits of bed net distribution.
If 100 dollars could be morally equivalent to saving one trillion lives, then I’d steal money too.
And here is a quote from Nick Bostrom’s paper Astronomical Waste (paragraph 3):
Given these estimates, it follows that the potential for approximately 10^38 human lives is lost every century that colonization of our local supercluster is delayed; or equivalently, about 10^29 potential human lives per second.
If 10^29 human lives are lost every second we delay technological development, then why wait to get the money through more legitimate means?
Of course, this all could backfire. There are risks involved. But the risks are not enough to avoid making the expected value of fraud extremely high. Taking all this into consideration, then, is it any surprise that SBF did what he did?
And again, this is not my math. This is the math pushed by prominent and leading figures in EA. I am just quoting them. Don’t shoot the messenger.
And on that note: I recommend you watch this YouTube video, and use it as a source of reflection:
That math shows that the stakes are high. But that just means that it’s all the more important to make actually prudent choices. My point is that, in the real world, stealing money does not serve the goal of increasing funding for your cause in expectation.
You keep conflating “funding good causes is really important” (EA message) with “stealing money is an effective way to fund important causes” (stupid criminal fantasy).
I think it’s really important to be clear that the EA message and the stupid criminal fantasy are not remotely the same claim.
Edited to add: it would certainly be bad to hold the two views in conjunction. But, between these two claims, the EA message is not the problem.
My point is that, in the real world, stealing money does not serve the goal of increasing funding for your cause in expectation.
Why not? What if we can generate tens of billions of dollars through fraudulent means? We can buy a lot of utility with that money, after all. Perhaps even save humanity from the brink of extinction.
And what if we think we have a fairly good reason to think that we will get away with it. Surely the expected value would start to look pretty enticing by then, no?
Frankly, I’d like to see your calculations. If you really believe that SBF’s fraud did not have net positive value in expectation, then prove it. Do the math for us. At what point does the risk no longer become acceptable, and how much money would it take to offset that risk?
Do you know? Have you run the calculations? Or do you just have faith that the value in expectation will be net negative? Because right now I’m not seeing calculations. I am just seeing unsubstantiated assertions.
But I’ll throw you a bone. For the sake of argument, let’s suppose that you crunch the numbers so that the math conveniently works out. Nice.
But is this math consistent with Nick Bostrom’s math, or Will’s math? Is it consistent with the view that 100 dollars donated to AI safety is worth one trillion human lives? Or that every second of delayed technological development is just as bad as the untimely death of 10^29 people?
On the face of it, it seems extremely improbable that this math could be consistent. Because what if Sam was very likely to get away with it, but just got unlucky? Alternatively, what if the risks are higher but SBF had the potential to become the world’s first trillionaire? Would that change things?
If it does, then this math seems flimsy. So, if we want to reject this flimsiness, then we need to say that Will or Nick’s math is wrong.
But here’s the rub: SBF could have calibrated his own decision theoretic musings to the tune of Nick and Will’s, no? And if he did, that would suggest that Nick and/or Will’s math is dangerous, would it not? And if their math is dangerous, that means that there must be something wrong with EA’s messaging. So perhaps it’s the case that EA—and EV thinking in general—does, in fact, bear some responsibility for this mess.
This brings us to your edit:
the EA message is not the problem.
Care to elaborate on this point?
How do you know this? Are you sure that the EA message is not the problem? What does it mean to say that a message is a ‘problem’, in this case? Would the EA message be a problem if it were true that hadEA never existed, then SBF would never have committed massive financial fraud?
Because this counterfactual claim seems very likely to be correct. (See this Bloomberg article here.) So this would seem to suggest that EA is part of the problem, no?
Because, surely, if EA is causally responsible for this whole debacle, then “the EA message” is at least part of the problem. Or do you disagree?
If you do disagree, then: What does it mean, in your view, for something to be a “problem”? And what exactly would it take for “the EA message” to be “the problem”?
And last, but certainly not least: Is there anything at all that could convince you that EV reasoning is not infallible?
“What if...? Have you run the calculations? … On the face of it...”
Did you even read the OP? Your comments amount to nothing more than “But my naive utilitarian calculations suggest that these bad acts could really easily be justified after all!” Which is simply non-responsive to the arguments against naive utilitarianism.
I’m not going to repeat the whole OP in response to your comments. You repeatedly affirm that you think naive calculations, unconstrained by our most basic social knowledge about reliable vs counterproductive means of achieving social goals, are suited to answering these questions. But that’s precisely the mistake that the OP is arguing against.
Is there anything at all that could convince you that EV reasoning is not infallible?
This is backwards. You are the one repeatedly invoking naive “EV reasoning” (i.e. calculations) as supposedly the true measure of expected value. I’m arguing that true expected value is best approximated when constrained by reliable heuristics.
If you do disagree, then: What does it mean, in your view, for something to be a “problem”?
I mean for it to be false, unjustified, and something we should vociferously warn people against. Not every causal contribution to a bad outcome is a “problem” in this sense. Oxygen also causally contributed to every bad action by a human—without oxygen, the bad act would not have been committed. Even so, oxygen is not the problem.
You repeatedly affirm that you think naive calculations, unconstrained by our most basic social knowledge about reliable vs counterproductive means of achieving social goals, are suited to answering these questions
Where did I say this?
I’m not going to repeat the whole OP in response to your comments.
You’re assuming that you responded to my question in the original post. But you didn’t. Your post just says “trust me guys, the math checks out”. But I see no math. So where did you get this from?
I’m arguing that true expected value is best approximated when constrained by reliable heuristics.
“Arguing”? Or asserting?
If these are arguments, they are not quite strong. No one outside of EA is convinced by this post. I’m not sure if you saw, but this post has even become the subject of ridicule on Twitter.
Not every causal contribution to a bad outcome is a “problem” in this sense. Oxygen also causally contributed to every bad action by a human—without oxygen, the bad act would not have been committed.
Okay, I didn’t realize we were going back to PHIL 101 here. If you need me to spell this out explicitly: SBF chose his career choice because he was encouraged by prominent EA leaders to earn to give. Without EA, he would have never had the means to start FTX. The-earn-to-give model encourages shady business practices.
The connection is obvious.
Saying this has nothing to do with EA is like saying the Stalin’s governance had nothing to do with Marxism.
Denying the link is delusional and makes us look like a cult.
I suspect part of the problem here is that many EAs and philosopher-types tend to conceptualize ethics in terms of abstract theories being the be-all end-all of ethics, e.g. I suspect if you tell them utilitarianism has a problem and they like it more than other theories, they have nothing to fall back upon and keep defending utilitarianism even when it doesn’t make sense. The idea that formal systems of ethics have their own limits and all questions of morality don’t ultimately come down to which abstract theory one favors tends to be a bit of a foreign concept in my experience.
I can’t speak for others, but this isn’t the reason I’m defending utilitarianism. I’d be more than happy to fall back on other types of consequentialism, or moral uncertainty, if necessary (in fact I lean much more towards these than utilitarianism in general). I’m defending it simply because I don’t think that the criticisms being raised are valid for most forms of utilitarianism. See my comments below for more detail on that.
That being said, I do think it’s perfectly reasonable to want a coherent ethical theory that can be used universally. Indeed the alternative is generally considered irrational and can lead to various reductios.
Apologies if this is rude, but your response is a good example of what I was talking about. I don’t think I’ll be able to write a short response without simply restating what I was saying in a slightly different way, but if you want to read a longer version of what I was talking about, you might be interested in my comment summarizing a much longer piece critiquing the use of utilitarianism in EA:
Utilitarianism is not a good theory of everything for morality. It’s helpful and important in some situations, such as when we have to make trade offs about costs and benefits that are relatively commensurate, deal with particular types of uncertainty, and for generating insights. But it doesn’t really work or help in other ways or situations. There are several reasons for this or ideas gesturing in this direction. For one, no theory or model is a theory of everything in any domain, so why should utilitarianism be any different for ethics? For another, utilitarianism doesn’t help us when we have to trade off different kinds of values against each other. Another is that in some situations, we inevitably have to exercise context-dependent judgment that cannot be captured by utilitarianism.
This is not an anti-intellectualism argument that system construction is useless. Rather, this is a judgment about the limits of a particular model or theory. While such a judgment may not be justified from some kind of first principle or more fundamental system, this doesn’t mean the judgment is wrong or unjustified. Part of the fundamental critique is that it is impossible/unworkable to find some kind of complete system that would guide our thinking in all situations; besides infinite regress problems, it is inescapable that we have to make particular moral judgments in specific contexts. This problem cannot be solved by an advanced AI or by assuming that there must be a single theory of everything for morality. Abstract theorizing cannot solve everything.
Utilitarianism has been incredibly helpful, probably critical, for effective altruism, such as in the argument for donating to the most effective global health charities or interventions. It can also lead to undesirable value dictatorship and fanaticism.
But this doesn’t mean EA necessarily has a problem with fanaticism either. It is possible to use utilitarianism in a wise and non-dogmatic manner. In practice most EAs already do something like this, and their actions are influenced by judgment, restraint, and pluralism of values, whatever their stated or endorsed beliefs might be.
The problem is that they don’t really understand why or how they do this beyond that it is desirable and perhaps necessary [is this right?]. People do get off the train to crazy town at some point, but don’t really know how to justify it within their professed/desired framework beside some ad-hoc patches like moral uncertainty. The desire for a complete system that would guide all actions seems reasonable to EAs. EAs lack an understanding of the limits of systemic thinking.
EA should move away from thinking that utilitarianism and abstract moral theories can solve all problems of morality, and instead seek to understand the world as it is better. This may lead to improvements to EA efforts in policy, politics, and other social contexts where game-theoretic considerations and judgment play critical roles, and where consequentialist reasoning can be detrimental.
No worries. It is interesting though that you think my comment is a great example when it was meant to be a rebuttal. What I’m trying to say is, I wouldn’t really identify as a ‘utilitarian’ myself, so I don’t think I really have a vested interest in this debate. Nonetheless, I don’t think utilitarianism ‘breaks down’ in this scenario, as you seem to be suggesting. I think very poorly-formulated versions do, but those are not commonly defended, and with some adjustments utilitarianism can accomodate most of our intuitions very well (including the ones that are relevant here). I’m also not sure what the basis is of the suggestion that utilitarianism works worse when a situation is more unique and there is more context to factor in.
To reiterate, I think the right move is (progressive) adjustments to a theory, and moral uncertainty (where relevant), which both seem significantly more rational than particularism. It’s very unclear to me how we can know that it’s ‘impossible or unworkable’ to find a system that would guide our thinking in all situations. Indeed some versions of moral uncertainty already seem to do this pretty well. I also would object to classifying moral uncertainty as an ‘ad-hoc patch’. It wasn’t initially developed to better accommodate our intuitions, but simply because as a matter of fact we find ourselves in the position of uncertainty with respect to what moral theory is correct (or ‘preferable’), just like with empirical uncertainty.
I think it was a good example (I changed the wording from ‘great’ to ‘good’) because my point was more about the role of abstract and formal theories of ethics rather than restricted to utilitarianism itself, and your response was defending abstract theories as the ultimate foundation for ethics. The point (which I am likely communicating badly to someone with different beliefs) is that formal systems have limits and are imperfectly applied by flawed humans with limited time, information, etc. It is all well and good to talk about making adjustments to theories to refine them, and indeed philosophers should do so, but applying them to real life is necessarily an imperfect process.
I agree with the distinction between naive and prudent utilitarianism, but I also think it all breaks down when you factor in the infinite expected value of mitigating extinction risks (assuming a potential techno utopian future, as some longtermists do) and the risk of AGI driven extinction in our lifetimes.
I’m pretty sure lots of pure prudent utilitarians would still endorse stealing money to fund AI safety research, especially as we get closer in time to the emergence of AGI.
(https://forum.effectivealtruism.org/posts/2wdanfCRFbNmWebmy/ea-should-consider-explicitly-rejecting-pure-classical-total)
A lot of people are disagreeing with you. But what you are saying is just uncontroversially correct, as anyone who studies philosophy outside of EA would tell you.
To those disagree-downvoting: I would like to suggest the possibility that the problem, here, is with utilitarianism. Not with anything freedomandutility said. Because what they said is literally just true as a matter of math. If you don’t like that, then you don’t like utilitarianism.
Are you assuming that “stealing money” wouldn’t (or couldn’t possibly?) prove counterproductive to the cause of AI safety research and funding? Because I’m pretty sure there’s no mathematical theorem that rules out the possibility of a criminal action turning out to be counterproductive in practice! And that’s the issue here, not some pristine thought experiment with frictionless planes.
I am using the math that the CEA has pushed. Here’s a quote from The Case For Strong Longtermism by Will MacAskill and Hilary Greaves (page 15):
If 100 dollars could be morally equivalent to saving one trillion lives, then I’d steal money too.
And here is a quote from Nick Bostrom’s paper Astronomical Waste (paragraph 3):
If 10^29 human lives are lost every second we delay technological development, then why wait to get the money through more legitimate means?
Of course, this all could backfire. There are risks involved. But the risks are not enough to avoid making the expected value of fraud extremely high. Taking all this into consideration, then, is it any surprise that SBF did what he did?
And again, this is not my math. This is the math pushed by prominent and leading figures in EA. I am just quoting them. Don’t shoot the messenger.
And on that note: I recommend you watch this YouTube video, and use it as a source of reflection:
That math shows that the stakes are high. But that just means that it’s all the more important to make actually prudent choices. My point is that, in the real world, stealing money does not serve the goal of increasing funding for your cause in expectation.
You keep conflating “funding good causes is really important” (EA message) with “stealing money is an effective way to fund important causes” (stupid criminal fantasy).
I think it’s really important to be clear that the EA message and the stupid criminal fantasy are not remotely the same claim.
Edited to add: it would certainly be bad to hold the two views in conjunction. But, between these two claims, the EA message is not the problem.
Why not? What if we can generate tens of billions of dollars through fraudulent means? We can buy a lot of utility with that money, after all. Perhaps even save humanity from the brink of extinction.
And what if we think we have a fairly good reason to think that we will get away with it. Surely the expected value would start to look pretty enticing by then, no?
Frankly, I’d like to see your calculations. If you really believe that SBF’s fraud did not have net positive value in expectation, then prove it. Do the math for us. At what point does the risk no longer become acceptable, and how much money would it take to offset that risk?
Do you know? Have you run the calculations? Or do you just have faith that the value in expectation will be net negative? Because right now I’m not seeing calculations. I am just seeing unsubstantiated assertions.
But I’ll throw you a bone. For the sake of argument, let’s suppose that you crunch the numbers so that the math conveniently works out. Nice.
But is this math consistent with Nick Bostrom’s math, or Will’s math? Is it consistent with the view that 100 dollars donated to AI safety is worth one trillion human lives? Or that every second of delayed technological development is just as bad as the untimely death of 10^29 people?
On the face of it, it seems extremely improbable that this math could be consistent. Because what if Sam was very likely to get away with it, but just got unlucky? Alternatively, what if the risks are higher but SBF had the potential to become the world’s first trillionaire? Would that change things?
If it does, then this math seems flimsy. So, if we want to reject this flimsiness, then we need to say that Will or Nick’s math is wrong.
But here’s the rub: SBF could have calibrated his own decision theoretic musings to the tune of Nick and Will’s, no? And if he did, that would suggest that Nick and/or Will’s math is dangerous, would it not? And if their math is dangerous, that means that there must be something wrong with EA’s messaging. So perhaps it’s the case that EA—and EV thinking in general—does, in fact, bear some responsibility for this mess.
This brings us to your edit:
Care to elaborate on this point?
How do you know this? Are you sure that the EA message is not the problem? What does it mean to say that a message is a ‘problem’, in this case? Would the EA message be a problem if it were true that had EA never existed, then SBF would never have committed massive financial fraud ?
Because this counterfactual claim seems very likely to be correct. (See this Bloomberg article here.) So this would seem to suggest that EA is part of the problem, no?
Because, surely, if EA is causally responsible for this whole debacle, then “the EA message” is at least part of the problem. Or do you disagree?
If you do disagree, then: What does it mean, in your view, for something to be a “problem”? And what exactly would it take for “the EA message” to be “the problem”?
And last, but certainly not least: Is there anything at all that could convince you that EV reasoning is not infallible?
“What if...? Have you run the calculations? … On the face of it...”
Did you even read the OP? Your comments amount to nothing more than “But my naive utilitarian calculations suggest that these bad acts could really easily be justified after all!” Which is simply non-responsive to the arguments against naive utilitarianism.
I’m not going to repeat the whole OP in response to your comments. You repeatedly affirm that you think naive calculations, unconstrained by our most basic social knowledge about reliable vs counterproductive means of achieving social goals, are suited to answering these questions. But that’s precisely the mistake that the OP is arguing against.
This is backwards. You are the one repeatedly invoking naive “EV reasoning” (i.e. calculations) as supposedly the true measure of expected value. I’m arguing that true expected value is best approximated when constrained by reliable heuristics.
I mean for it to be false, unjustified, and something we should vociferously warn people against. Not every causal contribution to a bad outcome is a “problem” in this sense. Oxygen also causally contributed to every bad action by a human—without oxygen, the bad act would not have been committed. Even so, oxygen is not the problem.
Where did I say this?
You’re assuming that you responded to my question in the original post. But you didn’t. Your post just says “trust me guys, the math checks out”. But I see no math. So where did you get this from?
“Arguing”? Or asserting?
If these are arguments, they are not quite strong. No one outside of EA is convinced by this post. I’m not sure if you saw, but this post has even become the subject of ridicule on Twitter.
Okay, I didn’t realize we were going back to PHIL 101 here. If you need me to spell this out explicitly: SBF chose his career choice because he was encouraged by prominent EA leaders to earn to give. Without EA, he would have never had the means to start FTX. The-earn-to-give model encourages shady business practices.
The connection is obvious.
Saying this has nothing to do with EA is like saying the Stalin’s governance had nothing to do with Marxism.
Denying the link is delusional and makes us look like a cult.
I suspect part of the problem here is that many EAs and philosopher-types tend to conceptualize ethics in terms of abstract theories being the be-all end-all of ethics, e.g. I suspect if you tell them utilitarianism has a problem and they like it more than other theories, they have nothing to fall back upon and keep defending utilitarianism even when it doesn’t make sense. The idea that formal systems of ethics have their own limits and all questions of morality don’t ultimately come down to which abstract theory one favors tends to be a bit of a foreign concept in my experience.
I can’t speak for others, but this isn’t the reason I’m defending utilitarianism. I’d be more than happy to fall back on other types of consequentialism, or moral uncertainty, if necessary (in fact I lean much more towards these than utilitarianism in general). I’m defending it simply because I don’t think that the criticisms being raised are valid for most forms of utilitarianism. See my comments below for more detail on that.
That being said, I do think it’s perfectly reasonable to want a coherent ethical theory that can be used universally. Indeed the alternative is generally considered irrational and can lead to various reductios.
Apologies if this is rude, but your response is a good example of what I was talking about. I don’t think I’ll be able to write a short response without simply restating what I was saying in a slightly different way, but if you want to read a longer version of what I was talking about, you might be interested in my comment summarizing a much longer piece critiquing the use of utilitarianism in EA:
No worries. It is interesting though that you think my comment is a great example when it was meant to be a rebuttal. What I’m trying to say is, I wouldn’t really identify as a ‘utilitarian’ myself, so I don’t think I really have a vested interest in this debate. Nonetheless, I don’t think utilitarianism ‘breaks down’ in this scenario, as you seem to be suggesting. I think very poorly-formulated versions do, but those are not commonly defended, and with some adjustments utilitarianism can accomodate most of our intuitions very well (including the ones that are relevant here). I’m also not sure what the basis is of the suggestion that utilitarianism works worse when a situation is more unique and there is more context to factor in.
To reiterate, I think the right move is (progressive) adjustments to a theory, and moral uncertainty (where relevant), which both seem significantly more rational than particularism. It’s very unclear to me how we can know that it’s ‘impossible or unworkable’ to find a system that would guide our thinking in all situations. Indeed some versions of moral uncertainty already seem to do this pretty well. I also would object to classifying moral uncertainty as an ‘ad-hoc patch’. It wasn’t initially developed to better accommodate our intuitions, but simply because as a matter of fact we find ourselves in the position of uncertainty with respect to what moral theory is correct (or ‘preferable’), just like with empirical uncertainty.
I think it was a good example (I changed the wording from ‘great’ to ‘good’) because my point was more about the role of abstract and formal theories of ethics rather than restricted to utilitarianism itself, and your response was defending abstract theories as the ultimate foundation for ethics. The point (which I am likely communicating badly to someone with different beliefs) is that formal systems have limits and are imperfectly applied by flawed humans with limited time, information, etc. It is all well and good to talk about making adjustments to theories to refine them, and indeed philosophers should do so, but applying them to real life is necessarily an imperfect process.