It’s embarrassing that EA has been so far reluctant to discuss the plainly obvious fact that SBF’s risk-taking is linked to his EA philosophy. It’s incredibly obvious that it was. Everyone outside of EA already knows this.
The reasoning is fairly straightforward: a double or nothing coin-flip has high value in expectation, even when you scale it to the billions of dollars. So risky decisions can still have high expected value. This is true even if the risk includes, for example, the situation we are currently in.
Because if that risk were small enough, or if the rewards for getting away with it were high enough, then SBFs gambles could have plausibly had high value in expectation. Consider also the fact that EV reasoning leads to practically unbounded upsides.
Put all this together and that leads us to fanaticism. As it happens, a good definition of ‘fanaticism’ can be found in SBF’s blog (linked in the OP):
The argument, roughly goes: when computing expected impact of causes, mine is 10^30 times higher than any other, so nothing else matters. For instance, there are 10^58 future humans, so increasing the odds that they exist by even .0001% is still worth 10^44 times more important that anything that impacts current humans.
So it seems like SBF was at least aware of fanaticism. And that’s no surprise. We’ve known this for a while. Because there has been talk on this forum for years about fanaticism. SBF surely was privy to some of these online discussions, as well as some offline discussions too. So perhaps he took fanaticism to heart. If he did, that would be unsurprising.
Because many prominent EAs have promoted fanaticism. Consider for example this forum post from the Global Priorities Institute called In Defense of Fanaticism. Here’s the abstract:
Consider a decision between: 1) a certainty of a moderately good outcome, such as one additional life saved; 2) a lottery which probably gives a worse outcome, but has a tiny probability of some vastly better outcome (perhaps trillions of additional blissful lives created). Which is morally better? By expected value theory (with a plausible axiology), no matter how tiny its probability of the better outcome, (2) will be better than (1) as long as that better outcome is good enough. But this seems fanatical. So you may be tempted to abandon expected value theory.
But not so fast — denying all such fanatical verdicts brings serious problems. For one, you must reject either that moral betterness is transitive or even a weak principle of tradeoffs. For two, you must accept that judgements are either: inconsistent over structurally-identical pairs of lotteries; or absurdly sensitive to small probability differences. For three, you must accept that the practical judgements of agents like us are sensitive to our beliefs about far-off events that are unaffected by our actions. And, for four, you may also be forced to accept judgements which you know you would reject if you simply learned more. Better to accept fanaticism than these implications.
And this blog posts corresponds to a paper from the Global Priorities Institute, which can be found here. Notice that the math in defence of fanaticism, here, is pretty rigorous. Now compare that to recent defences of utilitarian-minded EV reasoning (like, for example, this post).
There is an asymmetry here. The defences of fanaticism are quite rigorous, mathematically speaking, whereas recent defences of utilitarianism + EV theory involve no math at all; the defence of fanaticism engages with its critics arguments head-on, whereas recent defences of utilitarianism do not engage with any critical arguments at all. So what line of reasoning would be more convincing to SBF? A hand wave, or a rigorous mathematical argument?
Since SBF is clearly a quantitatively minded person, it’s quite plausible that the mathematical rigour was more appealing to him. And, unfortunately, that is quite plausibly why we are in the situation we are currently in.
To anyone who disagrees: I respectfully ask that you disagree with me mathematically. We know the quantity of money that SBF was working with, because we can estimate his expected earnings. We can estimate the risks he was working with. We can estimate the downsides of those risks (like bad PR and so on), too. We can give all these things a range to account for uncertainty, too. Once we have collected all these numbers, we can plug them into our EV calculus. So if you think that SBF made decisions with negative expected utility, please show me why by showing me your numbers.
But until then, I think it’s best we admit that SBF’s behaviour is linked to his utilitarian-minded EV reasoning.
If we fail to own up to this, then we are being dogmatic.
Great post.
It’s embarrassing that EA has been so far reluctant to discuss the plainly obvious fact that SBF’s risk-taking is linked to his EA philosophy. It’s incredibly obvious that it was. Everyone outside of EA already knows this.
The reasoning is fairly straightforward: a double or nothing coin-flip has high value in expectation, even when you scale it to the billions of dollars. So risky decisions can still have high expected value. This is true even if the risk includes, for example, the situation we are currently in.
Because if that risk were small enough, or if the rewards for getting away with it were high enough, then SBFs gambles could have plausibly had high value in expectation. Consider also the fact that EV reasoning leads to practically unbounded upsides.
Put all this together and that leads us to fanaticism. As it happens, a good definition of ‘fanaticism’ can be found in SBF’s blog (linked in the OP):
So it seems like SBF was at least aware of fanaticism. And that’s no surprise. We’ve known this for a while. Because there has been talk on this forum for years about fanaticism. SBF surely was privy to some of these online discussions, as well as some offline discussions too. So perhaps he took fanaticism to heart. If he did, that would be unsurprising.
Because many prominent EAs have promoted fanaticism. Consider for example this forum post from the Global Priorities Institute called In Defense of Fanaticism. Here’s the abstract:
And this blog posts corresponds to a paper from the Global Priorities Institute, which can be found here. Notice that the math in defence of fanaticism, here, is pretty rigorous. Now compare that to recent defences of utilitarian-minded EV reasoning (like, for example, this post).
There is an asymmetry here. The defences of fanaticism are quite rigorous, mathematically speaking, whereas recent defences of utilitarianism + EV theory involve no math at all; the defence of fanaticism engages with its critics arguments head-on, whereas recent defences of utilitarianism do not engage with any critical arguments at all. So what line of reasoning would be more convincing to SBF? A hand wave, or a rigorous mathematical argument?
Since SBF is clearly a quantitatively minded person, it’s quite plausible that the mathematical rigour was more appealing to him. And, unfortunately, that is quite plausibly why we are in the situation we are currently in.
To anyone who disagrees: I respectfully ask that you disagree with me mathematically. We know the quantity of money that SBF was working with, because we can estimate his expected earnings. We can estimate the risks he was working with. We can estimate the downsides of those risks (like bad PR and so on), too. We can give all these things a range to account for uncertainty, too. Once we have collected all these numbers, we can plug them into our EV calculus. So if you think that SBF made decisions with negative expected utility, please show me why by showing me your numbers.
But until then, I think it’s best we admit that SBF’s behaviour is linked to his utilitarian-minded EV reasoning.
If we fail to own up to this, then we are being dogmatic.