We unreasonably narrowed our probability space. As Violet Hour says below win somehow discounted this whole hypothesis without even considering it. Even a 1% chance of this massive scandal was worth someone thinking about it full-time. And we dropped the ball.
I don’t think our primary problem was with flawed probability assignments over some set of explicitly considered hypotheses. If I were to continue with the probabilistic metaphor, I’d be much more tempted to say that we erred in our formation of the practical hypothesis space — that is, the finite set of hypothesis that the EA community considered to be salient, and worthy of extended discussion.
Afaik (and at least in my personal experience), very few EAs seemed to think the potential malfeasance of FTX was an important topic to discuss. Because the topics weren’t salient, few people bothered to assign explicit probabilities. To me, the fact that we weren’t focusing on the right topics, in some nebulous sense, is more concerning than the ways in which we erred in assigning probabilities to claims within our practical hypothesis space.
Nathan (or anyone who agrees with the comment above), can you even list all the things that, if they have even a 1% chance of failing then they are “worth someone thinking about in full-time”?
Can you find at least 1 (or 3?) people who’d be competent at looking at these topics, and that you’d realistically remove from their current roles to look at the things-that-might-fail?
Saying you’d look specifically at the FTX-fraud possibility is hindsight bias. In reality, we’d have to look at many similar topics in advance, without knowing which of them is going to fail. Will you bite the bullet?
I think that the core scandal is massive fraud, not exchange failure. The base rate of the fraud is presumably lower (though idk the base rates—wouldn’t surprise me if in crypto the exchange failure rate was more like 30% and fraud rate was 5% − 5% failure seems crazy low)
Fraud is actually a high probability event. Over the the last 20 years especially with similar cases like ENRON in 2001 and LEHMAN Brothers in 2008, traditional institutions - (eg. publicly held corporations or banks) have coped with this through governance boards and internal audit departments.
I think newer industries like Crypto, AI and Space are slower to adapt to these concepts as newer pioneers are setting up managements that are unaware of the fraudulent past, the corruptive nature of the ability to acquire money and be able to hide/ misappropriate it. Well, I could understand why, startups won’t spend more money for governance and reviews as it does not have a direct impact to the overall product or service and as they eventually get bigger—they fail to realize the need to review the control features that worked when they were still small...
I should put fraud risk to as high as 10% - could go higher if the internal controls are weak or non-existent. Like who checks SBF? or all of the funders? that is a great thing to start reviewing moving forward in my opinion. The cost of any scandal is too large that it will affect the EA community’s ability to deliver its objectives.
So putting it another way,we failed at recursively applying it to ourselves. We didn’t apply the logic of x-risk and GCRs to ourselves, and didn’t go far enough.
And if the risk was 10%, shouldn’t that have been the headline. “TEN PERCENT CHANCE THAT THERE IS ABOUT TO BE A PANDEMIC THAT DEVASTATES THE GLOBAL ECONOMY, KILLS HUNDREDS OF THOUSANDS OF PEOPLE, AND PREVENTS YOU FROM LEAVING YOUR HOUSE FOR MONTHS”? Isn’t that a better headline than Coronavirus panic sells as alarmist information spreads on social media? But that’s the headline you could have written if your odds were ten percent!
I know some people think EAs should have been far more suspicious of SBF than professional investors, the media etc were, but this stat from Benjamin_Todd nevertheless seems relevant:
in early 2022, metaculus had a 1% chance of FTX making any default on customer funds over the year with ~40 forecasters
Updated
We unreasonably narrowed our probability space. As Violet Hour says below win somehow discounted this whole hypothesis without even considering it. Even a 1% chance of this massive scandal was worth someone thinking about it full-time. And we dropped the ball.
Upvoted, but I disagree with this framing.
I don’t think our primary problem was with flawed probability assignments over some set of explicitly considered hypotheses. If I were to continue with the probabilistic metaphor, I’d be much more tempted to say that we erred in our formation of the practical hypothesis space — that is, the finite set of hypothesis that the EA community considered to be salient, and worthy of extended discussion.
Afaik (and at least in my personal experience), very few EAs seemed to think the potential malfeasance of FTX was an important topic to discuss. Because the topics weren’t salient, few people bothered to assign explicit probabilities. To me, the fact that we weren’t focusing on the right topics, in some nebulous sense, is more concerning than the ways in which we erred in assigning probabilities to claims within our practical hypothesis space.
(disagree)
Nathan (or anyone who agrees with the comment above), can you even list all the things that, if they have even a 1% chance of failing then they are “worth someone thinking about in full-time”?
Can you find at least 1 (or 3?) people who’d be competent at looking at these topics, and that you’d realistically remove from their current roles to look at the things-that-might-fail?
Saying you’d look specifically at the FTX-fraud possibility is hindsight bias. In reality, we’d have to look at many similar topics in advance, without knowing which of them is going to fail. Will you bite the bullet?
Reminds me of this post:
https://slatestarcodex.com/2018/05/23/should-psychiatry-test-for-lead-more/
Which I recommend as “A Scott Alexander post that stuck with me and changes the way I think sometimes”
The failure rate of crypto exchanges is like 5%. I think this was a really large risk.
I think that the core scandal is massive fraud, not exchange failure. The base rate of the fraud is presumably lower (though idk the base rates—wouldn’t surprise me if in crypto the exchange failure rate was more like 30% and fraud rate was 5% − 5% failure seems crazy low)
Yeah, but this ignores my question.
(For example, how many other things are there with a failure rate of like 5% and that would be very important if they failed, to EA?)
Hi Nathan,
Fraud is actually a high probability event. Over the the last 20 years especially with similar cases like ENRON in 2001 and LEHMAN Brothers in 2008, traditional institutions - (eg. publicly held corporations or banks) have coped with this through governance boards and internal audit departments.
I think newer industries like Crypto, AI and Space are slower to adapt to these concepts as newer pioneers are setting up managements that are unaware of the fraudulent past, the corruptive nature of the ability to acquire money and be able to hide/ misappropriate it. Well, I could understand why, startups won’t spend more money for governance and reviews as it does not have a direct impact to the overall product or service and as they eventually get bigger—they fail to realize the need to review the control features that worked when they were still small...
I should put fraud risk to as high as 10% - could go higher if the internal controls are weak or non-existent. Like who checks SBF? or all of the funders? that is a great thing to start reviewing moving forward in my opinion. The cost of any scandal is too large that it will affect the EA community’s ability to deliver its objectives.
All the best,
Miguel
So putting it another way,we failed at recursively applying it to ourselves. We didn’t apply the logic of x-risk and GCRs to ourselves, and didn’t go far enough.
Yes, as Scott Alexander put it in the context of Covid, a failure, but not of prediction.
I know some people think EAs should have been far more suspicious of SBF than professional investors, the media etc were, but this stat from Benjamin_Todd nevertheless seems relevant: