Agree with your post and want to add one thing. Ultimately this was a failure of the EA ideas more so than the EA community. SBF used EA ideas as a justification for his actions. Very few EAs would condone his amoral stance w.r.t. business ethics, but business ethics isn’t really a central part of EA ideas. Ultimately, I think the main failure was EAs failing to adequately condemn naive utilitarianism.
I think back to the old Scott Alexander post about the rationalist community: Yes, We Have Noticed The Skulls | Slate Star Codex. I think he makes a valid point, that the rationalist community has tried to address the obvious failure modes of rationalism. This is also true of the EA community, in that there has absolutely been some criticism of galaxy brained naive utilitarianism. However, there is a certain defensiveness in Scott’s post, an annoyance that people keep bringing up past failure modes even though rationalists try really hard to not fail that way again. I suspect this same defensiveness may have played a role in EA culture. Utilitarianism has always been criticized for the potential that it could be used to justify...well, SBF-style behavior. EAs can argue that we have newer and better formulations of utilitarianism / moral theory that don’t run into that problem, and this is true (in theory). However, I do suspect that this topic was undervalued in the EA community, simply because we were super annoyed at critics that keep harping on the risks of naive utilitarianism even though clearly no real EA actually endorses naive utilitarianism.
Ultimately this was a failure of the EA ideas more so than the EA community. SBF used EA ideas as a justification for his actions. Very few EAs would condone his amoral stance w.r.t. business ethics, but business ethics isn’t really a central part of EA ideas. Ultimately, I think the main failure was EAs failing to adequately condemn naive utilitarianism.
So I disagree with this because:
It’s unclear whether it’s right to attribute SBF’s choices to a failure of EA ideas. Following SBF’s interview with Kelsey Piper and based on other things I’ve been reading, I don’t think we can be sure at this point whether SBF was generally more motivated by naive utilitarianism or by seeking to expand his own power and influence. And it’s unclear which of those headspaces led him to the decision to defraud FTX customers.
It’s plausible there actually were serious ways that the EA community failed with respect to SBF. According to acoupleaccounts, at least several people in the community had reason to believe SBF was dishonest and sketchy. Some of them spoke up about it and others didn’t. The accounts say that these concerns were shared with more central leaders in EA who didn’t take a lot of action based on that information (e.g. they could have stopped promoting Sam as a shining example of an EA after learning of reports that he was dishonest, even if they continued to accept funding from him). [1]
If this story is true (don’t know for sure yet), then that would likely point to community failures in the sense that EA had a fairly centralized network of community/funding that was vulnerable, and it failed to distance itself from a known or suspected bad actor. This is pretty close to the OP’s point about the EA community being high-trust and so far not developing sufficient mechanisms to verify that trust as it has scaled.
--
[1]: I do want to clarify that in addition to this story still not being unconfirmed, I’m mostly not trying to place a ton of blame or hostility on EA leaders who may have made mistakes. Leadership is hard, the situation sounds hard and I think EA leaders have done a lot of good things outside of this situation. What we find out may reduce how much responsibility I think the EA movement should put with those people, but overall I’m much more interested in looking at systemic problems/solutions than fixating on the blame of individuals.
Agree with your post and want to add one thing. Ultimately this was a failure of the EA ideas more so than the EA community. SBF used EA ideas as a justification for his actions. Very few EAs would condone his amoral stance w.r.t. business ethics, but business ethics isn’t really a central part of EA ideas. Ultimately, I think the main failure was EAs failing to adequately condemn naive utilitarianism.
I think back to the old Scott Alexander post about the rationalist community: Yes, We Have Noticed The Skulls | Slate Star Codex. I think he makes a valid point, that the rationalist community has tried to address the obvious failure modes of rationalism. This is also true of the EA community, in that there has absolutely been some criticism of galaxy brained naive utilitarianism. However, there is a certain defensiveness in Scott’s post, an annoyance that people keep bringing up past failure modes even though rationalists try really hard to not fail that way again. I suspect this same defensiveness may have played a role in EA culture. Utilitarianism has always been criticized for the potential that it could be used to justify...well, SBF-style behavior. EAs can argue that we have newer and better formulations of utilitarianism / moral theory that don’t run into that problem, and this is true (in theory). However, I do suspect that this topic was undervalued in the EA community, simply because we were super annoyed at critics that keep harping on the risks of naive utilitarianism even though clearly no real EA actually endorses naive utilitarianism.
So I disagree with this because:
It’s unclear whether it’s right to attribute SBF’s choices to a failure of EA ideas. Following SBF’s interview with Kelsey Piper and based on other things I’ve been reading, I don’t think we can be sure at this point whether SBF was generally more motivated by naive utilitarianism or by seeking to expand his own power and influence. And it’s unclear which of those headspaces led him to the decision to defraud FTX customers.
It’s plausible there actually were serious ways that the EA community failed with respect to SBF. According to a couple accounts, at least several people in the community had reason to believe SBF was dishonest and sketchy. Some of them spoke up about it and others didn’t. The accounts say that these concerns were shared with more central leaders in EA who didn’t take a lot of action based on that information (e.g. they could have stopped promoting Sam as a shining example of an EA after learning of reports that he was dishonest, even if they continued to accept funding from him). [1]
If this story is true (don’t know for sure yet), then that would likely point to community failures in the sense that EA had a fairly centralized network of community/funding that was vulnerable, and it failed to distance itself from a known or suspected bad actor. This is pretty close to the OP’s point about the EA community being high-trust and so far not developing sufficient mechanisms to verify that trust as it has scaled.
--
[1]: I do want to clarify that in addition to this story still not being unconfirmed, I’m mostly not trying to place a ton of blame or hostility on EA leaders who may have made mistakes. Leadership is hard, the situation sounds hard and I think EA leaders have done a lot of good things outside of this situation. What we find out may reduce how much responsibility I think the EA movement should put with those people, but overall I’m much more interested in looking at systemic problems/solutions than fixating on the blame of individuals.