This feels like a weird interpretation of Will’s comment, which doesn’t (in my view) imply that for-profit companies can’t do a lot of harm, but rather that if you start a company with the sole goal of making a profit, usually the worst outcome (with regards to your goal of making a profit) is that you go bankrupt.
As FTX just spectacularly demonstrated, Will was wrong. This is because even though FTX was ostensibly started with the sole goal of making a profit, it turns out there were other important implicit goals like “don’t steal billions of dollars from thousands of people”, implicit goals like that always exist, and failure to meet those implicit goals is very bad.
This feels like a weird interpretation of Will’s comment, which doesn’t (in my view) imply that for-profit companies can’t do a lot of harm, but rather that if you start a company with the sole goal of making a profit, usually the worst outcome (with regards to your goal of making a profit) is that you go bankrupt.
As FTX just spectacularly demonstrated, Will was wrong. This is because even though FTX was ostensibly started with the sole goal of making a profit, it turns out there were other important implicit goals like “don’t steal billions of dollars from thousands of people”, implicit goals like that always exist, and failure to meet those implicit goals is very bad.
This sounds like a human form of alignment failure, specifically, the What Failure Looks Like story part I.
Here’s a link to it:
https://www.lesswrong.com/posts/HBxe6wdjxK239zajf/what-failure-looks-like