Some thoughts - Utilitiarianism but being cautious around the weird/unilateral stuff is still good - We shouldn’t be surprised that we didn’t figure out SBF was fraudulent quicker than billions of dollars of cryto money… and Michael Lewis - Scandal prediction markets are the solution here and one day they will be normal. But not today. Don’t boo me, I’m right - Everyone wants whistleblowing, no one wants the correctly incentivised decentralised form of whistleblowing. - Gotta say, I feel for many random individual people who knew or interacted closely with SBF but weren’t at FTX who are gonna get caught up in that - We were fundamentally unserious about avoiding reputational risk from crypto. I hope we are more serious about not dying from AI - I like you all a lot - I don’t mind taking the money of some retired non-EA oil baron, but I think not returning FTX’s money perhaps incentivises future pro-crime EAs. I would like a credible signal - The community does not need democratised funding (though I’d happily test it at a small scale) though we aren’t getting enough whistleblowing so we should work on that - We deserve to be scrutinised and mocked, we messed up. We should own that - X-risk is still extremely compelling - I am uncertain how impactful my work is - Our critics are usually very low signal but have a few key things of value to say. It is hard to listen to find those things without wasting loads of time, but missing them is bad too - People knew SBF was a bully who broke promises. That that information didn’t flow to where it needed/ was ignored was a problem— I think we shouldn’t say we want criticism, because we don’t. We didn’t want it about FTX and we don’t in any other places. We want very specific criticism. Everyone does, because the world is big and we have limited time. So how do we get the criticism that’s most useful to us - The community should seek to make the best funding decisions it can over time. I think that’s with orgs doing it and prediction markets to remove bad apples, but you can think what you want. But democratisation isn’t a goal in and of itself—good sustainable decisionmaking is. Perhaps there should be a jury of randomly chosen community member, perhaps we should have elections. I don’t know, but I do feel we haven’t been taking governance seriously enough
I remain confused about “utilitarianism, but use good judgement”. IMO, it’s amongst the more transparent motte-and-baileys I’ve seen. Here are two tweets from Eliezer that I see are regularly re-shared:
The rules say we must use consequentialism, but good people are deontologists, and virtue ethics is what actually works.
Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you have become a god.
This describes Aristotelian Virtue Ethics—finding the golden mean between excess and deficiency. So are people here actually virtue ethicists who sometimes use math as a means of justification and explanation? Or do they continue to take utilitarianism to some of its weirder places, privately and publicly, but strategically seek shelter under other moral frameworks when criticized?
I’m finding it harder to take people who put “consequentialist” and “utilitarian” in their profiles and about mes seriously. If people abandon their stated moral framework on big important and consequential questions, then either they’re deluding themselves on what their moral framework actually is, or they really will act out the weird conclusions—but are being manipulative and strategic by saying “trust us, we have checks and balances”
And what happens when that double-checking comes back negative? And how much weight do you choose to give it? The answer seems to be rooted in matters of judgement and subjectivity. And if you’re doing it often enough, especially on questions of consequence, then that moral framework is better described as virtue ethics.
Out of curiosity, how would you say your process differs from a virtue ethicist trying to find the golden mean between excess and deficiency?
Some thoughts
- Utilitiarianism but being cautious around the weird/unilateral stuff is still good
- We shouldn’t be surprised that we didn’t figure out SBF was fraudulent quicker than billions of dollars of cryto money… and Michael Lewis
- Scandal prediction markets are the solution here and one day they will be normal. But not today. Don’t boo me, I’m right
- Everyone wants whistleblowing, no one wants the correctly incentivised decentralised form of whistleblowing.
- Gotta say, I feel for many random individual people who knew or interacted closely with SBF but weren’t at FTX who are gonna get caught up in that
- We were fundamentally unserious about avoiding reputational risk from crypto. I hope we are more serious about not dying from AI
- I like you all a lot
- I don’t mind taking the money of some retired non-EA oil baron, but I think not returning FTX’s money perhaps incentivises future pro-crime EAs. I would like a credible signal
- The community does not need democratised funding (though I’d happily test it at a small scale) though we aren’t getting enough whistleblowing so we should work on that
- We deserve to be scrutinised and mocked, we messed up. We should own that
- X-risk is still extremely compelling
- I am uncertain how impactful my work is
- Our critics are usually very low signal but have a few key things of value to say. It is hard to listen to find those things without wasting loads of time, but missing them is bad too
- People knew SBF was a bully who broke promises. That that information didn’t flow to where it needed/ was ignored was a problem—
I think we shouldn’t say we want criticism, because we don’t. We didn’t want it about FTX and we don’t in any other places. We want very specific criticism. Everyone does, because the world is big and we have limited time. So how do we get the criticism that’s most useful to us
- The community should seek to make the best funding decisions it can over time. I think that’s with orgs doing it and prediction markets to remove bad apples, but you can think what you want. But democratisation isn’t a goal in and of itself—good sustainable decisionmaking is. Perhaps there should be a jury of randomly chosen community member, perhaps we should have elections. I don’t know, but I do feel we haven’t been taking governance seriously enough
I remain confused about “utilitarianism, but use good judgement”. IMO, it’s amongst the more transparent motte-and-baileys I’ve seen. Here are two tweets from Eliezer that I see are regularly re-shared:
This describes Aristotelian Virtue Ethics—finding the golden mean between excess and deficiency. So are people here actually virtue ethicists who sometimes use math as a means of justification and explanation? Or do they continue to take utilitarianism to some of its weirder places, privately and publicly, but strategically seek shelter under other moral frameworks when criticized?
I’m finding it harder to take people who put “consequentialist” and “utilitarian” in their profiles and about mes seriously. If people abandon their stated moral framework on big important and consequential questions, then either they’re deluding themselves on what their moral framework actually is, or they really will act out the weird conclusions—but are being manipulative and strategic by saying “trust us, we have checks and balances”
I don’t think you have to abandon it, but you can look twice or ask trusted friends etc etc.
That doesn’t mean you can’t do the thing you intended to do.
And what happens when that double-checking comes back negative? And how much weight do you choose to give it? The answer seems to be rooted in matters of judgement and subjectivity. And if you’re doing it often enough, especially on questions of consequence, then that moral framework is better described as virtue ethics.
Out of curiosity, how would you say your process differs from a virtue ethicist trying to find the golden mean between excess and deficiency?