Create prediction markets and forecasting questions on AI risk and biorisk. I also work part-time at a prediction market.
Use my connections on Twitter to raise the profile of these predictions and increase the chance that decision-makers discuss these issues.
I liked CEA’s statement
Writing statements like this is really hard. It’s the equivalent of writing one tweet on something that you know everyone is gonna rip to pieces. I think there are tradeoffs here that people on the forum don’t seem to acknowledge. I am very confident (90%) that a page length discussion of this would have been worse in terms of outcomes.
I don’t think it was for us—I think it was for journalists etc. And I think it performed its job of EA not being dragged into all of this. Note how much better it was than either Anders’ statement or Bostrom’s—no one externally is discussing it, and in an adversarial environment that means it’s succeeded.
I think it was an acceptable level of accuracy. It’s very hard to write short things, but does EA roughly hold that all people are equal? Yes I think that’s not a bad 4 word summary. I think a better summary is “the value of beings doesn’t change based on their position in space or time and I reject the many heuristics humanity has used to narrow concern which have led to the suffering we see today—racism, sexism, speciesism, etc”. I think that while more precise that phrase isn’t that much more accurate and is worse in other ways—it’s just hard to understand.
It condemned the language Bostrom used without saying he was wrong to hold views. To me this is clearly more truth-seeking than practically any other organisation would have published.
I think we should judge CEA and by their track records, not one statement. I think they are a truth-seeking, effective organisation and if not, I don’t think it’s this statement that has changed that for me
I think the statement suggests that CEA doesn’t have a tolerance for language or flippancy like this, but doesn’t suggest that they are gonna sever ties with Bostrom. That seems like the right call.
I think people can dislike it. I really want people to feel free—but currently I don’t.
Conflict of interest: I like Shakeel and feel loyalty to him which probably distorts this.
I agree the we ignore experts over people who are more value aligned. Seems like a mistake.
Personal feelings (which I don’t imply are true or actionable)
I am annoyed and sad.
I want to feel like I can trust the leaders of this community are playing by a set of agreed rules. Eg I want to hear from them. And half of me trusts them and half feels I should take an outside view that leaders often seek to protect their own power. The disagreement between these parts causes hurt and frustration.
I also variously feel hurt, sad, afraid, compromised, betrayed.
I feel ugly that I talk so much about my feelings too. It feels kind of obscene.
I feel sad that saying negative things, especially about Will. I sense he’s worked really hard. I feel ungrateful and snide. Yuck.
Object level
I don’t think this article moves me much This article moves me a bit on a number of important things:
We have some more colour around the specific warnings that were given
It becomes much more likely that MacAskill backed Bankman-Fried in the aftermath of the the early Alameda disagreements which was ex-ante, dubious and ex-post disasterous. The comment about threatening Mac Auley is very concerning.
I update a bit that Sam used this support as cover
I sense that people ought to take the accusations of inappropriate sexual relationships more seriously to be consistent, though I personally I am uncertain cos we don’t have much information
edit mainly after talking to Naia in the comments, I update towards being uncertain about whether we knew SBF was unusually badly behaved (from being confident he wasn’t). ie maybe we did have the information required to be pretty sure he wasn’t worth funding or to keep him at arms length. As I say I am uncertain but previously I dismissed this
The 80k interview feels even worse researched/too soft than I previously thought
I still sense that core EAs take this seriously
I still think they don’t think they can talk
I still don’t understand why they can’t give a clear promise of when they will talk and that the lack of this makes me trust them less
I think we had lots of info that sam was a bit dodgy before the FTX crash, but that this was not above normal the levels of many CEOs of rapidly growing business (I have read most about Google’s early days and it was very shifty)
Perhaps EA should have higher standards, but I sense not.
I still think that we should have been much more careful linking ourselves reputationally to FTX
I think the big thing here to note is that even those who saw sam at his worst did not expect the FTX crash, so I guess the question is “should sam have been lauded, given his early behaviour at alameda”. I think no, but given what we knew I am uncertain whether he should have been condemned not condemned either
I think not talking while there is in investigation is reasonable
I have made both criticisms and defences of MacAskill and Beckstead and stand by them
I still think they are both very talented, perhaps more so as a result of the growth and wisdom this will engender in them (I have often thought it was dumb that people remove leaders who make mistakes) edit Though this article does add additional concerns
I would still like an argument that they shouldn’t be removed from boards, when almost any other org would. I would like the argument made and seen to be made.
I have noticed how hard it is to talk publicly about these things. Recently I’ve updated more in favour that there just are emotional, social and some career costs (and benefits) to trying to have accurate semi-public discussions about these things. People DM me. I hear people are annoyed, I build both social credit and debt. I think less than some say, but more than none.
I cannot deny that I am tempted to mediate my comments so that people will like me and probably do a bit
Missing context
I have a reasonable amount of time for the notion that EA leaders should have an independent investigation and I don’t think the article gives that enough credit
Many business leaders are disagreeable people who do grey things. Uber’s activities were deliberately illegal in many countries and I probably on balance support that. edit I am less in agreement with my tone here. The article mentions this, but in my opinion it should be written in big letters in the top that:
None of the early Alameda employees who witnessed Bankman-Fried’s behavior years earlier say they anticipated this level of alleged criminal fraud. There was no “smoking gun,” as one put it, that revealed specific examples of lawbreaking. Even if they knew Bankman-Fried was dishonest and unethical, they say, none of them could have foreseen a fraud of this scope.
If even they didn’t think this, I don’t think we should be surprised that core EAs didn’t either.
Relevant things people may or may not have known:
In the early days of Alameda, SBF reneged on deals with other EAs and had very poor financial management. Many core EAs knew this
Other:
My gut says that Naia Bouscal is telling the truth, since before I knew her in relation to this, I thought she was a pretty straight shooting twitter account.
Edited to combine two comments (one personal one more general) into one and add points as I think of them.
How do you feel?
What do you think?
Thanks for writing this, Will. I appreciate the honesty and ambition. Thank you for all you do and I hope you have people around you who love and support you.
I like the framing of judicious ambition. My key question around this and the related longtermism discussion is something like, What is the EA community for?
A democractic funding body?
A talent pool?
Community support?
Error checkers?
Are we the democratic body that makes funding decisions? No and I don’t want us to be. Doing the most good likely involves decisions that the median EA will disagree with. I would like to trial forecasting funding outcomes and voting systems, but I don’t assume that EA should be democratic. The question is what actually does the most good.
Are we a body of talented professionals who work on lower wages than they otherwise would? Yes, but I think we are more than that. Fundamentally it’s our work that is undervalued, rather than us. Animals, the global poor and future generations cannot pay to save their own lives, so we won’t be properly remunerated, except by the joy we take from doing it.
Are we community support for one another? Yes, and I think in regard to this dramatic shift in EA’s fortunes that’s particularly important. It is weird to move from scarcity to abundance. I wasn’t around in the EArly days, but I was working on cash-strapped community projects and it is good to take that discussion seriously.
Are we error checkers for community decisions? Maayybee. For me, this is one where we could do better. Currently, you either get sent the important google docs or you don’t. While I don’t think the community should make decisions, there is a lot of value in allowing them to look over them. But largely I don’t think the technology required to synthesise and correct huge amounts of information as an informal community exists yet. But given the potential value, it is worth looking into.
What is the EA community for? And what infrastructure is worth building in order to ensure that function as we scale as this post describes?
This list should have karma hidden and entries randomised. I guess most poeple do not read and vote all the way to the bottom. I certainly didn’t the first time I read it.
Fwiw I have little private information but think that:
I sense this misses some huge successes in EA getting where it is. Seems we’ve done pretty well all things considered. Wasn’t will part of that?
Will is a superlative networker
He is a very good public intellectual. Perhaps Ord could be if his books were backed to that extent. Perhaps Will could be better if he wrote different books. But he seems really good at it. I would guess that on that public intellectual side he’s a benefit not a cost
If I’d had the ability to direct billions in philanthropy I probably would have, even with nagging doubts.
It seems he’s maybe less good at representing the community or managing orgs. I don’t know if thats the case, but I can believe it.
If so, it seems possible there is a role as a public intellectual associated with EA but who isn’t the only one
I feel bad when writing criticism because personally I hope he’s well and I’m very grateful to him.
Also thanks Habryka for writing this. I think surfacing info like this is really valuable and I guess it has personal costs to you.
An attempt to express the “this is really bad” position.
These are not my views, but an attempt to describe others.
Imagine I am a person who occasionally experiences racism or who has friends or family for whom that is the case. I want a community I and my friends feel safe in. I want one that shares my values and acts predictably (dare I say, well-aligned). Not one that never challenges me, but one where lines aren’t crossed and if they are, I am safe. Perhaps where people will push back against awful behaviour so I don’t have to feel constantly on guard.
Bostrom’s email was bad and his apology:
Was focused on himself
Decided to take a long detour into the very subjects that make me feel unsafe in the first place
And to add to that, rather than the community saying “yes that was bad” a top response is “I stand with Bostrom”. I understand that people might say “trust us, you know we are good and not racist” but maybe I don’t trust them. Or maybe my friends or family are asking me about if I know this Bostrom guy or if he’s part of my community.
And maybe I am worried that Bostrom et al don’t have the interests of people of colour at heart when they think about the far future. Perhaps, like the first female astronaut, who was asked if she needed 100 of tampons for the week, I am concerned that the far future is not being built with anything like an understanding of the needs of me or people of colour.
Heck, perhaps Bostrom wouldn’t hire someone like me. His apology doesn’t inspire confidence. Whenever I meet someone who holds these views it’s only an amount of time before I see those views worked out in their actions. This might seem overblown, but it plays on my mind.
At that point, the endless focus on the object level might make me feel like this isn’t the community for me. I’m not trying to bully you into submission, but I’m signalling that if you don’t take a moment I might leave. Or people like me won’t join.
More than this these are just good values I want in a community I want to be part of. I want a community with empathy. And I want a community with a good reputation.
Imagine we’re both taking an uber and the driver takes a turn too fast. “Slow down” I shout. You say “it’s fine, it’s fine”. I say “Look, can we tell the driver pull into a side road and stop for a bit”. The question isn’t whether they took the corner too fast, it’s about whether you support me when I don’t feel safe. And if you don’t, I’m gonna get out of the car.
This was written after conversations with a few friends. I’m trying to do a good job but it’s probably flawed, nor do I claim to be a good spokesperson here, I’m not. I think some people are seeing this as an intellectual discussion while others are trying to feel safe and comfortable. If they don’t, those people might leave. While I might not mediate my tone in discourse for an adversary, I often do for my friends.
Some thoughts:
Vibewise, I find this unsurprising. I am not utterly shocked by any of the above and am not particularly surprised by much of it. Nonlinear does have a move fast and break things approach and I imagine they have patterns of behaviour that have predictably hurt people in the past. As evidence of this I made a market on it about it 8 months ago.
I like the nonlinear team personally and guess they do good/interesting work. I thought superlinear was a good test-case of bounty-based funding. I also use the Non-Linear library and find it valuable. I am confident that Kat, Emerson and Drew have good intentions on a deep level.
The above statements can both be true.
Thanks Ben for doing this, I think it was brave and good
I particularly like this advice “Going forward I think anyone who works with Kat Woods, Emerson Spartz, or Drew Spartz, should sign legal employment contracts, and make sure all financial agreements are written down in emails and messages that the employee has possession of. I think all people considering employment by the above people at any non-profits they run should take salaries where money is wired to their bank accounts, and not do unpaid work or work that is compensated by ways that don’t primarily include a salary being wired to their bank accounts.”
I think that EA is a very high trust ecosystem and I guess maybe Nonlinear shouldn’t be given that trust. But after reading the above, it’s up to you. I might advise the median EA not to work for them and I’d advise them not to hire people unless they are pretty hard-nosed though seemingly Kat has said she would change things anyway.
I am pretty curious about @spencerg’s statement that many simple errors were wrong with a recent draft this. That seems notable
I think the key question here is “Will such events happen again and what is the acceptable threshold for that chance”
If Nonlinear were welcomed unambiguously back into the EA fold I don’t think I’d be that confident that there wouldn’t be more stories like this in the next year. Maybe 20% that there would be.
I guess most people think that is above the acceptable tolerance.
I think it’s above tolerance for me too. It seems much higher than any other org I can think of. Notably it seems very avoidable.
I guess I do think things trade off against one another and maybe I’d like a way for us to say “this org is really effective but we don’t recommend most people work for them”. This is the sort of stance many have towards non-safety AI work as a means to upskill
I think rather than this being seen as punishment it can be seen as acceptable boundary setting—communities are allowed to want people given status and others not. This action will lower Nonlinear’s status and as a group we can choose to do that. Generally I think about things in terms of bad/unacceptable behaviour rather than bad people and I think a community gets to set the level of predictable bad behaviour it is exposed to.
What could Nonlinear do to convince me that they deserve the same levels of easy trust as other EA orgs? [1]
Provide evidence that this article is deeply flawed
Not threaten to sue Lightcone for publishing it
Acknowledge the patterns of behaviour that led to these outcomes and how they have changed—this would probably involve acknowledging that lots of these events took place
They could just decide that they don’t want this—they want to work in a different way to how EA orgs tend to, and that’s fine-adjacent, but then I would recommend they be treated differently too. I wonder sometimes whether it’s good to have, say, rationalism as a space willing to deal with less standard norms.
In my opinion too much post-FTX discussion has focused on individual EA behaviour and too little on powerful EA behavior. I think the median EA trying to avoid downside risk is overrated. As one’s influence grows the harm one does scales and it becomes better value to try to limit the bad and increase the good more. I am much more interested in a discussion of what Nonlinear should or shouldn’t do than for Catherine Richardson from Putney to worry if she she is spending too much money on rent.
Note that again, I don’t think they should close down, I’m just not sure they should be invited to present at EAGs, and I’d be happy for this to sit on a forum wiki page about them
I respect you a lot for doing something costly and weird because you thought it was the right thing to do, especially since it cuts against the longtermist norms of “more”.
No idea if you are right though.
Several people, including you, told me not to post this initially. To me that felt like “we should not do thing that damage FTX’s chances in an involving situation”. I’m not accusing, just confused—it feels hard to square that with this reply.
I assume this is redundant but I might as well check. Have you considered applying for FTX for funding to run the project which is modelled after your project already? Seems kind of a no brainer to avoid replication and use your tech/branding to deliver this, though I’m sure there are things I don’t understand.
As someone who is prone to think in the abstract I’m probably gonna leave it a week before trying to have a standard discussion of this.
That’s not to say there can’t be on depth discussion, but I sense a lot of hurt here and it feels like it might be better to wait.
What’s your response to this accusation, in Time? This behaviour doesn’t sound like you but Naia outright lying would surprise me from my interactions with her.
Bouscal recalled speaking to Mac Aulay immediately after one of Mac Aulay’s conversations with MacAskill in late 2018. “Will basically took Sam’s side,” said Bouscal, who recalls waiting with Mac Aulay in the Stockholm airport while she was on the phone. (Bouscal and Mac Aulay had once dated; though no longer romantically involved, they remain close friends.) “Will basically threatened Tara,” Bouscal recalls. “I remember my impression being that Will was taking a pretty hostile stance here and that he was just believing Sam’s side of the story, which made no sense to me.”
“He was treating it like a ‘he said-she said,’ even though every other long-time EA involved had left because of the same concerns,” Bouscal adds.
I feel sad/judged/ frustrated reading this, though that’s not to say it’s wrong.
Some thoughts
I flirt much less within EA than I once did, largely for these reasons
Yeah, on priors this seems like a normal wise thing to do
I guess there are some possible mid way norms like “women should initiate”
I sense that that women’s views on this are more complex than this suggests, after all many women choose to sleep with people like this. To deny them this option is probably a cost
It feels like we might be able to do better than this
I sense, like many things that the people who most need to hear this might ignore it and those that need to hear it least will enact it
I note that this prescription will affect neurodivergent people more (social clumsiness is much more common among such people). I’m okay with this, but are you?
Feels like we should, as everywhere else, try to quantify the harms and make a decision based on that
Businesses often try to improve only one thing, not everything at once. While I think EA can build better relationships paradigms it feels important that we are trying to make the world better not design a way to have more sex in communities.
adding things if I think of them
Suggestion of concensus opinion:
People should not be dating or living with those who work for them. If they start, they should tell HR so they can be transferred.
This piece is pretty long and I didn’t think I’d like it, but I put that aside. I think it’s pretty good with many suggestions I agree with.
Thanks for writing it. I guess it wasn’t easy, I know it’s hard to wrestle with communities you both love and disagree with. Thanks for taking the time and energy to write this.
I think it’s wise of you to do this while you can handover well. Thanks for the bravery of being able to say it’s too hard. And I’m glad you are looking after yourself. Thank you for all your work. I’m sad this has been so hard for you.
I love the concrete advice in this piece. Thanks for taking the time to write it.
I sense there is a lurking disagreement about whether what Owen did was clumsy or really bad and I guess that will make a lot of this pretty hard to discuss.
I haven’t thought about it much but removing people from boards after a massive miscalculation seems reasonable.
Like our prior should be to replace at least Nick and Will right?