RobBensinger

• RobBensingerApr 3, 2024, 6:20 PM

  5 points

  3 ∶ 1

  in reply to: David Thorstad’s comment on: Quick Up­date on Leav­ing the Board of EV

  I’m a pretty big fan of Nate’s public write-up on his relationship to Sam and FTX. Though, sure, this is going to be scarier for people who were way more involved and who did stuff that twitter mobs can more easily get mad about.

  This is part of why the main thing I’m asking for is a professional investigation, not a tell-all blog post by every person involved in this mess (though the latter are great too). An investigation can discover useful facts and share them privately, and its public write-up can accurately convey the broad strokes of what happened, and a large number of the details, while taking basic steps to protect the innocent.

  What links here?

  • We need an in­de­pen­dent in­ves­ti­ga­tion into how EA lead­er­ship has han­dled SBF and FTX by AnonymousEAForumAccount (Jul 24, 2024, 3:24 AM; 141 points)

• RobBensingerApr 3, 2024, 3:48 AM

  132 points

  44 ∶ 8

  on: Quick Up­date on Leav­ing the Board of EV

  Here’s a post with me asking the question flat out: Why hasn’t EA done an SBF investigation and postmortem?

  This seems like an incredibly obvious first step from my perspective, not something I’d have expected a community like EA to be dragging its heels on years after the fact.

  We’re happy to sink hundreds of hours into fun “criticism of EA” contests, but when the biggest disaster in EA’s history manifests, we aren’t willing to pay even one investigator to review what happened so we can get the facts straight, begin to rebuild trust, and see if there’s anything we should change in response? I feel like I’m in crazytown; what the heck is going on?

• RobBensingerApr 3, 2024, 3:40 AM

  34 points

  8 ∶ 2

  on: Why hasn’t EA done an SBF in­ves­ti­ga­tion and post­mortem?

  ⁴⁄₂ Update: A former board member of Effective Ventures US, Rebecca Kagan, has shared that she resigned from the board in protest last year, evidently in part because of various core EAs’ resistance to there being any investigation into what happened with Sam Bankman-Fried. “These mistakes make me very concerned about the amount of harm EA might do in the future.”

  Oliver Habryka says that I’m correct that EA still hasn’t yet conducted any sort of investigation about what happened re SBF/​FTX, beyond the narrow investigation into whether EV was facing legal risk:

  I am quite confident there is no such investigation. I have been trying to convince people to do such a thing for like 1.5 years, but appetite for it seems low among leadership.

  I might even offer for Lightcone to do it, if we could get EA orgs to collaborate.

  (I’ve had conversations with Kagan, Habryka, and a bunch of other EAs about this in the past, and I knew Kagan’s post was in the pipeline, though the stuff they’ve said on this topic has been independent of me and would have been written in my absence.)

  Someone else messaged me to say that they thought there had been an investigation, but after talking to staff at CEA, they’ve confirmed again that no investigation has ever taken place.

  Julia Wise and Ozzie Gooen apparently also called for an investigation, a full five months ago:

  • Action:

    • Find someone to run an investigation into how EA individuals and organizations could have better handled the FTX situation.

  • Barriers:

    • People who did things that were bad or will make them look bad will not want to tell you about it. Everyone’s lawyers will have told them not to talk about anything.

  • Existing work /​ resources:

    • EV’s investigation has a defined scope that won’t be relevant to all the things EAs want to know, and it won’t necessarily publish any of its results.

    • Oliver Habryka and friends have done a discussion series informally looking at what happened and how trust within EA related to the problem.

  [Update Apr. 5: Julia tells me “I would say I listed it as a possible project rather than calling for it exactly.”]

  So, bizarre as it seems, the situation does seem to be as it appears: there’s been literally no action on this in the ~17 months since FTX imploded.

  Will MacAskill’s appearance on Sam Harris podcast is out now, and I’m very happy to hear the new details from Will about what happened from his perspective.

  But while he talks (at 56:00) about a lot of individual EAs undergoing an enormous amount of “self-reflection, self-scrutiny” in the wake of FTX’s collapse, and he notes that a lot of EA’s leadership has been replaced with new leadership, I don’t consider this a replacement for the whole (obvious, from my perspective) “actually pay someone to look into what happened and write up a postmortem” thing. If anything I’d have expected EAs to have multiple such write-ups by now so we could compare different perspectives on what happened, not literally zero.

  What links here?

  • Why hasn’t EA done an SBF in­ves­ti­ga­tion and post­mortem? by RobBensinger (Apr 1, 2024, 2:28 AM; 166 points)

• RobBensingerApr 1, 2024, 5:27 PM

  17 points

  8 ∶ 1

  in reply to: Manuel Allgaier’s comment on: Why hasn’t EA done an SBF in­ves­ti­ga­tion and post­mortem?

  I’d add that I think 80K has done an awful lot to communicate “EA isn’t just about earning-to-give” over the years. At some point it surely has to be the case that they’ve done enough. This is part of why I want to distinguish the question “did we play a causal role here?” from questions like “did we foreseeably screw up?” and “should we do things differently going forward?”.

• RobBensingerApr 1, 2024, 6:56 AM

  19 points

  6 ∶ 1

  in reply to: huw’s comment on: Why hasn’t EA done an SBF in­ves­ti­ga­tion and post­mortem?

  Yes, there are many indirect ways EA might have had a causal impact here, including by influencing SBF’s ideology, funneling certain kinds of people to FTX, improving SBF’s reputation with funders, etc. Not all of these should necessarily cause EAs to hand-wring or soul-search — sometimes you can do all the right things and still contribute to a rare disaster by sheer chance. But a disaster like this is a good opportunity to double-check whether we’re living up to our own principles in practice, and also to double-check whether our principles and strategies are as beneficial as they sounded on paper.

Why hasn’t EA done an SBF in­ves­ti­ga­tion and post­mortem?

• RobBensingerMar 31, 2024, 9:32 PM

  13 points

  5 ∶ 1

  in reply to: Habryka [Deactivated]’s comment on: Re­cent and up­com­ing me­dia re­lated to EA

  It directly claims that the investigation was part of an “internal reflection process” and “instutitional reform”, and I have been shared on documents by CEA employees where the legal investigation was explicitly called out as not being helpful for facilitating a reflection process and institutional reform.

  This seems like the headline claim to me. EAs should not be claiming false things in the Washington Post, of all things.

• RobBensingerJan 6, 2024, 4:33 AM

  16 points

  4 ∶ 2

  in reply to: Evan_Gaensbauer’s comment on: MIRI 2024 Mis­sion and Strat­egy Update

  Every aspect of that summary of how MIRI’s strategy has shifted seems misleading or inaccurate to me.

AI Views Snapshots

• RobBensingerSep 20, 2023, 6:15 PM

  5 points

  2 ∶ 1

  in reply to: Davidmanheim’s comment on: Com­ments on Man­heim’s “What’s in a Pause?”

  I find myself agreeing with Nora on temporary pauses—and I don’t really understand the model by which a 6-month, or a 2-year, pause helps, unless you think we’re less than 6 months, or 2-years, from doom.

  This doesn’t make a lot of sense to me. If we’re 3 years away from doom, I should oppose a 2-year pause because of the risk that (a) it might not work and (b) it will make progress more discontinuous?

  In real life, if smarter-than-human AI is coming that soon then we’re almost certainly dead. More discontinuity implies more alignment difficulty, but on three-year timelines we have no prospect of figuring out alignment either way; realistically, it doesn’t matter whether the curve we’re looking at is continuous vs. discontinuous when the absolute amount of calendar time to solve all of alignment for superhuman AI systems is 3 years, starting from today.

  I don’t think “figure out how to get a god to do exactly what you want, using the standard current ML toolbox, under extreme time pressure” is a workable path to humanity surviving the transition to AGI. “Governments instituting and enforcing a global multi-decade pause, giving us time to get our ducks in order” does strike me as a workable path to surviving, and it seems fine to marginally increase the intractability of unworkable plans in exchange for marginally increasing the viability of plans that might work.

  If a “2-year” pause really only buys you six months, then that’s still six months more time to try to get governments to step in.

  If a “2-year” pause buys you zero time in expectation, and doesn’t help establish precedents like “now that at least one pause has occurred, more ambitious pauses are in the Overton window and have some chance of occurring”, then sure, 2-year moratoriums are useless; but I don’t buy that at all.

Com­ments on Man­heim’s “What’s in a Pause?”

• RobBensingerSep 13, 2023, 8:07 PM

  3 points

  3 ∶ 0

  in reply to: RobBensinger’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  Daniel Wyrzykowski replies:

  The contract is signed for when bad things and disagreements happen, not for when everything is going good. In my opinion “I had no contract and everything was good” is not as good example as “we didn’t have a contract, had major disagreement, and everything still worked out” would be.
  
  Even though I hate bureaucracy and admin work and I prefer to skip as much as reasonable to move faster, my default is to have a written agreement, especially if working with a given person/​org for the first time. Generally, the weaker party should have the final say on forgoing a contract. This is especially true the more complex and difficult situation is (eg. living/​travelling together, being in romantic relationships).
  
  I agree with the general view that both signing and not signing have prons and cons and sometimes it’s better to not sign and avoid the overhead.

• RobBensingerSep 13, 2023, 8:06 PM

  11 points

  2 ∶ 0

  in reply to: RobBensinger’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  Elizabeth van Nostrand replies:

  I feel like people are talking about written records like it’s a huge headache, but they don’t need to be. When freelancing I often negotiate verbally, then write an email with terms to the client., who can confirm or correct them. I don’t start work until they’ve confirmed acceptance of some set of terms. This has enough legal significance that it lowers my business insurance rates, and takes seconds if people are genuinely on the same page.

  What my lawyer parent taught me was that contracts can’t prevent people from screwing you over. (which is impossible). At my scale and probably most cases described here, the purpose of a contract is to prevent misunderstandings between people of goodwill. And it’s so easy to do notably better than nonlinear did here.

  Linda responds:

  This is a good point. I was thinking in terms of legal vs informal, not in terms of written vs verbal.
  
  I agree that having something written down is basically always better. Both for clarity, as you say, and because peoples memories are not perfect. And it have the added bonus that if there is a conflict, you have something to refer back to.

• RobBensingerSep 13, 2023, 8:05 PM

  18 points

  1 ∶ 0

  in reply to: RobBensinger’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  Duncan Sabien replies:

  [...]

  While I think Linda’s experience is valid, and probably more representative than mine, I want to balance it by pointing out that I deeply, deeply, deeply regret taking a(n explicit, unambiguous, crystal clear) verbal agreement, and not having a signed contract, with an org pretty central to the EA and rationality communities. As a result of having the-kind-of-trust that Linda describes above, I got overtly fucked over to the tune of many thousands of dollars and many months of misery and confusion and alienation, and all of that would’ve been prevented by a simple written paragraph with two signatures at the bottom.

  (Such a paragraph would’ve either prevented the agreement from being violated in the first place, or would at least have made the straightforward violation that occurred less of a thing that people could subsequently spin webs of fog and narrativemancy around, to my detriment.)

  As for the bit about telling your friends and ruining the reputation of the wrongdoer … this option was largely NOT available to me, for fear-of-reprisal reasons as well as not wanting to fuck up the subsequent situation I found myself in, which was better, but fragile. To this day, I still do not feel like it’s safe to just be fully open and candid about the way I was treated, and how many norms of good conduct and fair dealings were broken in the process. The situation was eventually resolved to my satisfaction, but there were years of suffering in between.

  [...]

• RobBensingerSep 13, 2023, 8:02 PM

  9 points

  3 ∶ 3

  on: Shar­ing In­for­ma­tion About Nonlinear

  Cross-posting Linda Linsefors’ take from LessWrong:

  I have worked without legal contracts for people in EA I trust, and it has worked well.
  
  Even if all the accusation of Nonlinear is true, I still have pretty high trust for people in EA or LW circles, such that I would probably agree to work with no formal contract again.
  
  The reason I trust people in my ingroup is that if either of us screw over the other person, I expect the victim to tell their friends, which would ruin the reputation of the wrongdoer. For this reason both people have strong incentive to act in good faith. On top of that I’m wiling to take some risk to skip the paper work.

  When I was a teenager I worked a bit under legally very sketch circumstances. They would send me to work in some warehouse for a few days, and draw up the contract for that work afterwards. Including me falsifying the date for my signature. This is not something I would have agreed to with a stranger, but the owner of my company was a friend of my parents, and I trusted my parents to slander them appropriately if they screwed me over.

  I think my point is that this is not something very uncommon, because doing everything by the book is so much overhead, and sometimes not worth it.
  
  It think being able to leverage reputation based and/​or ingroup based trust is immensely powerful, and not something we should give up on.
  
  For this reason, I think the most serious sin committed by Nonlinear, is their alleged attempt of silencing critics.
  Update to clarify: This is based on the fact that people have been scared of criticising Nonlinear. Not based on any specific wording of any specific message.
  Update: On reflection, I’m not sure if this is the worst part (if all accusations are true). But it’s pretty high on the list.
  
  I don’t think making sure that no EA every give paid work to another EA, with out a formal contract, will help much. The most vulnerable people are those new to the movement, which are exactly the people who will not know what the EA norms are anyway. An abusive org can still recruit people with out contracts and just tell them this is normal.
  
  I think a better defence mechanism is to track who is trust worthy or not, by making sure information like this comes out. And it’s not like having a formal contract prevents all kinds of abuse.
  
  Update based on responses to this comment: I do think having a written agreement, even just an informal expression of intentions, is almost always strictly superior to not having anything written down. When writing this I comment I was thinking in terms of formal contract vs informal agreement, which is not the same as verbal vs written.

• RobBensingerSep 12, 2023, 10:23 PM

  9 points

  2 ∶ 2

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  I’d be happy to talk with you way more about rationalists’ integrity fastidiousness, since (a) I’d expect this to feel less scary if you have a clearer picture of rats’ norms, and (b) talking about it would give you a chance to talk me out of those norms (which I’d then want to try to transmit to the other rats), and (c) if you ended up liking some of the norms then that might address the problem from the other direction.

  In your previous comment you said “it doesn’t seem obviously unethical to me for Nonlinear to try to protect its reputation”, “That’s a huge rationalist no-no, to try to protect a narrative”, and “or to try to affect what another person says about you”. But none of those three things are actually rat norms AFAIK, so it’s possible you’re missing some model that would at least help it feel more predictable what rats will get mad about, even if you still disagree with their priorities.

  Also, I’m opposed to cancel culture (as I understand the term). As far as I’m concerned, the worst person in the world deserves friends and happiness, and I’d consider it really creepy if someone said “you’re an EA, so you should stop being friends with Emerson and Kat, never invite them to parties you host or discussion groups you run, etc.” It should be possible to warn people about bad behavior without that level of overreach into people’s personal lives.

  (I expect others to disagree with me about some of this, so I don’t want “I’d consider it really creepy if someone did X” to shut down discussion here; feel free to argue to the contrary if you disagree! But I’m guessing that a lot of what’s scary here is the cancel-culture /​ horns-effect /​ scapegoating social dynamic, rather than the specifics of “which thing can I get attacked for?”. So I wanted to speak to the general dynamic.)

• RobBensingerSep 12, 2023, 10:05 PM

  83 points

  29 ∶ 3

  in reply to: chloe’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  Emerson approaches me to ask if I can set up the trip. I tell him I really need the vacation day for myself. He says something like “but organizing stuff is fun for you!”.

  [...]

  She kept insisting that I’m saying that because I’m being silly and worry too much and that buying weed is really easy, everybody does it.

  😬 There’s a ton of awful stuff here, but these two parts really jumped out at me. Trying to push past someone’s boundaries by imposing a narrative about the type of person they are (‘but you’re the type of person who loves doing X!’ ‘you’re only saying no because you’re the type of person who worries too much’) is really unsettling behavior.

  I’ll flag that this is an old remembered anecdote, and those can be unreliable, and I haven’t heard Emerson or Kat’s version of events. But it updates me, because Chloe seems like a pretty good source and this puzzle piece seems congruent with the other puzzle pieces.

  E.g., the vibe here matches something that creeped me out a lot about Kat’s text message to Alice in the OP, which is the apparent attempt to corner/​railroad Alice into agreement via a bunch of threats and strongly imposed frames, followed immediately by Kat repeatedly stating as fact that Alice will of course agree with Kat: “[we] expect you will do the same moving forward”, “Sounds like you’ve come to the same conclusion”, “It sounds like we’re now on the same page about this”.

  

  Working and living with Nonlinear had me forget who I was, and lose more self worth than I had ever lost in my life. I wasn’t able to read books anymore, nor keep my focus in meetings for longer than 2 minutes, I couldn’t process my own thoughts or anything that took more than a few minutes of paying attention.

  😢 Jesus.

• RobBensingerSep 11, 2023, 11:07 PM

  51 points

  17 ∶ 1

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  Maybe I’m wrong— I really don’t know, and there have been a lot of “I don’t know” kind of incidents around Nonlinear, which does give me pause— but it doesn’t seem obviously unethical to me for Nonlinear to try to protect its reputation.

  I think it’s totally normal and reasonable to care about your reputation, and there are tons of actions someone could take for reputational reasons (e.g., “I’ll wash the dishes so my roommate doesn’t think I’m a slob”, or “I’ll tweet about my latest paper because I’m proud of it and I want people to see what I accomplished”) that are just straightforwardly great.

  I don’t think caring about your reputation is an inherently bad or corrupting thing. It can tempt you to do bad things, but lots of healthy and normal goals pose temptation risks (e.g., “I like food, so I’ll overeat” or “I like good TV shows, so I’ll stay up too late binging this one”); you can resist the temptation without stigmatizing the underlying human value.

  In this case, I think the bad behavior by Nonlinear also would have been bad if it had nothing to do with “Nonlinear wants to protect its reputation”.

  Like, suppose Alice honestly believed that malaria nets are useless for preventing malaria, and Alice was going around Berkeley spreading this (false) information. Kat sends Alice a text message saying, in effect, “I have lots of power over you, and dirt I could share to destroy you if you go against me. I demand that you stop telling others your beliefs about malaria nets, or I’ll leak true information that causes you great harm.”

  On the face of it, this is more justifiable than “threatening Alice in order to protect my org’s reputation”. Hypothetical-Kat would be fighting for what’s true, on a topic of broad interest where she doesn’t stand to personally benefit. Yet I claim this would be a terrible text message to send, and a community where this was normalized would be enormously more toxic than the actual EA community is today.

  Likewise, suppose Ben was planning to write a terrible, poorly-researched blog post called Malaria Nets Are Useless for Preventing Malaria. Out of pure altruistic compassion for the victims of malaria, and a concern for EA’s epistemics and understanding of reality, Hypothetical-Emerson digs up a law that superficially sounds like it forbids Ben writing the post, and he sends Ben an email threatening to take Ben to court and financially ruin him if he releases the post.

  (We can further suppose that Hypothetical-Emerson lies in the email ‘this is a totally open-and-shut case, if this went to trial you would definitely lose’, in a further attempt to intimidate and pressure Ben. Because I’m pretty danged sure that’s what happened in real life; I would be amazed if Actual-Emerson actually believes the things he said about this being an open-and-shut libel case. I’m usually reluctant to accuse people of lying, but that just seems to be what happened here?)

  Again, I’d say that this Hypothetical-Emerson (in spite of the “purer” motives) would be doing something thoroughly unethical by sending such an email, and a community where people routinely responded to good-faith factual disagreements with threatening emails, frivolous lawsuits, and lies, would be vastly more toxic and broken than the actual EA community is today.

• RobBensingerSep 9, 2023, 11:26 AM

  9 points

  3 ∶ 0

  in reply to: Habryka [Deactivated]’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  I’m not sure I’ve imagined a realistic justifying scenario yet, but in my experience it’s very easy to just fail to think of an example even though one exists. (Especially when I’m baking in some assumptions without realizing I’m baking them in.)

• RobBensingerSep 9, 2023, 11:22 AM

  18 points

  7 ∶ 0

  in reply to: Linch’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  I do think the phrase is a bit childish and lacks some rigor

  I think the phrase is imprecise, relative to phrases like “prevent human extinction” or “maximize the probability that the reachable universe ends up colonized by happy flourishing civilizations”. But most of those phrases are long-winded, and it often doesn’t matter in conversation exactly which version of “saving the world” you have in mind.

  (Though it does matter, if you’re working on existential risk, that people know you’re being relatively literal and serious. A lot of people talk about “saving the planet” when the outcome they’re worried about is, e.g., a 10% loss in current biodiversity, rather than the destruction of all future value in the observable universe.)

  If a phrase is useful and tracks reality well, then if it sounds “childish” that’s more a credit to children than a discredit to the phrase.

  And I don’t know what “lacks some rigor” means here, unless it’s referring to the imprecision.

  Mostly, I like “saves the world” because it owns my weird beliefs about the situation I think we’re in, and states it bluntly so others can easily understand my view and push back against it if they disagree.

  Being in a situation where you think your professional network’s actions have a high chance of literally killing every human on the planet in the next 20 years, or of preventing this from happening, is a very unusual and fucked up situation to be in. I could use language that downplays how horrifying and absurd this all is, but that would be deceiving you about what I actually think. I’d rather be open about the belief, so it can actually be talked about.