Building bespoke quantitative models to support decisionmakers in AI and bio. Right now that means: forecasting capabilities gains due to post-training enhancements on top of frontier foundation models, and estimating the annual burden of airborne disease in the US.
Joel Becker
Karma: 1,717
Thank you very much for sharing, Chloe.
Ben, Kat, Emerson, and readers of the original post have all noticed that the nature of Ben’s process leads to selection against positive observations about Nonlinear. I encourage readers to notice that the reverse might also be true. Examples of selection against negative information include:
Ben has reason to exclude stories that are less objective or have a less strong evidence base. The above comment is a concrete example of this.
There’s also something related here about the supposed unreliability of Alice as a source: Ben needs to include this to give a complete picture/because other people (in particular the Nonlinear co-founders) have said this. I strongly concur with Ben when he writes that he “found Alice very willing and ready to share primary sources [...] so I don’t believe her to be acting in bad faith.” Personally, my impression is that people are making an incorrect inference about Alice from her characteristics (that are perhaps correlated with source-reliability in a large population, but aren’t logically related, and aren’t relevant in this case).
To the extent that you expect other people to have been silenced (e.g. via anticipated retaliation), you might expect not to hear relevant information from them.
To the extent that you expect Alice and Chloe to have had burnout-style experiences, you might expect not to read clarifications on or news about negative experiences.
Until this post came out, this was true of ~everything in the post.
There is a reason the post was published 1.5 years after the relevant events took place—people involved in the events really do not want to spend further mental effort on this.
Thank you very much for posting!
I think that you should add an edit removing Drew’s name, for this reason if nothing else. (Happy to expand.) Thank you.
Repost from LW:
My understanding (definitely fallible, but I’ve been quite engaged in this case, and am one of the people Ben interviewed) has been that Alice and Chloe are not concerned about this, and in fact that they both wish to insulate Drew from any negative consequences. This seems to me like an informative and important consideration. (It also gives me reason to think that the benefits of gaining more information about this are less likely to be worth the costs.)
Here’s another thing.
That’s a red line in my book, and I will not personally work with Nonlinear in the future because of it, and I recommend their exclusion from any professional communities that wish to keep up the standard of people not being silenced about extremely negative work experiences.
Let’s suppose that Nonlinear have crossed red lines, and that additional information from them won’t change this. (In reality I think that this is up in the air for the next week or so; I won’t allow my limited imagination to diminish the hope.)
Do you not believe in the possibility of rehabilitation in this case?
I haven’t read up on what norms here work well in other high-trust communities. But at least in criminal vs. society settings I would want to be a strong proponent of rehabilitation. It seems pretty plausible to me that, after thinking more about best norms in high-trust communities, I could come to think that “create horrendous work environment” and “create credible fear of severe retaliation” were things that could change (and be monitored) upon rehabilitation, and that it would be good for this to happen after X period of time.
Thank you Ben—please check comment mentions of Drew, too!
Yes, I think that the post does not do enough to make it clear that the central allegations are not about Drew Spartz. Happy to expand.
Agreed. I would have wanted the post itself to make this more clear.
I’m unclear on how this comment speaks to the content of the post, which is compatible with Kat being a courageous, frugal, and dedicated friend and leader.
As one of the people Ben interviewed:
This post closely reflects my understanding of the situation. (EDIT: at this time, before engaging with Nonlinear reply myself.)
Whenever this post touches on something that I can independently corroborate (EDIT: small minority of claims), I believe it to be accurate. Whenever the post communicates something that both Ben and I have heard from Alice and Chloe (EDIT: large majority of claims), it tells their account faithfully.
I appreciate Ben’s emphasis on red lines and the experiences of Alice and Chloe. When he leaves out stories that I think we are both aware of, my guess is that he has done so because these stories aren’t super relevant to the case at hand or aren’t super objective/strongly evidenced. This makes me think more favourably of the rest of his write-up.
To put it more strongly: I would like to make clear that I have never heard any claims of improper conduct by Drew Spartz (in relation to the events discussed in this post or otherwise).
Kat, I am really sorry about the severe emotional difficulty. It makes sense that having this post be public would be an extremely challenging thing to deal with, all the more so if you have decisive contrary evidence. I will be interested in engaging with whatever you present, once you have the opportunity.
I think it is important to say, as one of the people who Ben interviewed: my very strong impression has been that Ben is interested in the truth, and that he is acting in good faith. My guess is that if you have strong, contrary evidence regarding the most important claims, then Ben will engage with this evidence with an open mind and will signal boost if relevant.
God bless your clear thinking and strong stance-taking, Habryka.
If you can share (publicly or privately) strong evidence contradicting “claims [...] that wildly distort the true story” (emphasis mine), I pre-commit to signal boosting.
For what it’s worth, I wouldn’t be surprised if you do have strong counter-evidence to some claims (given the number of claims made, the ease with which things can be lost in translation, your writing this email, etc.). But, as of right now, I would be surprised if my understanding of the important stuff—roughly, the items covered in Ben’s epistemic state and the crossing of red lines—was wildly distorted. I hope that it is.
[EDIT, Nov 13: it sounds like the Nonlinear reply might be in the 100s of pages. This might be the right move from their point of view, but reading 3-figure pages stretches my pre-commitment above further than I would have intended at the time. I’d like to amend the commitment to “engaging with the >=20 pages-equivalent that seems most relevant to me or Nonlinear, or skimming >=50 pages-equivalent.” If people think this is breaking the spirit of my earlier commitment, I’ll seriously consider standing by the literal wording of that commitment (engaging with ~everything). Feel free to message about this.]
Dan Hendrycks’ lecture on “Safety-Capabilities Balance” might be helpful here.
I’ll speak only for myself—I haven’t spoken to everyone else about this. I think Reslab lost momentum for 3 reasons. In descending order of importance:
Uncertainty about the funding environment. We actually heard only positive things from grantmakers. But it still seemed like there might be a significant risk of not receiving funding conditional on (reasonably) successful outcomes. (Also, less confidently, the time delays in grantmaker responses were significant enough to dampen momentum.) In these conditions, it was hard to convince high-opportunity-cost staff + external stakeholders to go full speed ahead.
Not having someone whose main focus was Reslab. Everyone had non-Reslab projects that were more important to them than Reslab.
Reputational damage from FTX. I have the weak sense that advertising to engineering talent outside the EA ecosystem was going to be more difficult after the collapse of FTX.
No problem, thanks for engaging!
I wrote that most of GPT-4’s suggestions were “obvious or unhelpful.” I would include the ones you pointed to in this box. Pilot funding and tiered grants are presumably things you already do implicitly—e.g. by not committing funding for multiple years in one go, or by not giving huge resources to grantees that you don’t think highly of—and where you wouldn’t benefit much from making this more explicit. And mentorship or guidance seems unhelpful because it’s much too time-costly.
I’m guessing that GPT-4 is trying to point to ‘ways to lower the information asymmetry’ characteristic of adverse selection. All three of these methods give money-cheap ways of gaining more information before making money-expensive decisions.
This post is helpful and appropriately cautious! Thanks Linch.
It feels like adverse selection is a common enough phenomenon that there must be helpful case studies to learn from. I explored this with GPT, and got the following solutions for philanthropic grantmaking:
Third-party assessments: An independent body can evaluate projects or grantees and provide a certification.
Open feedback mechanisms: Existing and past donors can leave reviews or feedback on projects, helping to inform potential future donors.
Tiered grants: Offer different levels of funding based on the risk or novelty of the project. Riskier projects might get smaller, initial amounts with the possibility of more significant funding later if they show promise.
Pilot funding rounds: Similar to probationary periods, fund a project for a short time or with limited funds to assess its viability before committing more.
Collaborative funding: Multiple grantmakers can come together to fund a project, thereby sharing the risk.
Transparency in rejections: While specific details might remain confidential, grantmakers can provide general reasons for rejection, helping to guide potential donors.
Mentorship or guidance: Instead of just providing funds, offer mentorship or guidance to projects, helping them to develop in areas where they might be lacking.
I’m pleased with (2) -- I’ve been putting time into open feedback on Manifund. And (5) is suggestive of something helpful: when it is ok for projects to receive only partial funding and each project applies to the same set of funders, then funders funding only “their part” reduces possible damage without the need to share private information. (Not putting in their part might be helpful information itself.)
Otherwise, these suggestions seem obvious or unhelpful. But I expect that a couple-of-hours dive into how philanthropists or science funders have dealt with these dynamics would be better. Nice project for someone! (@alex lawsen (previously alexrjl)?)
Thank you for posting this beautiful reminder. I’m delighted for Henry’s good news.
Seconding this… I once saw Jeff give a 30 minute talk, completely unprepared, without using a filler word even once. Easy podcast guest to edit!