I recently founded Apollo Research: https://www.apolloresearch.ai/
I was previously doing a Ph.D. in ML at the International Max-Planck research school in Tübingen, worked part-time with Epoch and did independent AI safety research.
For more see https://www.mariushobbhahn.com/aboutme/
I subscribe to Crocker’s Rules
I think I’m sympathetic to the criticism but I still feel like EA has sufficiently high hurdles to stop the grifters.
a) It’s not like you get a lot of money just by saying the right words. You might be able to secure early funds or funds for a local group but at some point, you will have to show results to get more money.
b) EA funding mechanisms are fast but not loose. I think the meme that you can get money for everything now is massively overblown. A lot of people who are EA aligned didn’t get funding from the FTX foundation, OpenPhil or the LTFF. The internal bars for funders still seem to be hard to cross and I expect this to hold for a while.
c) I’m not sure how the grifters would accumulate power and steer the movement off the rails. Either they start as grifters but actually get good results and then rise to power (at that point they might not be grifters anymore) or they don’t get any results and don’t rise to power. Overall, I don’t see a strong mechanism by which the grifters rise to power without either stopping being grifters or blowing their cover. Maybe you could expand on that. I think the company analogy that you are making is less plausible in an EA context because (I believe) people update stronger on negative evidence. It’s not just some random manager position that you’re putting at risk, there are lives at stake. But maybe I’m too naive here.
I personally have no stake in defending Conjecture (In fact, I have some questions about the CoEm agenda) but I do think there are a couple of points that feel misleading or wrong to me in your critique.
1. Confidence (meta point): I do not understand where the confidence with which you write the post (or at least how I read it) comes from. I’ve never worked at Conjecture (and presumably you didn’t either) but even I can see that some of your critique is outdated or feels like a misrepresentation of their work to me (see below). For example, making recommendations such as “freezing the hiring of all junior people” or “alignment people should not join Conjecture” require an extremely high bar of evidence in my opinion. I think it is totally reasonable for people who believe in the CoEm agenda to join Conjecture and while Connor has a personality that might not be a great fit for everyone, I could totally imagine working with him productively. Furthermore, making a claim about how and when to hire usually requires a lot of context and depends on many factors, most of which an outsider probably can’t judge.
Given that you state early on that you are an experienced member of the alignment community and your post suggests that you did rigorous research to back up these claims, I think people will put a lot of weight on this post and it does not feel like you use your power responsibly here.
I can very well imagine a less experienced person who is currently looking for positions in the alignment space to go away from this post thinking “well, I shouldn’t apply to Conjecture then” and that feels unjustified to me.
2. Output so far: My understanding of Conjecture’s research agenda so far was roughly: “They started with Polytopes as a big project and published it eventually. On reflection, they were unhappy with the speed and quality of their work (as stated in their reflection post) and decided to change their research strategy. Every two weeks or so, they started a new research sprint in search of a really promising agenda. Then, they wrote up their results in a preliminary report and continued with another project if their findings weren’t sufficiently promising.” In most of their public posts, they stated, that these are preliminary findings and should be treated with caution, etc. Therefore, I think it’s unfair to say that most of their posts do not meet the bar of a conference publication because that wasn’t the intended goal.
Furthermore, I think it’s actually really good that the alignment field is willing to break academic norms and publish preliminary findings. Usually, this makes it much easier to engage with and criticize work earlier and thus improves overall output quality.
On a meta-level, I think it’s bad to criticize labs that do hits-based research approaches for their early output (I also think this applies to your critique of Redwood) because the entire point is that you don’t find a lot until you hit. These kinds of critiques make it more likely that people follow small incremental research agendas and alignment just becomes academia 2.0. When you make a critique like that, at least acknowledge that hits-based research might be the right approach.
3. Your statements about the VCs seem unjustified to me. How do you know they are not aligned? How do you know they wouldn’t support Conjecture doing mostly safety work? How do you know what the VCs were promised in their private conversations with the Conjecture leadership team? Have you talked to the VCs or asked them for a statement?
Of course, you’re free to speculate from the outside but my understanding is that Conjecture actually managed to choose fairly aligned investors who do understand the mission of solving catastrophic risks. I haven’t talked to the VCs either, but I’ve at least asked people who work(ed) at Conjecture.
In conclusion:
1. I think writing critiques is good but really hard without insider knowledge and context.
2. I think this piece will actually (partially) misinform a large part of the community. You can see this already in the comments where people
without context say this is a good piece and thanking you for “all the insights”(update: I misunderstood a comment and don’t think my original phrasing applies anymore).3. The EA/LW community seems to be very eager to value critiques highly and for good reason. But whenever people use critiques to spread (partially) misleading information, they should be called out.
4. That being said, I think your critique is partially warranted and things could have gone a lot better at Conjecture. It’s just important to distinguish between “could have gone a lot better” and “we recommend not to work for Conjecture” or adding some half-truths to the warranted critiques.
5. I think your post on Redwood was better but suffered from some of the same problems. Especially the fact that you criticize them for having not enough tangible output when following a hits-based agenda just seems counterproductive to me.