I currently lead EA funds.
Before that, I worked on improving epistemics in the EA community at CEA (as a contractor), as a research assistant at the Global Priorities Institute, on community building, and Global Health Policy.
Unless explicitly stated otherwise, opinions are my own, not my employer’s.
You can give me positive and negative feedback here.
Disproportionately deterring bad projects is a crux. I think if people are running a “minimise criticism policy” they aren’t going to end up doing very useful things (or they’ll do everything in secret or far from EA). I currently don’t think nearly enough people are trying to start projects and many project explorations seem good to me on net, so the discrimination power needs to be pretty strong for the benefits to pencil.
I think there are positives about criticism which I didn’t focus on, but yeah if I were to write a more comprehensive post I think the points you raised are good to include.
I’m not convinced that criticism is very counterfactually educational for 3rd parties. Particularly when imo lots of criticism is bad. Feels like it could go either way. If more criticism came from people who had run substantial projects or operated in the same field or whatever I think I’d trust their takes more. Many of the examples raised in this thread are good imo and have this property.
I agree that this is one of the upsides of criticism on the forum. I don’t think it outweighs the costs in many cases.
Have you run a public EA project before or spent time talking to founders of similar projects? This seems extremely off to me.
Damnit!
Rohin and Ben provided some examples that updated me upwards a little on critique posts being useful.
I think most of my points are fairly robust to the different definitions you gave so the line isn’t super important to me. This feels a bit nitpicky.
I don’t think that “criticism clearly mattering (e.g. getting someone to significantly improve their project)” is a very specific claim. I think that one of the main responses people would like to see to criticism of a specific project is for that project to change in line with the criticism. Unlike many of the other proposed benefits of criticism, it is a very empirical claim.
It suspect you think that this post should have been closer to “here are some points for and against criticism” on the EA Forum, but I don’t think posts need to be balanced or well-rounded like that, especially because, from my perspective, the forum is too pro-criticism but yeah, seems fine for you not to engage with this kind of content—I definitely don’t think you’re obliged to.
Those are great examples.
on
Almost all EA projects have low downside risk in absolute terms
I agree with some of the points on point 1, though other than FTX, I don’t think the downside risk of any of those examples is very large. I’d walk back my claim to the downside risk to most EA projects seems low (but there are ofc exceptions).
on
There are almost no examples of criticism clearly mattering
Agree that criticisms of AI companies can be good, I don’t really consider them EA projects but it wasn’t clear that was what I was referring to in my post—my bad. Responding quickly to some of the other ones.
Concerns with Intentional Insights
This seems good, though it was a long time ago.
It’s hard to tell, but I’d guess Critiques of Prominent AI Safety Labs changed who applied to the critiqued organizations
Idk if these are “EA” projects. I think I’m much more pessimistic than you are that these posts made better things happen in the world. I’d guess that people overupdated on these somewhat. That said, I quite like these posts and the discussion in the commentts.
Gossip-based criticism of Leverage clearly mattered and imo it would have been better if it was more public
This also seems good, though it was a long time ago and I wasn’t around when leverage was a thing.
Sharing Information About Nonlinear clearly mattered in the sense of having some impact, though the sign is unclear
Sign seems pretty negative to me.
Same deal for Why did CEA buy Wytham Abbey?
Sign seems pretty negative to me. Like even the title is misleading and this generated a lot of drama.
Back in the era when EA discussions happened mainly on Facebook there were all sorts of critiques and flame wars between protest-tactics and incremental-change-tactics for animal advocacy, I don’t think this particularly changed what any given organization tried to do, but it surely changed views of individual people
Not familiar but maybe this is useful? Idk.
Open Phil and RP both had pieces that were pretty critical of clean meat work iirc that were large updates for me. I don’t think they were org-level critiques, but I could imagine a version of them being critiques of GFI.
So overall, I think I stand by the claim that there aren’t many criticisms that clearly mattered, but this was a positive update for me. Maybe I should have said that a very small fraction of critical EA forum posts have clear positive effects or give people useful information.
This was a great comment—thanks for writing it.
Mostly, people with active projects are discouraged by critiques and starting new public ambitious projects is much less fun if there are a bunch of people on a forum who are out to get you.
I didn’t realise this, but apparently “regret minimisation” was popularised by Jeff Bezos, and this is literally how Jeff describes it. I guess it’s a very natural concept and this name is pretty descriptive.
https://www.fastcompany.com/90662406/jeff-bezos-uses-a-simple-framework-for-making-big-decisions-heres-how-it-works
(edit: I was mostly thinking of public criticism of EA projects—particularly projects with <10 FTE. This isn’t clear from the post. )
I think it’s wild how pro-criticism of projects the EA forum is when:
Most people agree that there are a lack of good projects, and public critique clearly creates barriers to starting new projects
Almost all EA projects have low downside risk in absolute terms
There are almost no examples of criticism clearly mattering (e.g. getting someone to significantly improve their project)
Criticism obviously driving away valuable people from the forum—like (at least in part) the largest ever EA donor
(Less important, but on priors, I’m not sure you should expect high-quality criticism of EA projects because they are often neglected, and most useful criticism comes from people who have operated in a similar area before. )
Edit: I’m curious about counterexamples or points against any of the bullets. There are lots of disagree reacts, and presumably some of those people have seen critques that were actually useful.
Oh right, I’m a bit confused about why you asked this question. It seems unreasonable to say that there are no hypothetical EA orgs that should be democratically elected, but also that result isn’t very decision relevant.
Like I can imagine a fictional “EAs for democratically elected EA orgs” org which should imo be democratically elected, does that mean I should have said yes to this poll? Or should we mostly consider real and highly plausible new orgs?
I like this post a lot. I often use “regret minimisation” as a frame when making important decisions, which feels quite similar to “doing things you would endorse on your deathbed”.
Some Positions In EA Leadership Should Be Elected
I think the way the question is framed is a bit leading. The optimal amount of democratically elected EA leadership is probably non-zero (idk lots of things are non-zero), but I struggled to come up with any examples of currently existing EA leadership positions that I felt should be elected.
I agree with this take, but I think that this is primarily an agency and “permission to just do things” issue, rather than people not being good at prioritisation. It requires some amount of being willing to be wrong publicly and fail in embarrassing ways to actual go and explore under-explored things, and in general, I think that current EA institutions (including people on the EA forum) don’t celebrate people actually getting a bunch of stuff done and testing a bunch of hypothesis (otoh, shutting down your project is celebrated which is good imo—though ~only if you write a forum post about it).
I guess overall, I don’t think that people are as bottlenecked on how much prioritisation they are doing, but are pretty bottlenecked on “just doing things” and not staring into the void enough to realise that they should move on from their current project or pivot (even when people on the forum will get annoyed with them). In part, these considerations converge because it turns out that many projects are fairly bottlenecked by the quality of execution rather than by the ideas themselves, and actually trying out a project and iterating on it seems to reliably improve projects.
In general, my modal advice for EAs has shifted from “try to think hard about what is impactful” towards “consider just doing stuff now” or “suppose you have 10x more agency, what would you do? Maybe do that thing instead?”.
There’s an adjacent take I agree which is more like:
1. AI will likely create many high-stakes decisions and a confusing environment
2. The situation would be better if we could use AI to stay in-step with AI progress on our ability to figure stuff out
3. rather than waiting until the world is very confusing, maybe we should use AIs right now to do some kinds of intellectual writing, in ways we expect to improve as AIs improve (even if AI development isn’t optimising for intellectual writing).
I think this could look a bit like company with mostly AI workers that produces writing on a bunch of topics, or as a first step, heavily LM written (but still high-quality) substack.
I’d bet that current models with less than $ 100,000 of post-training enhancements achieve median human performance on this task.
Seems plausible the metaculus judges would agree, especially given that that comment is quite old.
I think the mainline plan looks more like use the best agents/model internally and release significantly less capable general agents/models, very capable but narrow agents/models, or AI generated products.
Yeah that’s fair. I’m a lot more bullish on getting AI systems that satisfy the linked question’s definition than my own one.
I’m a huge fan of having high standards. Posts that are like “we reproduced this published output and think they made these concrete errors” are often great. But I notice much more “these people did a bad job or spent too much money” takes often from people who afaict haven’t done a bunch of stuff themselves so aren’t very calibrated, and don’t seem very scope sensitive. If people saw their projects being critiqued and were then motivated to go and do more things more quickly I’d think that was great (or were encouraged to do more things more quickly from “fear” of critiques) I think we’d be in a better equilibrium.
For example people often point out that LW and the forum are somewhat expensive per user as evidence they are being mismanaged and imo this is a bad take which is rarely made by people who have built or maintained popular software projects/forums or used the internet enough to know that discussion of the kind in these venues is really quite rare and special.
To be clear, I think the “but have they actually done stuff” critique should also be levelled at grantmakers. I’m sympathetic to grantmakers who are like “the world is burning and I just need to do a leveraged thing right now” but my guess is that if more grantmakers had run projects in the reference class of things they want to fund (or founded any complicated or unusual and ambitious projects) we’d be in a better position. I think this general take is very common in YC/VC spaces, which perform a similar function to grantmaking for their ecosystem.
Many examples of criticism in replies, are high quality posts that I think improve standards. I may spend an hour going through the criticism tag and sorting them into posts I think are useful/anti-useful to check.