Note: the results have now been published in a separate post here.
It has come to my attention that some people are reluctant to post or comment on the EA Forum, even some people who read the forum regularly and enjoy the quality of discourse here.
What stops you posting?
What might make it easier?
You can give an anonymous answer on this Google Form. I intend to share responses in a follow-up post.
I’m anticipating a lot of replies being about how to reduce the aspects of the Forum that make people feel bad about posting, such as harsh criticism, and I think this is good as far as it goes. However I think it’s important to think about why people do things as being a balance between costs and benefits, and also think about how we could make the benefits larger or more salient.
“What makes you stop posting?” could be reframed as “What makes you post in the first place?”, and “What might make it easier?” could be reframed as “What might make you publish posts that were more challenging for you (practically or emotionally)?”
The quality of many forum posts is very high, including from people who are not paid by a research org to write them and have no direct connection to the community (such as these two). So even if you only factor in the time cost, you would still have to suppose some pretty large benefits to explain why people write them.
I have some ideas about what these benefits are:
If you see yourself as a temporarily embarrassed academic who has had to get a proper job as a result of economic forces, posting on the EA Forum (or LessWrong) is about as close as you can get to publishing in an academic journal without actually doing that. Your ideas will be taken seriously by a community of people you respect, and you are actually likely to get more substantive engagement than if you were a non-top-tier academic publishing in a non-top-tier journal. This kind of intellectual discussion is exciting to a lot of people, and is reason enough in itself.
Related to your ideas being taken seriously, they can also steer a community of thousands of people and billions of dollars. It’s reasonably common for this to happen, for instance GiveWell changed how they do their cost effectiveness analyses partly as a result of that post by @Froolow that I linked above. There are lots of good examples from LessWrong too, such as Katja Grace’s Let’s think about slowing down AI which was clearly [citation needed] pivotal in getting that idea recognised as a respectable mainstream position.
You can get ~material benefits~ from posting on the forum. We have found that a lot of people get jobs at least via, and possibly because of, the Forum. Also it might make people relate to you better at least socially, if not professionally, if you have a couple of good Forum posts explaining your best ideas. For people in largely EA social circles this can be a big benefit.
These are all things that seem like they could be leant on to get people to publish more and better content on the Forum, for instance GiveWell did a Change Our Mind contest which clearly increased the second one.
On the issue of criticism specifically, I am a bit less optimistic about this being a lever to pull to get people to post more. I have written before about why I think reassuring people that they won’t be criticised can be wrongheaded[1]. Obviously I think it’s good to make sure criticism is of ideas and not people/their values, and to be polite in a common sense way such as trying to give criticism as a compliment sandwich.
But ultimately it’s not the bread from a sandwich that people remember, and even being criticised for just your ideas but not your values feels bad to most people. But in order to get the benefit of people taking your ideas seriously they do need to be open to criticism, so I think it’s quite difficult to reduce this in practice.
And then on the question of whether reducing “the bad kind of criticism” (i.e. criticism of values/personal attacks) would actually make people post more[2], one bit of evidence that goes against this is that the posts that get the most engagement tend to be “drama” posts where the comments actually involve proportionately more of “the bad kind of criticism”. Obviously there are a lot of confounding factors here but one relevant idea is that “standing up against criticism of your values” can actually feel better/more wholesome than standing up against criticism of your ideas, because if your idea is proven wrong then that’s just a bit embarrassing, whereas your values usually can’t really be proven wrong.
So anyway, overall I think a better argument to make to try and persuade people to post more is less like “Don’t worry you won’t be criticised” and more like “It’s brave to post on the Forum, for the same reason that it’s brave to stand up and talk in front of a group of people. It’s brave because you’re opening yourself up to the very real downside of being criticised, but there are all these great upsides too, so you might just have to take some of the hits if you want to get them”.
And then, from the perspective of people with a community-minded interest[3] in getting more people to post more of their ideas, I think leaning on the benefits side (such as the three things I mentioned) could be at least as effective as leaning on the costs side.
Note: I am a developer on the Forum, but this comment doesn’t necessarily represent the views of the whole team
TL;DR I think it’s often a false reassurance, and so people will see through it or be wrongly convinced that they won’t be criticised, which is unfair to them if they then are
As mentioned above, I think it’s good to reduce this for lots of other reasons
Which includes the actual Forum team, which I work on, but also other users who have the best interests of the community at heart
This was a really good point, and it made me think for quite a while. I’ve posted on the forum a lot since re-entering the EA community (to the point I’ve consciously tried to do it less so it’s not spammy!), but I’ve never really thought about why I put so much effort into posts or, indeed, my comments. There’s not much of a difference between the two, really, since one of my recent comments on someone’s post was 1,207 words long haha. All in good faith, though!
I don’t gain anything from posting. I have a good job outside of EA, I’m not part of any EA groups, and I don’t particularly want or need anything from anyone in EA. So there’s nothing concrete there. I’ve never really thought about it, but it boils down to sharing knowledge. If the things I know about can help someone else somewhere do good better, or address a problem, or whatever then I like the idea that maybe my posts are useful to people. My specialist area is also kind of niche and difficult to enter, so I like the idea of making it more understandable and approachable.
I never get any karma really, or even high reads, but I do get high retention so people (~50%) tend to read my posts all the way through which I really like. So that ties in with what I think my core motivation is.
I’m a huge fan of this. It’s rare, but if ever I really disagree with someone’s post I’ll always highlight what I liked about it too. In my experience aside from being polite, it also results in better conversation.
Edit: Grammar
Just chiming in with a quick note: I collected some tips on what could make criticism more productive in this post: “Productive criticism: what could help?”
I’ll also add a suggestion from Aaron: If you like a post, tell the author! (And if you’re not sure about commenting with something you think isn’t substantive, you can message the author a quick note of appreciation or even just heart-react on the post.) I know that I get a lot out of appreciative comments/messages related to my posts (and I want to do more of this myself).
I think it would be particularly valuable to tell the author (and the rest of the Forum) how you think reading the post created impact. More concrete / personal examples might include: I am more likely to donate to X / using Y strategy because I read this post, or I will be more likely to advise people who come to me for career advice to do Z. I think many potential authors may be wondering whether the time spent writing actually produces impact in comparison to the counterfactual uses of their time, so feedback on that point should be helpful.
Good question.
I have a few draft forum posts over the years that I don’t really have an interest in publishing.
I feel like I’ve found it pretty stressful each time I’ve posted or even commented. (yes, even this) After posting, I waste so much time checking the comments and changes in karma. I, naturally, feel every downvote or negative comment far more than upvotes or positive comments. Positive comments are actively discouraged (downvoted) here, which might make this worse.
Some of the most frustrating comments are ones that correct some minor details that, don’t impact the posts core point, or mention some exceptions as if they discount the whole idea.
I think this is a big part of why I don’t finish my draft posts. The drafts are short and just cover the main point I have, but I need to 10x my time, effort and length so that I address every obvious exception, edge case, rebuttal…
I think ultimately I find it to be a very negative feedback loop. Posting is stressful. Every single time. It’s not like I’m anxious before posting and then after I post I get a big hit of dopamine from the responses. This would provide some positive feed back loop like posting on social media.
My neurotransmitters take no account of expected value calculations.
Do you think it would be helpful to post the drafts as-is with a burner account during Draft Amnesty Week and pre-commit to not looking at the comments section for a week after that? If you’re experiencing negative emotions with every marginal downvote or negative comment, it might be helpful to just see it all in one go (which will make it difficult to feel individual downvotes individually).
Also, do you think it would be helpful to say that this is an abbreviated post for which you are interested in feedback about whether it makes sense to invest the time for the 10x work to polish and expand you describe?
One reason that people might make people hesitant would be that people tend to be very critical of posts, particularly if they are not in line with or adjacent to established cause areas or lines of inquiry.
Something that might make things better:
If you’re criticizing a post, is the point you are making core to the idea that the poster is making? Is it possible that small revisions or extensions would make their idea address your criticism?
Think of the idea in EV terms. Perhaps if you have an intuition regarding its EV, there might be a way to test whether an idea is a particularly effective way of doing good.
Try to identify the key question(s) on which your disagreement with the poster depends. This creates an empirical framework that could potentially lead to agreement in the future, if further understanding regarding the key point(s) is capable of being attained.
Generally, I think the EA community is very good with being critical and identifying potential flaws in ideas or limitations. But I think there is a tendency to assume that all good ideas have already been explored or exhausted, especially if something sounds similar to something that the EA is familiar with. I think open-minded curiosity, with a mind towards constructing empirical tests that might be probative, would probably be far more helpful than just the exhaustive redteaming that is typical of EA forum responses.
Similar to what Brad says, posts not in line with EA mainstream, or just exploring or giving ideas, or not written in EA-style, or drafty are often down-voted very early on without engaging in discussion or giving any reason for the down-vote.
Even though forum moderators try to engage people to write even if the post is not perfectly polished or thought through—most people are very busy!-- to incentivize exchange of ideas, the dynamics of the forum make it basically useless as such posts are usually very quickly hidden. It often feels useless to write anything.
Another reason, which is understandable and difficult to avoid, but that I find a bit surprising in this forum, is the status and political dynamics: I’ve seen several times good posts with very low karma and afterwards very similar posts written by known EAs that have lots of karma.
I know some people disagree, but very early downvoting (especially strong downvoting) often ends up functionally being a vote to cut off discussion of a post before it even starts. Those votes make it less likely that others will see and click on the post, will cause them to come to the post with a more negative lens (consciously or otherwise), and will make them less likely to engage (why make the effort if you think the post is headed into oblivion soon enough because of low karma?). That’s a hard trifecta for a post to overcome.
Like motions to cut off debate early in most parliamentary systems, the bar for functionally cutting off discussion of a post at an early juncture should be relatively high. (It is 2⁄3 majority in most parliamentary systems, IIRC.)
I wonder if we would benefit from something like a system that hides the karma (and treats it as zero for visibility purposes) of posts that have less than <some quantity> of engagement. That way posts would get a “grace period” before getting hidden.
The potential cost there is that there are occasionally some really bad posts that deserve the rapid downvote—such as those that don’t respect important Forum norms, even if not quite to the point of formal moderator action. In those cases, the hail of downvotes allows the Forum to deal with those quickly and without moderator action that could be seen as “censoring” someone. But maybe those kinds of posts are rare enough to incur the cost.
Very much agree with your suggestions for healthy engagement with posts, thanks for writing them.
Also, FWIW, I’ve seen a lot less of a worrying trend towards criticism than I expected before joining the Forum team 4 months ago. Before joining, I had the idea that Forum users would tear ideas apart, sometimes in kind of harsh ways. I’d also internalised the meme that this was a reason for people not to post.
I’ve been pleasantly surprised by what I’ve seen. Specifically, if a post seems unsuitable for the Forum, or particularly ill-conceived, it is generally quietly downvoted rather than openly critiqued. In many cases, posts that I thought might be particularly open to criticism were given very helpful, good faith comments.
The more critical comments I’ve seen have been on the work of organisations rather than individuals. Although that might be difficult for the organisations, it also seems more fair—orgs are being funded for the content they produce, so it matters to all of us that it is as good and correct as it can be.
If anyone reading has the opposite impression, I’d love to hear about it (here or in DM).
Note: I am the content manager on the Forum, but these are my personal impressions, not those of the team.
I’m not sure if this is better or worse. People often get confused and frustrated by downvotes without explanation (cf. the “Why am I being downvoted?” comments in response to one’s own comments, sometimes from fairly experienced users). And newer users are less likely to intuit the probable reason for the downvotes.
Is there a “Why might my post/comment be getting downvotes and/or little engagement” writeup somewhere? If not, maybe I should sketch that at some point as it might give people some general understanding to those users (and/or allow those offering feedback to do so more efficiently—“much of it relates to reason 3 in the writeup” rather than writing reason 3 out themselves).
In short, for me it is much less about feeling like the EA Forum is unwelcome of posts from people with less expertise (e.g. me) than others. It is more that if I share a post, I want it to be adding significantly new insights and not be an iteration of what has been written about already. This consequently takes time to roughly know what content exists already. My status quo when coming across something that is novel to me, I by default assume that it has probably been covered by others already.
However, I have not figured out whether I should start posting because the personal benefits of it (having a project to work towards, formulating arguments, etc.) could be sufficient enough for me to post, without paying too much attention to the benefits for other Fourm readers.
I think there’s at least two categories:
The beginner who is scared of ridicule.
The senior, who don’t have time to write to the forum standard without risking reputation.
I’m more interested in what we can do to encourage the latter group. My impression is that many senior people are reluctant to post, as they don’t have time to write something sufficiently well-argued and respond to the comments.
Instead many good discussions take place in signal groups, google docs and email threads. In a perfect world, these discussions would be in the forum. The issue right now is that if those conversations would take place on the forum, too many people chime in with long, eloquently written, but wrong arguments that the subject matter experts now have to spend additional time shutting down or look like they are dodging a hard question.
Additionally lowering their bar for public engagement, puts them at risk of attack. There are people reading this forum just to find ammunition for hit pieces. A poorly worded comment from the leader of an organization will be used against them.
If you compare this recent post I did, it has about the same karma on lesswrong and the forum, but on Lesswrong there are 36 comments, on here, 6. If I were gonna focus on writing forecasting articles, I’d write for LessWrong.
(I posted this response to a pre-edit version of Nathan’s post)
I’d sort of noticed your EA Forum drop-off (though perhaps this is my pattern-matching after the fact), if you had any more thoughts on the ‘vibe-shift’ as you perceive would be open to hearing them—either in reply or DM.
When I post my ‘EA-Forum Data in 2023’ post I’ll make a note to see if there’s a drop-off in commenting/Forum engagement. I do think Forum usage in late 2022/early 2023 was abnormally high (for obvious reasons)
Yeah I initially wrote that late at night in a mood. Ooops.
I think it might be worth testing what has happened to comments from high karma users/old accounts in the last year compared to previous years. I would predict a significantly higher drop off.
Why? I guess I think there is inter-party conflict in EA between those who wish for it to be a charming well behaved space and those who wish anything to be able to be discussed. And each group taxes the other a bit and finds friction. This is expensive to all parties so demand shifts down. I am a bit disagreeable but I don’t really want to spend my community time in slight conflict. So I use it less.
Perhaps also check what has happened to LessWrong usage and linked accounts on lesswrong.
I have said for a long time that the forum would benefit from some kind of mediation between parties in times of stress, before we all grow to dislike one another. I don’t think I’m the person to do that, so I don’t, but someone could.
Honestly, I just care about the community a bit less than I once did. I’ve had some bad experiences (and I am sure there are those who think I am the problem) so I feel less confident this place will be a place for me in 5 years time, so why would I invest here at the moment.
I think also the quality of the comments has gone down. I have less expectation that I am gonna read interesting things.
Perhaps it sounds strange from me that is so active on the forum: But what is the value of the forum, and therefore of contributing to it? Maybe the answer is lying around somewhere, but I have not seen anything. Don’t get me wrong, I think perhaps the forum is super valuable, kind of, or perhaps even more important than EAGs—these venues, virtual and physical are more or less the only and therefore core venues for the community to interact.
If there is some cost effectiveness calc for the forum, I would be super keen to see that written up as a post clearly and easy-to-read. Ideally also with some follow on effectiveness calc or estimate for people volunteering time to post here.
It would, for example, be super valuable to have something like “if you spend 5 minutes writing a comment that gets at least 10 in Karma, the is likely as cost effective as earning $200/hour and donating to AMF” or “if you spend 20 hours writing a post that gets at least 50 karma it is equivalent to earning $300/hour and donating all of that to AMF”.
When combined with the difficulty of assessing impact of Forum posts and comments, I think the relationship between karma and impact is too murky to make impact-per-karma a meaningful measure. Posts/comments of a specialized/technical nature will likely have a significantly higher IpK value than meta commentary.
Yeah perhaps, but I have no idea even why the forum is considered cost effective. Is it more because it advances work on causes? Or is it more because it builds and maintains the community? I think the answer to this question will go some way to understanding the strength of your claim. In general, I just wished this information was out there, just something about why we think the forum going is a good use of our time. Right now, I have not seen anything. I remember seeing more information about EAGs and their benefits and their cost effectiveness.
Hi Ulrik,
You may want to check my post on What is the relationship between impact and EA Forum karma?.
Thanks! That does seem super relevant. Is my understanding below roughly correct?
”A post on a non-community topic that receives ~500 karma is roughly equivalent in impact to a high quality research paper in a peer reviewed journal. And such a post receiving about 100 karma or a bit below is about 1-10% of the impact of such a journal article.”
You are welcome!
This makes sense assuming i) impact increases logarithmically with karma, ii) Nuño’s impact estimates of the EA Forum Prize winners are correct, and iii) the relationship between karma and impact among these posts generalises to other posts. However, I have so little confidence about these assumptions that I would not use karma as the only proxy for impact. At best, I would use the logarithm of karma as one component of a weighted factor model (WFM). I note in the post that:
The results have now been released!
Here are the top reasons given, from 20 responses:
Read the full post here.
My two cents (acknowledging I’m just getting acquainted to the community):
TL;DR’s seem important, also “calls to action” may help. If those are included, I imagine less barrier to post. In reading the other posts, it looks like there’s a karma system that makes things challenging here. I wonder if a living “posting guide” might help celebrate discourse and conversation, rather than get bogged down by criticism?
Main question: Are articles making it clear what kind of response would be celebrated?