Related: What else would you do with twice the resources? What would you forego if you had only half of them? What do you think the effect on impact of either funding effect would be?
(The motivation here is that I think transparency around funding tradeoffs is particularly important for meta work, both optically and epistemically. It also promotes a culture in which inappropriate spending, like fancy trips to the Bahamas, is less likely to occur.)
The Online team overall spends ~$2M per year. 75% of that is on “people costs” for our 8 full-time employees, all-in.[1] Is that all on the Forum? Kinda. Our main focus right now, and for the past 6 months, has been the core Forum product, due to, basically, the community spike here. However, during normal times, including starting ~now, we spend 30-60% of our effort on expansionary projects, such as this major project from last year. I would model us like a tech startup, where most of our money is going to “R&D” that should pay off in future impact.
Which then nicely answers part of Jason’s question. If we had less money, then, given that our costs are so staff-driven, we would need to have fewer staff, and then we would be investing less in future impact. As for what we would do if we had double the funding, we are currently more bottlenecked by trying to remain a two-pizza team. It’s possible that we would make a hire on the margin, but we wouldn’t accept double the funding.
It seems that we’re spending 2 million a year on a glorified subreddit. What’s the donors case for this being a good use of funds? Isn’t the forum good enough as is? Some reddit mods handle ~100x the content, with 100x more traffic, for 0% of the cost.
What have you accomplished that justifies the spend on the forum to date? Couldn’t 99% of this have been accomplished by forking lesswrong and making minor tweaks?
It’s worth clarifying that the Online team’s budget was much less than $2M budget before 2022, and a lot of the development of the Forum happened between 2018 and 2021 with one developer (JP). The budget and team grew in the last 12 months with FTX funding and significant growth in the EA movement, and now we’re evaluating what portion of the $2M budget to put towards continuing to develop the Forum versus new projects we’re exploring. It’s also worth noting that a portion of the $2M budget goes towards the content team (Lizka), and moderation and support contractors, who I think are super valuable but not the bulk of our budget, so I’ll instead focus on the product/engineering team and the platform in the next part of my response.
When I think of how the Forum compares to other platforms, I see some major tradeoffs. Compared to social media sites like reddit, twitter, and facebook groups, the EA Forum encourages long form content and high quality discussion much better, is a more focused space (with no ads or non-EA content), and indexes and organizes content better. Compared to individual or group blogs, the Forum is much more open (anyone can post), and prioritizes discussion better (commenting features are more hidden on many blogs).
Put together, I think these mean that the EA Forum is a more attractive space for EAs, and ultimately the network of users is a key factor in determining the value of the site.
This isn’t to say that the Forum can claim 100% counterfactual value for every interaction that happens in this space (compared to a world where it didn’t exist and we had a subreddit), but I do think the Forum has been valuable. I wish we could measure this value in a really clean way without too much effort (and we have near term plans to do more work here), but on the other hand I think it’s important we avoid the trap of focusing too much on measurement and evaluation for a public good that has fairly diffuse impact.
While I’m sympathetic to the view that ~$2M is too much to spend, the quality of moderation here is much higher than in any open-access, high-volume space I am aware of on Reddit.[1] So I don’t think it is helpful to compare the mod workload here with what “reddit mods handle” (usually large mod teams on the major subreddits).
I think some subreddits do a good job of moderating to create a culture which is different from the default reddit culture, e.g. /r/askhistorians. See this post for an example, where there are a bunch of comments deleted, including one answer which didn’t cite enough sources. Maybe this is what you have in mind when you refer to “moderating with an iron fist” though, which you mention might be destructive!
Seems like the challenge with reddit moderation is that users are travelling between subreddits all the time, and most have low quality/effort discussion norms. Whereas on the Forum, the userbase is more siloed, which I guess would make good quality moderation easier.
I think benchmarking at reddit moderation is probably the wrong benchmark. Firstly, because the tail risk of unpaid moderation is really bad (e.g. the base rate of moderator driven meltdowns in big subreddits is really high). Secondly, I just don’t think we should underpay people in EA because (a) it creates financial barriers to entry to EA that have long-term effects (e.g. publishing unpaid internships have made the wider labour market for journalism terrible) (b) it’ll create huge amounts of more informal barriers that mean we lean on more informal relationships in EA even more.
~$120,000 (sans benefits). It varies greatly by role and location. You can get a sense for roughly what a given role might pay by looking at ourjobpostings. As mentioned elsethread, these salaries are aimed at not being huge sacrifices for tech workers living in expensive american cities, while also not being egregiously luxurious in lower salary places like Oxford. I imagine some engineers might look at that number and think it’s low compared to their expectations, and some non-engineer Brits might think it’s quite high. I encourage you to look at the job postings for the ranges.
It sounds like the 1.5 million does not include $ on the forum team or facilitators—what does the total spending/average salary look like if you include these groups?
Ah, thanks for finding that, I posted my reply before seeing this comment.
By Forum team do you mean non-CEA moderators? We have volunteers on our team, which throws off our actual spend, but I’ve budgeted as if we have to pay for all of our moderation contracting, and have budgeted $20,000/yr. We pay moderators $40/hr, and facilitators $30/hr.
How much is spent, total, on the EA forum each year?
Related: What else would you do with twice the resources? What would you forego if you had only half of them? What do you think the effect on impact of either funding effect would be?
(The motivation here is that I think transparency around funding tradeoffs is particularly important for meta work, both optically and epistemically. It also promotes a culture in which inappropriate spending, like fancy trips to the Bahamas, is less likely to occur.)
The Online team overall spends ~$2M per year. 75% of that is on “people costs” for our 8 full-time employees, all-in.[1] Is that all on the Forum? Kinda. Our main focus right now, and for the past 6 months, has been the core Forum product, due to, basically, the community spike here. However, during normal times, including starting ~now, we spend 30-60% of our effort on expansionary projects, such as this major project from last year. I would model us like a tech startup, where most of our money is going to “R&D” that should pay off in future impact.
Which then nicely answers part of Jason’s question. If we had less money, then, given that our costs are so staff-driven, we would need to have fewer staff, and then we would be investing less in future impact. As for what we would do if we had double the funding, we are currently more bottlenecked by trying to remain a two-pizza team. It’s possible that we would make a hire on the margin, but we wouldn’t accept double the funding.
Including taxes, travel, team retreats, professional development budgets, etc.
It seems that we’re spending 2 million a year on a glorified subreddit. What’s the donors case for this being a good use of funds? Isn’t the forum good enough as is? Some reddit mods handle ~100x the content, with 100x more traffic, for 0% of the cost.
What have you accomplished that justifies the spend on the forum to date? Couldn’t 99% of this have been accomplished by forking lesswrong and making minor tweaks?
It’s worth clarifying that the Online team’s budget was much less than $2M budget before 2022, and a lot of the development of the Forum happened between 2018 and 2021 with one developer (JP). The budget and team grew in the last 12 months with FTX funding and significant growth in the EA movement, and now we’re evaluating what portion of the $2M budget to put towards continuing to develop the Forum versus new projects we’re exploring. It’s also worth noting that a portion of the $2M budget goes towards the content team (Lizka), and moderation and support contractors, who I think are super valuable but not the bulk of our budget, so I’ll instead focus on the product/engineering team and the platform in the next part of my response.
When I think of how the Forum compares to other platforms, I see some major tradeoffs. Compared to social media sites like reddit, twitter, and facebook groups, the EA Forum encourages long form content and high quality discussion much better, is a more focused space (with no ads or non-EA content), and indexes and organizes content better. Compared to individual or group blogs, the Forum is much more open (anyone can post), and prioritizes discussion better (commenting features are more hidden on many blogs).
Put together, I think these mean that the EA Forum is a more attractive space for EAs, and ultimately the network of users is a key factor in determining the value of the site.
This isn’t to say that the Forum can claim 100% counterfactual value for every interaction that happens in this space (compared to a world where it didn’t exist and we had a subreddit), but I do think the Forum has been valuable. I wish we could measure this value in a really clean way without too much effort (and we have near term plans to do more work here), but on the other hand I think it’s important we avoid the trap of focusing too much on measurement and evaluation for a public good that has fairly diffuse impact.
Hope that was helpful.
This isn’t a convincing less of analysis to me, as these two things can both be true at the same time:
The EA Forum as a whole is very valuable
The marginal $1.8M spent on it isn’t that valuable
i.e., you don’t seem to be thinking on the margin.
While I’m sympathetic to the view that ~$2M is too much to spend, the quality of moderation here is much higher than in any open-access, high-volume space I am aware of on Reddit.[1] So I don’t think it is helpful to compare the mod workload here with what “reddit mods handle” (usually large mod teams on the major subreddits).
Curating higher-quality content by moderating with an iron fist is easier, but would destroy a significant portion of the Forum’s value in my opinion.
I think some subreddits do a good job of moderating to create a culture which is different from the default reddit culture, e.g. /r/askhistorians. See this post for an example, where there are a bunch of comments deleted, including one answer which didn’t cite enough sources. Maybe this is what you have in mind when you refer to “moderating with an iron fist” though, which you mention might be destructive!
Seems like the challenge with reddit moderation is that users are travelling between subreddits all the time, and most have low quality/effort discussion norms. Whereas on the Forum, the userbase is more siloed, which I guess would make good quality moderation easier.
I think benchmarking at reddit moderation is probably the wrong benchmark. Firstly, because the tail risk of unpaid moderation is really bad (e.g. the base rate of moderator driven meltdowns in big subreddits is really high). Secondly, I just don’t think we should underpay people in EA because (a) it creates financial barriers to entry to EA that have long-term effects (e.g. publishing unpaid internships have made the wider labour market for journalism terrible) (b) it’ll create huge amounts of more informal barriers that mean we lean on more informal relationships in EA even more.
Just noting that there is already an effective altruism subreddit. I think we should post in both places and see if the difference is worth the cost
What is the average salary of the forum team?
~$120,000 (sans benefits). It varies greatly by role and location. You can get a sense for roughly what a given role might pay by looking at our job postings. As mentioned elsethread, these salaries are aimed at not being huge sacrifices for tech workers living in expensive american cities, while also not being egregiously luxurious in lower salary places like Oxford. I imagine some engineers might look at that number and think it’s low compared to their expectations, and some non-engineer Brits might think it’s quite high. I encourage you to look at the job postings for the ranges.
The salary of the content specialist is here.
It sounds like the 1.5 million does not include $ on the forum team or facilitators—what does the total spending/average salary look like if you include these groups?
Ah, thanks for finding that, I posted my reply before seeing this comment.
By Forum team do you mean non-CEA moderators? We have volunteers on our team, which throws off our actual spend, but I’ve budgeted as if we have to pay for all of our moderation contracting, and have budgeted $20,000/yr. We pay moderators $40/hr, and facilitators $30/hr.