Over the last few years, the EA Forum has taken a few turns that have annoyed me:
It has become heavier and slower to load
It has added bells and whistles, and shiny notifications that annoy me
It hasn’t made space for disagreable people I think would have a lot to add. Maybe they had a bad day, and instead of working with them forum banned them.
It has added banners, recommended posts, pinned posts, newsletter banners, etc., meaning that new posts are harder to find and get less attention.
To me, getting positive, genuine exchanges in the forum as I was posting my early research was hugely motivating, but I think this is less likely to happen if the forum is steering readers somewhere else
If I’m trying to build an audience as a researcher, the forum offers a larger audience in the short term in exchange for less control over the long-term, which I think ends up being a bad bargain
It has become very expensive (in light of which this seems like a good move), and it just “doesn’t feel right”.
Initially I dealt with this by writing my own frontend, but I ended up just switching to my blog instead.
Just as a piece of context, the EA Forum now has about ~8x more active users than it had at the beginning of those few years. I think it’s uncertain how good growth of this type is, but it’s clear that the forum development had a large effect in (probably) the intended direction of the people who run the forum, and it seems weird to do an analysis of the costs and benefits of the EA Forum without acknowledging this very central fact.
I don’t have data readily available for the pre-CEA EA Forum days, but my guess is it had a very different growth curve (due to reaching the natural limit of the previous forum platform and not getting very much attention), similar to what LessWrong 1.0 was at before I started working on it.
$2m / 4000 users = $500/user. They have every reason to inflate their figures, and there’s no oversight, so it’s not even clear these numbers can be trusted.
Many subreddits cost nothing, despite having 1000x more engagement. They could literally have just forked LessWrong and used volunteer mods.
No self-interested person is ever going to point this out because it pisses off the mods and CEA, who ultimately decide whose voices can be heard—collectively, they can quietly ban anyone from the forum / EAG without any evidence, oversight, or due process.
No self-interested person is ever going to point this out because it pisses off the mods and CEA, who ultimately decide whose voices can be heard—collectively, they can quietly ban anyone from the forum / EAG without any evidence, oversight, or due process.
I’ve heard the claim that the EA Forum is too expensive, repeatedly, on the EA Forum, from diverse users including yourself. If CEA is trying to suppress this claim, they’re doing a very bad job of it, and I think it’s just silly to claim that making that first claim is liable to get you banned.
You wrote that “No self-interested person is ever going to point this out because it pisses off the mods and CEA, who ultimately decide whose voices can be heard—collectively, they can quietly ban anyone from the forum / EAG without any evidence, oversight, or due process.” To me, this implies that you believe all self-interested people would refrain from pointing it out for the reason it upsets the mods, who can ban anyone without evidence or oversight. For it to be true that the mods’ capacity to ban people prevents self-interested people from posting on this topic, the mods would have to be likely to exercise that capacity with regards to this topic (well, or our hypothetical self-interested poster would have to mistakenly believe that they are).
$500/monthly user is actually pretty reasonable. As an example, Facebook revenue in the US is around $200/user/year, which is roughly in the same ballpark (and my guess is the value produced by the EA Forum for a user is higher than for the average Facebook user, though it’s messy since Facebook has such strong network effects).
Also, 4000 users is an underestimate since the majority of people benefit from the EA Forum while logged out (on LW about 10-20% of our traffic comes from logged-in users, my guess is the EA Forum is similar, but not confident), and even daily users are usually not logged in. So it’s more like $50-$100/user, which honestly seems quite reasonable to me.
No subreddit is free. If there is a great subreddit somewhere, it is probably the primary responsibility of at least one person. You can run things on volunteer labor but that doesn’t make them free. I would recommend against running a crucial piece of infrastructure for a professional community of 10,000+ people on volunteer labor.
Also, 4000 users is an underestimate since the majority of people benefit from the EA Forum while logged out (on LW about 10-20% of our traffic comes from logged-in users, my guess is the EA Forum is similar, but not confident), and even daily users are usually not logged in.
You posted this graph
If I understand it correctly, it shows that about 50% of EA forum traffic comes from logged-in users, not 10%-20%.
Also, it seems that EA forum gets about 14,000 views per day. So you spend about $2,000,000/(365*14,000) = $0.4 per view. That’s higher than I would expect.
Note that many of these views might not be productive. For me personally, most of the views are like “I open the frontpage automatically when I want to procrastinate, see that nothing is new & interesting or that I didn’t even want to use the forum, and then close it”. I also sometimes incessantly check if anyone commented or voted on my post or comment, and that sort of behaviour can drive up the view count.
That is definitely relevant data! Looking at the recent dates (and hovering over the exact data at the link where the graphs are from) it looks like its around 60% logged-out, 40% logged in.
I do notice I am surprised by this and kind of want confirmation from the EA Forum team they are not doing some kind of filtering on traffic here. When I compare these numbers naively to the Google Analytics data I have access to for those dates, they seem about 20%-30% too low, and it makes me think there is some filtering going on (though my guess is that 80%-90% logged-out traffic definitely still does not seem representative)
Yeah, that seems like the right comparison? Revenue is a proxy for value produced, so if you are arguing about whether something is worth funding philanthropically, revenue seems like the better comparison than costs. Though you can also look at costs, which I expect to not be more than a factor 2 off.
Isn’t a lot of FB’s revenue generated by owning a cookie, and cooperating with other websites to track you across pages? I don’t think it’s fair to count that revenue as generated by the social platform, for these purposes.
Your argument also feels slippery to me in general. Registering that now in case you have a good answer to my specific criticism and the general motte-and-bailey feeling sticks around.
I am not sure what you mean by the first. Facebook makes almost all of its revenue with ads. It also does some stuff to do better ad-targeting, for which it uses cookies and does some cross-site tracking, which I do think drives up profit, though my guess is that isn’t responsible for a large fraction of the revenue (though I might be wrong here).
But that doesn’t feel super relevant here. The primary reason why I brought up FB is to establish a rough order-of-magnitude reference class for what normal costs and revenue numbers are associated with internet platforms for a primarily western educated audience.
My best guess is the EA Forum could probably also finance itself with subscriptions, ads and other monetization strategies at its current burn rate, based on these number, though I would be very surprised if that’s a good idea.
I think the more relevant order of magnitude reference class would be the amount per user Facebook spent on core platform maintenance and moderation (and Facebook has a lot more scaling challenges to solve as well as users to spread costs over, so a better comparator would be the running expenses of a small professional forum)
I don’t think FB revenues are remotely relevant to how much value the forum creates, which may be significantly more per user than Facebook if it positively influences decisions people make about employment, founding charities and allocating large chunks of money to effective causes. But the effectiveness of the use of the forum budget isn’t whether the total value created is more than the total costs of running, it’s decided at the margin by whether going the extra mile with the software and curation actually adds more value.
Or put another way, would people engage differently if the forum was run on stock software by a single sysadmin and some regular posters granted volunteer mod privileges?
Or put another way, would people engage differently if the forum was run on stock software by a single sysadmin and some regular posters granted volunteer mod privileges?
Well, I mean it isn’t a perfect comparison, but we know roughly what that world looks like because we have both the LessWrong and OG EA Forum datapoints, and both point towards “the Forum gets on the order of 1/5th the usage” and in the case of LessWrong to “the Forum dies completely”.
I do think it goes better if you have at least one well-paid sysadmin, though I definitely wouldn’t remotely be able to do the job on my own.
As to costs, I’d have to dig further but looking at the net profit margin for Meta as a whole suggests a fairly significant adjustment. Looking at the ratio between cost of revenue and revenue suggests an even larger adjustment, but is probably too aggressive of an adjustment.
If Meta actually spent $200 per user to achieve the revenue associated with Facebook, that would be a poor return on investment indeed (i.e., 0%). So I think comparing its revenue per user figure to the Forum’s cost per user creates too easy of a test for the Forum in assessing the value proposition of its expenditures.
Yep, totally, it’s a pretty bad proxy. I think the obvious analogy at least for the EA Forum would be that the organizations who are hiring people from the EA Forum are in a comparable position to advertisers, but it’s not amazing.
$500/monthly user is actually pretty reasonable. As an example, Facebook revenue in the US is around $200/user/year, which is roughly in the same ballpark
Facebook ARPU (average revenue per user) in North America is indeed crazy high, but I think misleading as for some reason they include revenues from Whatsapp and Instagram, but only count Facebook users as MAU in the denominator. (edit: I think this doesn’t matter that much) Also, they seem to be really good at selling ads
In any case:
I don’t think this is a good measure of value, I don’t think the average user would pay $200/year for Facebook. (I actually think Facebook’s value is plausibly negative for the average user. Some people are paying for tools that limit their use of Facebook/Instagram, but I guess that’s beside the point)
Reddit’s revenue per user in the US is ~$20 / user / year, 10x less
This doesn’t take into account the value of the marginal dollar given to the EA Forum. (Maybe 90% of the value is from the first million/year?)
I’m not sure if $6000 is in the same ballpark as $200 (edit: oops, see comment below, the number is $500 not $6000)
But I strongly agree with your other points, and mostly I think this is a discussion for donors to the EA Forum, not for users. If someone wants to donate $2M to the EA Forum, I wouldn’t criticize the forum for it or stop using it. It’s not my money.
Users might worry about why someone would donate that much to the Forum, and what kind of influence comes with it, but I think that’s a separate discussion, and I’m personally not that worried about it. (But of course, I’m biased as a volunteer moderator)
I think criticizing CEA for the Forum expenditures is fair game. If an expenditure is low-value, orgs should not be seeking funding for it. Donors always have imperfect information, and the act of seeking funding for an activity conveys the organization’s tacit affirmation that the activity is indeed worth funding. I suppose things would be different if a donor gave an unsolicited $2MM/year gift that could only be used for Forum stuff, but that’s not my understanding of EVF’s finances.
I also think criticizing donors is fair game, despite agreeing that their funds are not our money. First, charitable donations are tax advantaged, so as a practical matter those of us who live in the relevant jurisdiction are affected by the choice to donate to some initiative rather than pay taxes on the associated income. I also think criticizing non-EA charitable donors for their grants is fair game for this reason as well.
Second, certain donations can make other EA’s work more difficult. Suppose a donor really wants to pay all employees at major org X Google-level wages. It’s not our money, and yet such a policy would have real consequences on other orgs and initiatives. Here, I think a pattern of excessive spending on insider-oriented activities, if established, could reasonably be seen as harmful to community values and public perception.
(FWIW, my own view is that spending should be higher than ~$0 but significantly lower than $2MM.)
I think the $500 figure is derived from ($2MM annual spend / 4000 monthly active users). The only work monthly is doing there is helping define who is a user. So I don’t think multiplying the figure by 12 is necessary to provide comparison to Facebook.
That being said, I think there’s an additional reason the $200 Facebook figure is inflated. If we’re trying to compare apples to apples (except for using revenue as an overstated proxy for expenditure), I suggest that we should only consider the fraction of implied expenses associated with the core Facebook experience that is analogous to the Forum. Thus, we shouldn’t consider, e.g., the implied expenditures associated with Facebook’s paid ads function, because the Forum has no real analogous function.
My understanding is that moderation costs comprise only a small portion of Forum expenditures, so you don’t even need to stipulate volunteer mods to make something close to this argument.
(Also: re Reddit mods, you generally get what you pay for . . . although there are some exceptions)
Could you elaborate on what you mean by this?
Thanks for referring to these blog posts!
Over the last few years, the EA Forum has taken a few turns that have annoyed me:
It has become heavier and slower to load
It has added bells and whistles, and shiny notifications that annoy me
It hasn’t made space for disagreable people I think would have a lot to add. Maybe they had a bad day, and instead of working with them forum banned them.
It has added banners, recommended posts, pinned posts, newsletter banners, etc., meaning that new posts are harder to find and get less attention.
To me, getting positive, genuine exchanges in the forum as I was posting my early research was hugely motivating, but I think this is less likely to happen if the forum is steering readers somewhere else
If I’m trying to build an audience as a researcher, the forum offers a larger audience in the short term in exchange for less control over the long-term, which I think ends up being a bad bargain
It has become very expensive (in light of which this seems like a good move), and it just “doesn’t feel right”.
Initially I dealt with this by writing my own frontend, but I ended up just switching to my blog instead.
Just as a piece of context, the EA Forum now has about ~8x more active users than it had at the beginning of those few years. I think it’s uncertain how good growth of this type is, but it’s clear that the forum development had a large effect in (probably) the intended direction of the people who run the forum, and it seems weird to do an analysis of the costs and benefits of the EA Forum without acknowledging this very central fact.
(Data: https://data.centreforeffectivealtruism.org/)
I don’t have data readily available for the pre-CEA EA Forum days, but my guess is it had a very different growth curve (due to reaching the natural limit of the previous forum platform and not getting very much attention), similar to what LessWrong 1.0 was at before I started working on it.
$2m / 4000 users = $500/user. They have every reason to inflate their figures, and there’s no oversight, so it’s not even clear these numbers can be trusted.
Many subreddits cost nothing, despite having 1000x more engagement. They could literally have just forked LessWrong and used volunteer mods.
No self-interested person is ever going to point this out because it pisses off the mods and CEA, who ultimately decide whose voices can be heard—collectively, they can quietly ban anyone from the forum / EAG without any evidence, oversight, or due process.
I’ve heard the claim that the EA Forum is too expensive, repeatedly, on the EA Forum, from diverse users including yourself. If CEA is trying to suppress this claim, they’re doing a very bad job of it, and I think it’s just silly to claim that making that first claim is liable to get you banned.
Where did I say that anyone was trying to suppress this claim? Where did I claim that making it would get you banned?
You wrote that “No self-interested person is ever going to point this out because it pisses off the mods and CEA, who ultimately decide whose voices can be heard—collectively, they can quietly ban anyone from the forum / EAG without any evidence, oversight, or due process.” To me, this implies that you believe all self-interested people would refrain from pointing it out for the reason it upsets the mods, who can ban anyone without evidence or oversight. For it to be true that the mods’ capacity to ban people prevents self-interested people from posting on this topic, the mods would have to be likely to exercise that capacity with regards to this topic (well, or our hypothetical self-interested poster would have to mistakenly believe that they are).
It could just get you treated worse in your every interaction with them, which would be a sufficient deterrent
$500/monthly user is actually pretty reasonable. As an example, Facebook revenue in the US is around $200/user/year, which is roughly in the same ballpark (and my guess is the value produced by the EA Forum for a user is higher than for the average Facebook user, though it’s messy since Facebook has such strong network effects).
Also, 4000 users is an underestimate since the majority of people benefit from the EA Forum while logged out (on LW about 10-20% of our traffic comes from logged-in users, my guess is the EA Forum is similar, but not confident), and even daily users are usually not logged in. So it’s more like $50-$100/user, which honestly seems quite reasonable to me.
No subreddit is free. If there is a great subreddit somewhere, it is probably the primary responsibility of at least one person. You can run things on volunteer labor but that doesn’t make them free. I would recommend against running a crucial piece of infrastructure for a professional community of 10,000+ people on volunteer labor.
You posted this graph
If I understand it correctly, it shows that about 50% of EA forum traffic comes from logged-in users, not 10%-20%.
Also, it seems that EA forum gets about 14,000 views per day. So you spend about $2,000,000/(365*14,000) = $0.4 per view. That’s higher than I would expect.
Note that many of these views might not be productive. For me personally, most of the views are like “I open the frontpage automatically when I want to procrastinate, see that nothing is new & interesting or that I didn’t even want to use the forum, and then close it”. I also sometimes incessantly check if anyone commented or voted on my post or comment, and that sort of behaviour can drive up the view count.
That is definitely relevant data! Looking at the recent dates (and hovering over the exact data at the link where the graphs are from) it looks like its around 60% logged-out, 40% logged in.
I do notice I am surprised by this and kind of want confirmation from the EA Forum team they are not doing some kind of filtering on traffic here. When I compare these numbers naively to the Google Analytics data I have access to for those dates, they seem about 20%-30% too low, and it makes me think there is some filtering going on (though my guess is that 80%-90% logged-out traffic definitely still does not seem representative)
I think you’re comparing costs for EAF to revenue on FB.
Yeah, that seems like the right comparison? Revenue is a proxy for value produced, so if you are arguing about whether something is worth funding philanthropically, revenue seems like the better comparison than costs. Though you can also look at costs, which I expect to not be more than a factor 2 off.
Isn’t a lot of FB’s revenue generated by owning a cookie, and cooperating with other websites to track you across pages? I don’t think it’s fair to count that revenue as generated by the social platform, for these purposes.
Your argument also feels slippery to me in general. Registering that now in case you have a good answer to my specific criticism and the general motte-and-bailey feeling sticks around.
I am not sure what you mean by the first. Facebook makes almost all of its revenue with ads. It also does some stuff to do better ad-targeting, for which it uses cookies and does some cross-site tracking, which I do think drives up profit, though my guess is that isn’t responsible for a large fraction of the revenue (though I might be wrong here).
But that doesn’t feel super relevant here. The primary reason why I brought up FB is to establish a rough order-of-magnitude reference class for what normal costs and revenue numbers are associated with internet platforms for a primarily western educated audience.
My best guess is the EA Forum could probably also finance itself with subscriptions, ads and other monetization strategies at its current burn rate, based on these number, though I would be very surprised if that’s a good idea.
I think the more relevant order of magnitude reference class would be the amount per user Facebook spent on core platform maintenance and moderation (and Facebook has a lot more scaling challenges to solve as well as users to spread costs over, so a better comparator would be the running expenses of a small professional forum)
I don’t think FB revenues are remotely relevant to how much value the forum creates, which may be significantly more per user than Facebook if it positively influences decisions people make about employment, founding charities and allocating large chunks of money to effective causes. But the effectiveness of the use of the forum budget isn’t whether the total value created is more than the total costs of running, it’s decided at the margin by whether going the extra mile with the software and curation actually adds more value.
Or put another way, would people engage differently if the forum was run on stock software by a single sysadmin and some regular posters granted volunteer mod privileges?
Well, I mean it isn’t a perfect comparison, but we know roughly what that world looks like because we have both the LessWrong and OG EA Forum datapoints, and both point towards “the Forum gets on the order of 1/5th the usage” and in the case of LessWrong to “the Forum dies completely”.
I do think it goes better if you have at least one well-paid sysadmin, though I definitely wouldn’t remotely be able to do the job on my own.
As to costs, I’d have to dig further but looking at the net profit margin for Meta as a whole suggests a fairly significant adjustment. Looking at the ratio between cost of revenue and revenue suggests an even larger adjustment, but is probably too aggressive of an adjustment.
If Meta actually spent $200 per user to achieve the revenue associated with Facebook, that would be a poor return on investment indeed (i.e., 0%). So I think comparing its revenue per user figure to the Forum’s cost per user creates too easy of a test for the Forum in assessing the value proposition of its expenditures.
No it fucking isn’t—that’s an approximation of the value it generates advertisers, not users.
Yep, totally, it’s a pretty bad proxy. I think the obvious analogy at least for the EA Forum would be that the organizations who are hiring people from the EA Forum are in a comparable position to advertisers, but it’s not amazing.
Facebook ARPU (average revenue per user) in North America is indeed crazy high
, but I think misleading as for some reason they include revenues from Whatsapp and Instagram, but only count Facebook users as MAU in the denominator.(edit: I think this doesn’t matter that much) Also, they seem to be really good at selling adsIn any case:
I don’t think this is a good measure of value, I don’t think the average user would pay $200/year for Facebook. (I actually think Facebook’s value is plausibly negative for the average user. Some people are paying for tools that limit their use of Facebook/Instagram, but I guess that’s beside the point)
Reddit’s revenue per user in the US is ~$20 / user / year, 10x less
This doesn’t take into account the value of the marginal dollar given to the EA Forum. (Maybe 90% of the value is from the first million/year?)
I’m not sure if $6000 is in the same ballpark as $200(edit: oops, see comment below, the number is $500 not $6000)But I strongly agree with your other points, and mostly I think this is a discussion for donors to the EA Forum, not for users. If someone wants to donate $2M to the EA Forum, I wouldn’t criticize the forum for it or stop using it. It’s not my money.
Users might worry about why someone would donate that much to the Forum, and what kind of influence comes with it, but I think that’s a separate discussion, and I’m personally not that worried about it. (But of course, I’m biased as a volunteer moderator)
I think criticizing CEA for the Forum expenditures is fair game. If an expenditure is low-value, orgs should not be seeking funding for it. Donors always have imperfect information, and the act of seeking funding for an activity conveys the organization’s tacit affirmation that the activity is indeed worth funding. I suppose things would be different if a donor gave an unsolicited $2MM/year gift that could only be used for Forum stuff, but that’s not my understanding of EVF’s finances.
I also think criticizing donors is fair game, despite agreeing that their funds are not our money. First, charitable donations are tax advantaged, so as a practical matter those of us who live in the relevant jurisdiction are affected by the choice to donate to some initiative rather than pay taxes on the associated income. I also think criticizing non-EA charitable donors for their grants is fair game for this reason as well.
Second, certain donations can make other EA’s work more difficult. Suppose a donor really wants to pay all employees at major org X Google-level wages. It’s not our money, and yet such a policy would have real consequences on other orgs and initiatives. Here, I think a pattern of excessive spending on insider-oriented activities, if established, could reasonably be seen as harmful to community values and public perception.
(FWIW, my own view is that spending should be higher than ~$0 but significantly lower than $2MM.)
I think the $500 figure is derived from ($2MM annual spend / 4000 monthly active users). The only work monthly is doing there is helping define who is a user. So I don’t think multiplying the figure by 12 is necessary to provide comparison to Facebook.
That being said, I think there’s an additional reason the $200 Facebook figure is inflated. If we’re trying to compare apples to apples (except for using revenue as an overstated proxy for expenditure), I suggest that we should only consider the fraction of implied expenses associated with the core Facebook experience that is analogous to the Forum. Thus, we shouldn’t consider, e.g., the implied expenditures associated with Facebook’s paid ads function, because the Forum has no real analogous function.
Where does the “$200/user/year” figure come from? They report $68.44 average revenue per user for the US and Canada in their 2023 Q4 report.
ARPU is per quarter. $68.44/quarter or $200/year is really high but:
1. it includes revenues from Instagram and Whatsapp, but only counts Facebook users
2. Facebook is crazy good at selling ads, compared to e.g. Reddit (or afaik anything else)
Thanks, many websites seem to report this without the qualifier “per quarter”, which confused me.
Yeah I had the exact same reaction, I couldn’t believe it was so high but it is
Wild
My understanding is that moderation costs comprise only a small portion of Forum expenditures, so you don’t even need to stipulate volunteer mods to make something close to this argument.
(Also: re Reddit mods, you generally get what you pay for . . . although there are some exceptions)
This is really useful context!