Should the EA Forum team stop optimizing for engagement? I heard that the EA forum team tries to optimize the forum for engagement (tests features to see if they improve engagement). There are positives to this, but on net it worries me. Taken to the extreme, this is a destructive practice, as it would
normalize and encourage clickbait;
cause thoughtful comments to be replaced by louder and more abundant voices (for a constant time spent thinking, you can post either 1 thoughtful comment or several hasty comments. Measuring session length fixes this but adds more problems);
cause people with important jobs to spend more time on EA Forum than is optimal;
avoid community members and “EA” itself from keeping their identities small, as politics is an endless source of engagement;
distract from other possible directions of improvement, like giving topics proportionate attention, adding epistemic technology like polls and prediction market integration, improving moderation, and generally increasing quality of discussion.
I’m not confident that EA Forum is getting worse, or that tracking engagement is currently net negative, but we should at least avoid failing this exercise in Goodhart’s Law.
Thanks for this shortform! I’d like to quickly clarify a bit about our strategy. TL;DR: I don’t think the Forum team optimizes for engagement.
We do track engagement, and engagement is important to us, since we think a lot of the ways in which the Forum has an impact are diffuse or hard to measure, and they’d roughly grow or diminish with engagement.
But we definitely don’t optimize for it, and we’re very aware of worries about Goodharting.
Besides engagement, we try to track estimates for a number of other things we care about (like how good the discussions have been, how many people have gotten jobs as a result of the Forum, etc), and we’re actively working on doing that more carefully.
And for what it’s worth, I think that none of our major projects in the near future (like developing subforums) are aimed at increasing engagement, and neither have been our recent projects (like promoting impactful jobs).
And for what it’s worth, I think that none of our major projects in the near future (like developing subforums) are aimed at increasing engagement, and neither have been our recent projects (like promoting impactful jobs).
I wasn’t counting that as a major project, but Draft Amnesty Day also wasn’t aimed at optimizing engagement (and I’d be surprised[1] if it helped or hurt engagement in a significant way). That was motivated by a desire to get people to publish drafts (which could have cool ideas) that they’ve been sitting on for a while. :)
Should the EA Forum team stop optimizing for engagement?
I heard that the EA forum team tries to optimize the forum for engagement (tests features to see if they improve engagement). There are positives to this, but on net it worries me. Taken to the extreme, this is a destructive practice, as it would
normalize and encourage clickbait;
cause thoughtful comments to be replaced by louder and more abundant voices (for a constant time spent thinking, you can post either 1 thoughtful comment or several hasty comments. Measuring session length fixes this but adds more problems);
cause people with important jobs to spend more time on EA Forum than is optimal;
avoid community members and “EA” itself from keeping their identities small, as politics is an endless source of engagement;
distract from other possible directions of improvement, like giving topics proportionate attention, adding epistemic technology like polls and prediction market integration, improving moderation, and generally increasing quality of discussion.
I’m not confident that EA Forum is getting worse, or that tracking engagement is currently net negative, but we should at least avoid failing this exercise in Goodhart’s Law.
Thanks for this shortform! I’d like to quickly clarify a bit about our strategy. TL;DR: I don’t think the Forum team optimizes for engagement.
We do track engagement, and engagement is important to us, since we think a lot of the ways in which the Forum has an impact are diffuse or hard to measure, and they’d roughly grow or diminish with engagement.
But we definitely don’t optimize for it, and we’re very aware of worries about Goodharting.
Besides engagement, we try to track estimates for a number of other things we care about (like how good the discussions have been, how many people have gotten jobs as a result of the Forum, etc), and we’re actively working on doing that more carefully.
And for what it’s worth, I think that none of our major projects in the near future (like developing subforums) are aimed at increasing engagement, and neither have been our recent projects (like promoting impactful jobs).
What about Draft Amnesty?
I wasn’t counting that as a major project, but Draft Amnesty Day also wasn’t aimed at optimizing engagement (and I’d be surprised[1] if it helped or hurt engagement in a significant way). That was motivated by a desire to get people to publish drafts (which could have cool ideas) that they’ve been sitting on for a while. :)
Edit: can confirm that at a glance, engagement on Friday/this weekend looks normal