Maybe turn off strong voting in comments or even comment karma from counting to users’ total karma in such posts? How do we decide which posts to consider controversial, though? Just the mods do it (they kept object-level election posts in the personal blog)?
An approach some forum use is the ratio of up and downvotes: −38+40 is not the same as +2 ! This allows you to have a smooth measure of the degree of controversy rather than a binary classification.
One underlying reason your comment got a lot of upvotes was because the post was viewed many times. Controversy leads to pageviews. Arguably “net upvotes” is an OK metric for post quality (where popularity is important) whereas “net upvotes”/”pageviews” might make more sense for comments.
Side-issue: isn’t Karma from posts weighted at 10x compared to Karma in comments? Or at least, I think it once was. And that would help a bit in this particular instance.
We no longer weigh frontpage posts 10x, though we might want to reinstitute some kind of weighing again. I think the 10x was historically too much, and made it so that by far the primary determinant of who had how much karma was how many frontpage posts you had, which felt like it undervalued comments, but it’s pretty plausible (and even likely to me) that the current system is now too skewed in the other direction.
My current relationship towards karma is something like: The point of karma for comments is to provide local information in a thread about a mixture of importance, quality and readership, and it’s pretty hard to disentangle those without making the system much more complex. Overall the karma of a post is a pretty good guess on how many people will want to read it, so it makes sense to use it for some recommendation systems, but the karma of comments feel a lot more noisy to me. As a long-term reward I think we shouldn’t really rely on karma at all and instead use systems like the LessWrong review to establish in a much more considered way which posts were actually good.
We’ve also deemphasized how much karma someone has on the site quite a bit because I don’t want to create the impression that it’s at all a robust measure of the quality of someone’s contributions. So, for example, we no longer have karma leaderboards.
A topic could be controversial in society but the votes could still go mostly one way on the EA Forum itself, though. For example, I wouldn’t be surprised if Democrat-favouring election posts were not scored as very controversial on the EA Forum, given the political leanings of EA. Do we also want to consider posts on controversial topics more broadly?
Maybe turn off strong voting in comments or even comment karma from counting to users’ total karma in such posts? How do we decide which posts to consider controversial, though? Just the mods do it (they kept object-level election posts in the personal blog)?
An approach some forum use is the ratio of up and downvotes: −38+40 is not the same as +2 ! This allows you to have a smooth measure of the degree of controversy rather than a binary classification.
One underlying reason your comment got a lot of upvotes was because the post was viewed many times. Controversy leads to pageviews. Arguably “net upvotes” is an OK metric for post quality (where popularity is important) whereas “net upvotes”/”pageviews” might make more sense for comments.
Side-issue: isn’t Karma from posts weighted at 10x compared to Karma in comments? Or at least, I think it once was. And that would help a bit in this particular instance.
We no longer weigh frontpage posts 10x, though we might want to reinstitute some kind of weighing again. I think the 10x was historically too much, and made it so that by far the primary determinant of who had how much karma was how many frontpage posts you had, which felt like it undervalued comments, but it’s pretty plausible (and even likely to me) that the current system is now too skewed in the other direction.
My current relationship towards karma is something like: The point of karma for comments is to provide local information in a thread about a mixture of importance, quality and readership, and it’s pretty hard to disentangle those without making the system much more complex. Overall the karma of a post is a pretty good guess on how many people will want to read it, so it makes sense to use it for some recommendation systems, but the karma of comments feel a lot more noisy to me. As a long-term reward I think we shouldn’t really rely on karma at all and instead use systems like the LessWrong review to establish in a much more considered way which posts were actually good.
We’ve also deemphasized how much karma someone has on the site quite a bit because I don’t want to create the impression that it’s at all a robust measure of the quality of someone’s contributions. So, for example, we no longer have karma leaderboards.
A topic could be controversial in society but the votes could still go mostly one way on the EA Forum itself, though. For example, I wouldn’t be surprised if Democrat-favouring election posts were not scored as very controversial on the EA Forum, given the political leanings of EA. Do we also want to consider posts on controversial topics more broadly?