I think the proposed karma system, particularly when combined with the highly rated posts being listed higher, is a quite bad idea. In general, if you are trying to ensure quality of posts and comments while spreading the forum out more broadly there are hard tradeoffs with different strengths and weaknesses. Indeed, I might prefer some type of karma weighting system to overly strict moderation but even then the weights proposed here don’t seem justifiable.
What problem is being solved by giving up to 16 times maximum weight that would not be solved with giving users with high karma “merely” a maximum of 2 times the amount of possible weight? 4 times?
However, we obviously don’t want this to become a tyranny of a few users. There are several users, holding very different viewpoints, who currently have high karma on the Forum, and we hope that this will help maintain a varied discussion, while still ensuring that the Forum has strong discussion standards.
While it may be true now that there are multiple users with high karma with very different viewpoints, any imbalance among competing viewpoints at the start of a weighted system could possibly feedback on itself. That is to say, if viewpoint X has 50% of the top posters (by weight in the new system), Y has 30%, and Z 20%, viewpoint Z could easily see their viewpoint shrink relative to the others because the differential voting will compound itself over time.
Thanks for the comments on this Marcus (+ Kyle and others elsewhere).
I certainly appreciate the concern, but I think it’s worth noting that any feedback effects are likely to be minor.
As Larks notes elsewhere, the scoring is quasi-logarithmic — to gain one extra point of voting power (i.e. to have your vote be able to count against that of a single extra brand-new user) is exponentially harder each time.
Assuming that it’s twice as hard to get from one ‘level’ to the next (meaning that each ‘level’ has half the number of users than the preceding one), the average ‘voting power’ across the whole of the forum is only 2 votes. Even if you make the assumption that people at the top of the distribution are proportionally more active on the forum (i.e. a person with 500,000 karma is 16 times as active as a new user), the average voting power is still only ≈3 votes.
Given a random distribution of viewpoints, this means that it would take the forum’s current highest-karma users (≈5,000 karma) 30-50 times as much engagement in the forum to get from their current position to the maximum level. Given that those current karma levels have been accrued over a period of several years, this would entail an extreme step-change in the way people use the forum.
(Obviously this toy model makes some simplifying assumptions, but these shouldn’t change the underlying point, which is that logarithmic growth is slooooooow, and that the difference between a logarithmically-weighted system and the counterfactual 1-point system is minor.)
This means that the extra voting power is a fairly light thumb on the scale. It means that community members who have earned a reputation for consistently providing thoughtful, interesting content can have a slightly greater chance of influencing the ordering of top posts. But the effect is going to be swamped if only a few newer users disagree with that perspective.
The emphasis on can in the preceding sentence is because people shouldn’t be using strong upvotes as their default voting mechanism — the normal-upvote variance will be even lower. However, if we thought this system was truly open to abuse, a very simple way we could mitigate this is to limit the number of strong upvotes you can make in a given period of time.
There’s an intersection here with the community norms we uphold. The EA Forum isn’t supposed to be a place where you unreflectively pursue your viewpoint, or about ‘winning’ a debate; it’s a place to learn, coordinate, exchange ideas, and change your mind about things. To that end, we should be clear that upvotes aren’t meant to signal simple agreement with a viewpoint. I’d expect people to upvote things they disagree with but which are thoughtful and interesting etc. I don’t think for a second that there won’t be some bias towards just upvoting people who agree with you, but I’m hoping that as a community we can ensure that other things will be more influential, like thoughtfulness, usefulness, reasonableness etc.
Finally, I’d also say that the karma system is just one part of the way that posts are made visible. If a particular minority view is underrepresented, but someone writes a thoughtful post in favour of that view, then the moderation team can always promote it to the front page. Whether this seems good to you obviously depends on your faith in the moderation team, but again, given that our community is built on notions like viewpoint diversity and epistemic humility, then the mods should be upholding these norms too.
The problem now is that some people use upvotes to indicate agreement, while others use it to indicate helpfulness (and many, I suspect, use it interchangeably). Having two types of votes clearly separates these two signals. A vote to the right would mean agree, a vote to the left would be disagree. It doesn’t necessarily need to be a sidevote, anothey symbol might be better, but it’s the idea of two types of votes that counts.
Downside: people are unfamiliar with it, and may be complex to implement. Further complicates the dynamics of upvotes that other people have mentioned in this comment section. However, I think it’s fairly straightforward and people will easily pick up on it. Because it won’t be confused with other systems (I don’t know other fora with multiple types of votes), people will easily read the mouse-over text to find out what the votes mean.
Thanks for the input Sam! How about the visibility of down-voted posts? If I recall right, currently if a post has −2 votes, per default it won’t be visible. Now if the first reader of the post is an older member with a lot of voting power, does this mean that they can single-handedly make a new post invisible?
My suggestion here would be to remove the default criterion for which posts are visible, so that per default all posts are visible (irrespective of the downvotes), but that people can select in their settings a threshold of votes a post should have in order to be visible.
My suggestion here would be to remove the default criterion for which posts are visible, so that per default all posts are visible (irrespective of the downvotes), but that people can select in their settings a threshold of votes a post should have in order to be visible.
Our proposal for how this would work is that all posts would be visible on personal blogs, but that posts with a negative karma score wouldn’t show up on the “frontpage” (the default view). People would still be able to see it on the “All posts” view until the post reached −5 karma, and would be able to upvote it back onto the frontpage. Sometimes this might lead to us losing quality posts, but it also helps prevent users seeing very low quality posts (e.g. spam).
I think the proposed karma system, particularly when combined with the highly rated posts being listed higher, is a quite bad idea. In general, if you are trying to ensure quality of posts and comments while spreading the forum out more broadly there are hard tradeoffs with different strengths and weaknesses. Indeed, I might prefer some type of karma weighting system to overly strict moderation but even then the weights proposed here don’t seem justifiable.
What problem is being solved by giving up to 16 times maximum weight that would not be solved with giving users with high karma “merely” a maximum of 2 times the amount of possible weight? 4 times?
While it may be true now that there are multiple users with high karma with very different viewpoints, any imbalance among competing viewpoints at the start of a weighted system could possibly feedback on itself. That is to say, if viewpoint X has 50% of the top posters (by weight in the new system), Y has 30%, and Z 20%, viewpoint Z could easily see their viewpoint shrink relative to the others because the differential voting will compound itself over time.
Thanks for the comments on this Marcus (+ Kyle and others elsewhere).
I certainly appreciate the concern, but I think it’s worth noting that any feedback effects are likely to be minor.
As Larks notes elsewhere, the scoring is quasi-logarithmic — to gain one extra point of voting power (i.e. to have your vote be able to count against that of a single extra brand-new user) is exponentially harder each time.
Assuming that it’s twice as hard to get from one ‘level’ to the next (meaning that each ‘level’ has half the number of users than the preceding one), the average ‘voting power’ across the whole of the forum is only 2 votes. Even if you make the assumption that people at the top of the distribution are proportionally more active on the forum (i.e. a person with 500,000 karma is 16 times as active as a new user), the average voting power is still only ≈3 votes.
Given a random distribution of viewpoints, this means that it would take the forum’s current highest-karma users (≈5,000 karma) 30-50 times as much engagement in the forum to get from their current position to the maximum level. Given that those current karma levels have been accrued over a period of several years, this would entail an extreme step-change in the way people use the forum.
(Obviously this toy model makes some simplifying assumptions, but these shouldn’t change the underlying point, which is that logarithmic growth is slooooooow, and that the difference between a logarithmically-weighted system and the counterfactual 1-point system is minor.)
This means that the extra voting power is a fairly light thumb on the scale. It means that community members who have earned a reputation for consistently providing thoughtful, interesting content can have a slightly greater chance of influencing the ordering of top posts. But the effect is going to be swamped if only a few newer users disagree with that perspective.
The emphasis on can in the preceding sentence is because people shouldn’t be using strong upvotes as their default voting mechanism — the normal-upvote variance will be even lower. However, if we thought this system was truly open to abuse, a very simple way we could mitigate this is to limit the number of strong upvotes you can make in a given period of time.
There’s an intersection here with the community norms we uphold. The EA Forum isn’t supposed to be a place where you unreflectively pursue your viewpoint, or about ‘winning’ a debate; it’s a place to learn, coordinate, exchange ideas, and change your mind about things. To that end, we should be clear that upvotes aren’t meant to signal simple agreement with a viewpoint. I’d expect people to upvote things they disagree with but which are thoughtful and interesting etc. I don’t think for a second that there won’t be some bias towards just upvoting people who agree with you, but I’m hoping that as a community we can ensure that other things will be more influential, like thoughtfulness, usefulness, reasonableness etc.
Finally, I’d also say that the karma system is just one part of the way that posts are made visible. If a particular minority view is underrepresented, but someone writes a thoughtful post in favour of that view, then the moderation team can always promote it to the front page. Whether this seems good to you obviously depends on your faith in the moderation team, but again, given that our community is built on notions like viewpoint diversity and epistemic humility, then the mods should be upholding these norms too.
Speculative feature request: side votes
The problem now is that some people use upvotes to indicate agreement, while others use it to indicate helpfulness (and many, I suspect, use it interchangeably). Having two types of votes clearly separates these two signals. A vote to the right would mean agree, a vote to the left would be disagree. It doesn’t necessarily need to be a sidevote, anothey symbol might be better, but it’s the idea of two types of votes that counts.
Downside: people are unfamiliar with it, and may be complex to implement. Further complicates the dynamics of upvotes that other people have mentioned in this comment section. However, I think it’s fairly straightforward and people will easily pick up on it. Because it won’t be confused with other systems (I don’t know other fora with multiple types of votes), people will easily read the mouse-over text to find out what the votes mean.
Thanks for the input Sam! How about the visibility of down-voted posts? If I recall right, currently if a post has −2 votes, per default it won’t be visible. Now if the first reader of the post is an older member with a lot of voting power, does this mean that they can single-handedly make a new post invisible?
My suggestion here would be to remove the default criterion for which posts are visible, so that per default all posts are visible (irrespective of the downvotes), but that people can select in their settings a threshold of votes a post should have in order to be visible.
Our proposal for how this would work is that all posts would be visible on personal blogs, but that posts with a negative karma score wouldn’t show up on the “frontpage” (the default view). People would still be able to see it on the “All posts” view until the post reached −5 karma, and would be able to upvote it back onto the frontpage. Sometimes this might lead to us losing quality posts, but it also helps prevent users seeing very low quality posts (e.g. spam).