That’s weird. Someone could be downvoting in expectation of upvotes, but I agree that datum makes “someone just doesn’t like these comments” much more likely.
I think downvoting with the expectation of upvotes is very reasonable. It’s probably a better norm than voting based on current karma when only some people follow the vote based on current karma level policy and you care about the steady state karma behaviour.
The main reason the pattern Peter described is weird is that the mystery downvoter, who is drastically outnumbered by mystery upvoters, usually beats all of them to the ballot box.
I would be more favorably inclined toward your theory if it were applied when a few upvotes were already on board. A very early downvote can have a disproportionate effect on a comment’s visibility in some cases, or potentially influence other voters too much in a bandwagon effect of sorts. That’s why, e.g., Reddit has a “contest mode” available, in which comments are randomly sorted and karma not displayed.
I feel quite confused about what empirical evidence from the forum would change your mind. I personally don’t think that the comment folding of weakly negative comments has much effect on future engagement (particularly if it’s a reasonable comment).
I initially thought that if your hypothesis about hiding influencing voting is correct and people regularly do the voting on expected Karma, then I think you should expect to see quite a few “reasonable to you” low negative karma comments (as they have been voted negative and then stayed negative due to hiding low visibility). I don’t think I have really observed this, but maybe you can give an example?
Maybe you’d say that the comments in the position are actually ’band wagon-ed” to be very negative (in which case you should be able to find some reasonable-to-you comments that are very negative, but you also probably shouldn’t believe the low negative karma hiding comments leads to little viewership (you should believe it leads to more viewership).
Maybe you could say, “Well both things are true at different times,” which produces a bimodal distribution—I think that this gets a substantial “complexity penalty”, though it’s plausible—that said, I’d be pretty interested in seeing comments that are reasonable to you ending up negatively downvoted at all after more than 4 days, I think it’s pretty rare.
Could you possibly make a prediction about a karma score on some kind of comment that you expect to see that is much more likely in worlds where you are right relative to worlds where you are wrong?
(I’m sure people will look at this and think we are rabbit-holing about something unimportant, but I think understanding karma dynamics is fairly important, and this isn’t derailing any interesting discussion. afaict)
My view could be disconfirmed by various sorts of A/B testing. The most obvious of which would involve manipulating the amount of karma displayed to viewers shortly after posting. Let’s say +3 of true karma for one group (A), true karma for a second group (B), and and −3 of true karma for the third (C). If my view that early negative votes can have a disproportionate effect is true, then we should see more downvotes and/or fewer upvotes coming from group C than groups A or B. Over a sufficient number of comments, this should be powered enough to reach statistical significance if there is a meaningful effect. This is similar to the methodology used by Muchnik et al. in their study on social influence bias on an unnamed upvoting/downvoting site.
Notably, Muchnik et al. observed significant differences even though their karma manipulation was limited to +1/-1 at the time of karma creation. I suggested +3/-3 here because the types of voters I think likely to employ a preemptive strategy would typically have +2/-2 weight on ordinary votes, and there could be more than one person employing the strategy.
A less direct study design would be to show some users the comments as currently displayed (A), while other users see something akin to Reddit “contest mode” (B: random order, scores hidden). As one might guess, this strategy is used by some subreddits to mitigate the trend of users disproportionately upvoting comments that already had upvotes. For instance, on one major subreddit [link contains a few curse words], the top comment used to be on average posted 4.47 minutes after the post was made. Using contest mode in the first hour increased that to ~11 minutes, and the length of the top comment to almost double. This design couldn’t prove my view, but a finding of no difference between A and B in a sufficiently powered sample would render it rather unlikely.
One might reasonably counter that Forum users are less likely to be swayed by others’ votes than users on a large subreddit. This is probably true, but the social influence bias is well-documented in other contexts as well (e.g., the classic line-drawing study by Asch), and I think that justifies a relatively strong prior that it exists here as well.
I think the absence of quality comments with net negative karma after a few days, and similar non-experimental observations, would be of limited relevance to establishing—in either direction—whether the early negative vote has a disproportionate effect on the ultimate karma level. There’s no way to know what karma a comment would have received if the very early downvote were rendered later in the voting process.
To the extent my reference to “bandwagoning” may have suggested that the end result would be negative as in a Reddit-style bandwagoning, I’ll withdraw that unintended suggestion. I suspect there are enough users who will upvote reasonable-quality comments if in negative karma to avoid this from happening.
Finally, I think it’s not advisable for any user to care solely about the final karma value. Earlier karma values serve two important purposes. First, karma provides feedback to authors about whether the community sees value in their contributions, and thus influences further contributions. Most authors aren’t checking their comment’s karma a week or two after posting. Second, then-current karma influences the order in which posts and comments are displayed—and presumably many people see a comment in the first day or two. Anticipatory downvoting seems clearly negative to me for these purposes, so the expected value of the effect on final karma has to be high enough to overcome those negatives. So, for users who care primarily but not exclusively about final karma, there is still a thumb on the scale here.
We all have colds ahead of planned holiday travel in my household, so I am behind on some things and will leave the last word to you if you wish.
That’s weird. Someone could be downvoting in expectation of upvotes, but I agree that datum makes “someone just doesn’t like these comments” much more likely.
I think downvoting with the expectation of upvotes is very reasonable. It’s probably a better norm than voting based on current karma when only some people follow the vote based on current karma level policy and you care about the steady state karma behaviour.
The main reason the pattern Peter described is weird is that the mystery downvoter, who is drastically outnumbered by mystery upvoters, usually beats all of them to the ballot box.
I would be more favorably inclined toward your theory if it were applied when a few upvotes were already on board. A very early downvote can have a disproportionate effect on a comment’s visibility in some cases, or potentially influence other voters too much in a bandwagon effect of sorts. That’s why, e.g., Reddit has a “contest mode” available, in which comments are randomly sorted and karma not displayed.
I feel quite confused about what empirical evidence from the forum would change your mind. I personally don’t think that the comment folding of weakly negative comments has much effect on future engagement (particularly if it’s a reasonable comment).
I initially thought that if your hypothesis about hiding influencing voting is correct and people regularly do the voting on expected Karma, then I think you should expect to see quite a few “reasonable to you” low negative karma comments (as they have been voted negative and then stayed negative due to hiding low visibility). I don’t think I have really observed this, but maybe you can give an example?
Maybe you’d say that the comments in the position are actually ’band wagon-ed” to be very negative (in which case you should be able to find some reasonable-to-you comments that are very negative, but you also probably shouldn’t believe the low negative karma hiding comments leads to little viewership (you should believe it leads to more viewership).
Maybe you could say, “Well both things are true at different times,” which produces a bimodal distribution—I think that this gets a substantial “complexity penalty”, though it’s plausible—that said, I’d be pretty interested in seeing comments that are reasonable to you ending up negatively downvoted at all after more than 4 days, I think it’s pretty rare.
Could you possibly make a prediction about a karma score on some kind of comment that you expect to see that is much more likely in worlds where you are right relative to worlds where you are wrong?
(I’m sure people will look at this and think we are rabbit-holing about something unimportant, but I think understanding karma dynamics is fairly important, and this isn’t derailing any interesting discussion. afaict)
My view could be disconfirmed by various sorts of A/B testing. The most obvious of which would involve manipulating the amount of karma displayed to viewers shortly after posting. Let’s say +3 of true karma for one group (A), true karma for a second group (B), and and −3 of true karma for the third (C). If my view that early negative votes can have a disproportionate effect is true, then we should see more downvotes and/or fewer upvotes coming from group C than groups A or B. Over a sufficient number of comments, this should be powered enough to reach statistical significance if there is a meaningful effect. This is similar to the methodology used by Muchnik et al. in their study on social influence bias on an unnamed upvoting/downvoting site.
Notably, Muchnik et al. observed significant differences even though their karma manipulation was limited to +1/-1 at the time of karma creation. I suggested +3/-3 here because the types of voters I think likely to employ a preemptive strategy would typically have +2/-2 weight on ordinary votes, and there could be more than one person employing the strategy.
A less direct study design would be to show some users the comments as currently displayed (A), while other users see something akin to Reddit “contest mode” (B: random order, scores hidden). As one might guess, this strategy is used by some subreddits to mitigate the trend of users disproportionately upvoting comments that already had upvotes. For instance, on one major subreddit [link contains a few curse words], the top comment used to be on average posted 4.47 minutes after the post was made. Using contest mode in the first hour increased that to ~11 minutes, and the length of the top comment to almost double. This design couldn’t prove my view, but a finding of no difference between A and B in a sufficiently powered sample would render it rather unlikely.
One might reasonably counter that Forum users are less likely to be swayed by others’ votes than users on a large subreddit. This is probably true, but the social influence bias is well-documented in other contexts as well (e.g., the classic line-drawing study by Asch), and I think that justifies a relatively strong prior that it exists here as well.
I think the absence of quality comments with net negative karma after a few days, and similar non-experimental observations, would be of limited relevance to establishing—in either direction—whether the early negative vote has a disproportionate effect on the ultimate karma level. There’s no way to know what karma a comment would have received if the very early downvote were rendered later in the voting process.
To the extent my reference to “bandwagoning” may have suggested that the end result would be negative as in a Reddit-style bandwagoning, I’ll withdraw that unintended suggestion. I suspect there are enough users who will upvote reasonable-quality comments if in negative karma to avoid this from happening.
Finally, I think it’s not advisable for any user to care solely about the final karma value. Earlier karma values serve two important purposes. First, karma provides feedback to authors about whether the community sees value in their contributions, and thus influences further contributions. Most authors aren’t checking their comment’s karma a week or two after posting. Second, then-current karma influences the order in which posts and comments are displayed—and presumably many people see a comment in the first day or two. Anticipatory downvoting seems clearly negative to me for these purposes, so the expected value of the effect on final karma has to be high enough to overcome those negatives. So, for users who care primarily but not exclusively about final karma, there is still a thumb on the scale here.
We all have colds ahead of planned holiday travel in my household, so I am behind on some things and will leave the last word to you if you wish.