Something which has come up a few times, and recently a lot in the context of Debate Week (and the reaction to Leif’s post) is things getting downvoted quickly and being removed from the Front Page, which drastically drops the likelihood of engagement.[1]
So a potential suggestion for the Frontpage might be:
Hide the vote score of all new posts if the absolute score of the post is below some threshold (I’ll use 20 as an example)
If a post hits −20, it drops off the front page
After a post hits 20+, it’s karma score is permanently revealed
Galaxy-brain version is that Community/Non-Community grouping should only take effect once a post hits these thresholds[2]
This will still probably leave us with too many new posts to fit on the front page, so some rules to sort which stay and which get knocked off:
Some consideration to total karma should probably count (how much to weight it is debatable)
Some consideration to how recent the post is should count too (e.g. I’d probably want to see a new post that got 20+ karma quickly than 100+ karma over weeks)
Some consideration should also go to engagement—some metric related to either number of votes or comment countwould probably indicate which posts are generating community engagement, though this could lead to bikeshedding/Matthew effect if not implemented correctly. I still think it’s directionally correct though
Of course the user’s own personal weighting of topic importance can probably contribute to this as well
There will always be some trade-offs when designing some ranking on many posts with limited space. But the idea above is that no post should quickly drop off the front page because a few people quickly down-vote it into negative karma.
Maybe some code like this already exists, but this thought popped into my head and I thought it was worth sharing on this post.
In a couple of places I’ve seen people complain about the use of the Community tag to ‘hide’ particular discussions/topics. Not saying I fully endorse this view.
This is a problem we have thought about, and personally I think it is quite bad, in that it causes a lot of randomness in which posts get past ~10 karma. We agree that essentially showing new posts to more people is the answer (even if they start to get downvoted).
The fairly standard solution in recommendation algorithms is to view this as an explore/exploit problem, and to add some randomness to tune the tradeoff between then (see Bandits for Recommender Systems). This would mean each user would get a slightly different ordering of the list (stable per user) in order to make sure each post gets an appropriate number of views (based on the quality signal we have so far).
Here’s an internal doc with our plans on this. We may not get to it that soon, because of giving season and other stuff, but it is one of our priorities after that.
Hide the vote score of all new posts
I’m not that optimistic about this. For algorithm purposes the thing that matters is the number of opportunities a post gets to be voted on (~views) before dropping off the frontpage. Presumably the theory here is that seeing post has low karma makes you less likely to view it and/or biased towards also downvoting it.
Based on my own experience I don’t think this is the case. If I see a 0 to negative karma post on the frontpage I think I’m more likely to click it out of morbid curiosity, and then if it’s at least ok I’ll upvote it because I think the current score is overly harsh. I know a lot of people apply this logic about voting (correcting the current score rather than applying a fully independent judgement), and we generally endorse this.
Some consideration should also go to engagement—some metric related to either number of votes or comment count
I think this is a good idea (thinking about comment counts specifically), although I agree it could be prone to going wrong, e.g. making doom threads even doomier. One solution I can think of is to give a boost for comments but only up to a fairly low cutoff (say, 6 comments). Thoughts on this?
For a ~20 karma post I think having 5+ comments is a good signal that its a valuable-but-niche post.
Digression but I would recommend reading about Thompson sampling :) (wikipedia, inscrutable LessWrong post). It’s a good model to have for thinking about explore-exploit tradeoffs in general.
Some consideration should also go to engagement—some metric related to either number of votes or comment countwould probably indicate which posts are generating community engagement, though this could lead to bikeshedding/Matthew effect if not implemented correctly. I still think it’s directionally correct though
I think number and weight of upvotes (not netted against downvotes) is an important criterion here, especially when it comes to the risk of controversial material getting buried before most users have a chance to see it. I think this may be practically much the same as what you’re suggesting.
If something has a good number of upvotes and downvotes, my assumption is that we ideally want to present that content to the user and let them make their own decision on whether it is worth reading / engaging with. In other words, conditioned on there being a critical mass of upvotes, the presence of the downvotes doesn’t update the probability of “this is worth showing to other users and letting them make their own decision” very much for me.
If something has had enough impressions on the front page and hasn’t gotten much engagement, then the odds of future users wanting to engage with it seems fairly low.
Something which has come up a few times, and recently a lot in the context of Debate Week (and the reaction to Leif’s post) is things getting downvoted quickly and being removed from the Front Page, which drastically drops the likelihood of engagement.[1]
So a potential suggestion for the Frontpage might be:
Hide the vote score of all new posts if the absolute score of the post is below some threshold (I’ll use 20 as an example)
If a post hits −20, it drops off the front page
After a post hits 20+, it’s karma score is permanently revealed
Galaxy-brain version is that Community/Non-Community grouping should only take effect once a post hits these thresholds[2]
This will still probably leave us with too many new posts to fit on the front page, so some rules to sort which stay and which get knocked off:
Some consideration to total karma should probably count (how much to weight it is debatable)
Some consideration to how recent the post is should count too (e.g. I’d probably want to see a new post that got 20+ karma quickly than 100+ karma over weeks)
Some consideration should also go to engagement—some metric related to either number of votes or comment count would probably indicate which posts are generating community engagement, though this could lead to bikeshedding/Matthew effect if not implemented correctly. I still think it’s directionally correct though
Of course the user’s own personal weighting of topic importance can probably contribute to this as well
There will always be some trade-offs when designing some ranking on many posts with limited space. But the idea above is that no post should quickly drop off the front page because a few people quickly down-vote it into negative karma.
Maybe some code like this already exists, but this thought popped into my head and I thought it was worth sharing on this post.
My poor little piece on gradient descent got wiped out by debate week 😭 rip
In a couple of places I’ve seen people complain about the use of the Community tag to ‘hide’ particular discussions/topics. Not saying I fully endorse this view.
This is a problem we have thought about, and personally I think it is quite bad, in that it causes a lot of randomness in which posts get past ~10 karma. We agree that essentially showing new posts to more people is the answer (even if they start to get downvoted).
The fairly standard solution in recommendation algorithms is to view this as an explore/exploit problem, and to add some randomness to tune the tradeoff between then (see Bandits for Recommender Systems). This would mean each user would get a slightly different ordering of the list (stable per user) in order to make sure each post gets an appropriate number of views (based on the quality signal we have so far).
Here’s an internal doc with our plans on this. We may not get to it that soon, because of giving season and other stuff, but it is one of our priorities after that.
I’m not that optimistic about this. For algorithm purposes the thing that matters is the number of opportunities a post gets to be voted on (~views) before dropping off the frontpage. Presumably the theory here is that seeing post has low karma makes you less likely to view it and/or biased towards also downvoting it.
Based on my own experience I don’t think this is the case. If I see a 0 to negative karma post on the frontpage I think I’m more likely to click it out of morbid curiosity, and then if it’s at least ok I’ll upvote it because I think the current score is overly harsh. I know a lot of people apply this logic about voting (correcting the current score rather than applying a fully independent judgement), and we generally endorse this.
I think this is a good idea (thinking about comment counts specifically), although I agree it could be prone to going wrong, e.g. making doom threads even doomier. One solution I can think of is to give a boost for comments but only up to a fairly low cutoff (say, 6 comments). Thoughts on this?
For a ~20 karma post I think having 5+ comments is a good signal that its a valuable-but-niche post.
Digression but I would recommend reading about Thompson sampling :) (wikipedia, inscrutable LessWrong post). It’s a good model to have for thinking about explore-exploit tradeoffs in general.
I think number and weight of upvotes (not netted against downvotes) is an important criterion here, especially when it comes to the risk of controversial material getting buried before most users have a chance to see it. I think this may be practically much the same as what you’re suggesting.
If something has a good number of upvotes and downvotes, my assumption is that we ideally want to present that content to the user and let them make their own decision on whether it is worth reading / engaging with. In other words, conditioned on there being a critical mass of upvotes, the presence of the downvotes doesn’t update the probability of “this is worth showing to other users and letting them make their own decision” very much for me.
If something has had enough impressions on the front page and hasn’t gotten much engagement, then the odds of future users wanting to engage with it seems fairly low.