We could give weight to the average vote per comment/âpost, e.g. a factor calculated by adding all weighted votes on someoneâs comments and posts and then dividing by the number of votes on their comments (not the number of comments/âposts, to avoid penalizing comments in threads that arenât really read).
We could also use a prior on this factor, so that users with a small number of highly upvoted things donât get too much power.
I donât fully understand what youâre saying, but my guess is that youâre suggesting we should take a userâs total karma and divide that by votes. I donât understand what it means to âgive weight to thisââdoes the resulting calculation become their strong vote power? I am not being arch, I literally fully donât understand, like Iâm dumb.
I know someone who has some data and studied the forum voting realizations and weak/âstrong upvotes. They are are totally not a nerd, I swear!
Thoughts:
A proximate issue with the idea I think you are proposing is that currently, voting patterns and concentration of voting or strong upvotes differs in a systematic way by the âclass of post/âcommentâ:
There is a class of âworkadayâ comments/âposts that no one finds problematic and gets just regular upvotes on.
Then there is a class of âIâm fighting the War in Heaven for the Lisan al Gaibâ comments/âposts that gets a large amount of strong upvotes. In my opinion, the karma gains here arenât by merit or content.
I had an idea to filter for this (that is sort of exactly the opposite of yours) to downweight the karma of these comments by their environment, to get a âtrue measureâ of the content. Also, the War in Heaven posts have a sort of frenzy to them. Itâs not impossible that giving everyone a 2x â 10x multiplier on their karma might contribute to this frenzy, so moderating this algorithmically seems good.
A deeper issue is that people will be much more risk averse in a system like this that awards them for their average, not total karma.
In my opinion, people are already too risk averse, in a way that prevents confrontation, but at least that leads to a lot of generally positive comments/âposts, which is a good thing.
Now, this sort of gives a new incentive, to aim for zingers or home runs. This seems pretty bad. I actually donât think it is that dangerous/âpowerful in terms of actual karma gain, because as you mentioned, this can be moderated algorithmically. I think itâs more a problem that this can lead to changes in perception or culture shifts.
Now, this sort of gives a new incentive, to aim for zingers or home runs. This seems pretty bad.
Hmm ya, that seems fair. It might generally encourage preaching to a minority of strong supporters who strong upvote, with the rest indifferent and abstaining from voting.
We could have a meta voting system that awards karma or adjusts upvoting power dependent on getting upvotes from different groups of people.
Examples motivating this vision:
If the two of us had a 15 comment long exchange, and we upvoted each other each time, we would gain a lot of karma. I donât think our karma gains should be worth hugely more than say, 4 âoutsideâ people upvoting both of us once for that exchange.
If you receive a strong upvote for the first time, from someone from another âfactionâ, or from a person who normally doesnât upvote you, that should be noted, rewarded and encouraged (but still anonymized[1]).
On the other hand, upvotes from the same group of people who upvoted you 50x in the past, and does it 5x a week, should be attenuated somewhat.
This is a little tricky:
Thereâs some care here, so we arenât getting the opposite problem of encouraging arbitrary upvotes from random people. We probably want to filter to get substantial opinions. (At the risk of being ideological, I think âdiversityâ, which is the idea here, is valuable but needs to be thoughtfully designed and not done blindly.)
Some of the ideas, like solving for âfactionsâ, involves âthe vote graphâ (a literal mathematical object, analogous to the friend graph on FB but for votes). This requires literal graph theory, e.g. I would probably consult a computer scientist or applied math lady.
I could also see using the view graph as useful.
This isnât exactly the same as your idea, but a lot of your idea can be folded in (while less direct, thereâs several ways we can bake in something like âpeople who get strong upvotes consistently are more rewardedâ).
Maybe there is a tension between notifying people of upvoting diversity, and giving away identity, which might be one reason why some actual graph theory probably needs to be used.
I think this is an interesting idea. I would probably recommend against trying to âgroupâ users, because it would be messy and hard to understand, and I am just generally worried about grouping users and treating them differently based on their groups. Just weakening your upvotes on users you often upvote seems practical and easy to understand.
Would minority views/âinterests become less visible this way, though?
I agree. Thereâs issues. It seems comple and not transparent at first[1]
I would probably recommend against trying to âgroupâ users, because it would be messy and hard to understand, and I am just generally worried about grouping users and treating them differently based on their groups. Just weakening your upvotes on users you often upvote seems practical and easy to understand.
Would minority views/âinterests become less visible this way, though?
I think the goal of using the graphs and building âfactionsâ (âfactionâ is not the best word I would use here, like if you were to put this on a grant proposal) is that it makes it visible and legible.
This might be more general and useful than it seems and can be used prosocially.
For example, once legible, you could identify minority views and give them credence (or just make this a âsliderâ for people to examine).
Like you said, this is hard to execute. I think this is hard in the sense that the designer needs to find the right patterns for both socialization and practical use. Once found, the patterns can ultimately can be relatively simple and transparent.
Misc comments:
To be clear, I think the ideas Iâm discussing in this post and reforms to the forum is at least a major project, up to a major set of useful interventions, maybe comparable to all of âprediction marketsâ (in the sense of the diversity of different projects it could support, investment, and potential impact that would justify it).
This isnât something I am actively pursuing (but these forum discussions are interesting and hard to resist).
Yes, I think thatâs right, I am guessing because both involve graph theory (and I guess in pagerank using the edges turns it into a linear algebra problem too?).
Note that I hardly know any more about graph theory.
PageRank mostly involved graph theory in the mere observation that thereâs a directed graph of pages linking to each other. It then immediately turns to linear algebra, where the idea is that you want a pageâs weight to correspond to the sum of the weights of the pages linking to itâand this exactly describes finding an eigenvector of the graph matrix.
On second thought I guess your idea for karma is more complicated, maybe Iâll look at some simple examples and see what comes up if I happen to have the time.
Thatâs interesting to know about pagerank. Itâs smart it just goes to linear algebra.
I think building the graph requires data that isnât publicly available like identity of votes and views. It might be hard to get a similar dataset to see if a method works or not. Some of the âclustering techniquesâ might not apply to other data.
I donât fully understand what youâre saying, but my guess is that youâre saying is like, that a userâs total karma and divide that by votes.
Ya, thatâs right, total karma divided by the number of votes.
I donât understand what it means to âgive weight to thisââdoes the resulting calculation become their strong vote power?
What I proposed could be what determines strong vote power alone, but strong vote power could be based on a combination of multiple things. What I meant by âgive weight to thisâ is that it could just be one of multiple determinants of strong vote power.
A deeper issue is that people will be much more risk averse in a system like this that awards them for their average, not total karma.
This is why I was thinking it would only be one factor. We could use both their total and average to determine the strong vote power. But also, maybe a bit of risk-aversion is good? The EA Forum is getting used more, so it would be good if the comments and posts people made were high quality rather than noise.
Also, again, it wouldnât penalize users for making comments that donât get voted on; it encourages them to chase strong upvotes and avoid downvotes (relative to regular upvotes or no votes).
I did have thought about a meta karma system, but where the voting effect or output differs by context (so a strong upvote or strong downvote has a different effect on comments or posts depending on the situation).
This would avoid the current situation of âI need to make sure the comment I like ranks highly or has the karma I wantâ and also prevent certain kinds of hysteresis.
While the voting effect or output can change radically in this system, based on context, IMO, itâs important make sure that the latent underlying voting power of the user accumulates in a fairly mundane/âsimple/âgradual way, otherwise you get these weird incentives.
Maybe this underweights many people appreciating a given comment or post and regular upvoting it, but few of them appreciating it enough to strong upvote it, relative to a post or comment getting both a few strong upvotes and at most a few regular votes?
We could give weight to the average vote per comment/âpost, e.g. a factor calculated by adding all weighted votes on someoneâs comments and posts and then dividing by the number of votes on their comments (not the number of comments/âposts, to avoid penalizing comments in threads that arenât really read).
We could also use a prior on this factor, so that users with a small number of highly upvoted things donât get too much power.
I donât fully understand what youâre saying, but my guess is that youâre suggesting we should take a userâs total karma and divide that by votes. I donât understand what it means to âgive weight to thisââdoes the resulting calculation become their strong vote power? I am not being arch, I literally fully donât understand, like Iâm dumb.
I know someone who has some data and studied the forum voting realizations and weak/âstrong upvotes. They are are totally not a nerd, I swear!
Thoughts:
A proximate issue with the idea I think you are proposing is that currently, voting patterns and concentration of voting or strong upvotes differs in a systematic way by the âclass of post/âcommentâ:
There is a class of âworkadayâ comments/âposts that no one finds problematic and gets just regular upvotes on.
Then there is a class of âIâm fighting the War in Heaven for the Lisan al Gaibâ comments/âposts that gets a large amount of strong upvotes. In my opinion, the karma gains here arenât by merit or content.
I had an idea to filter for this (that is sort of exactly the opposite of yours) to downweight the karma of these comments by their environment, to get a âtrue measureâ of the content. Also, the War in Heaven posts have a sort of frenzy to them. Itâs not impossible that giving everyone a 2x â 10x multiplier on their karma might contribute to this frenzy, so moderating this algorithmically seems good.
A deeper issue is that people will be much more risk averse in a system like this that awards them for their average, not total karma.
In my opinion, people are already too risk averse, in a way that prevents confrontation, but at least that leads to a lot of generally positive comments/âposts, which is a good thing.
Now, this sort of gives a new incentive, to aim for zingers or home runs. This seems pretty bad. I actually donât think it is that dangerous/âpowerful in terms of actual karma gain, because as you mentioned, this can be moderated algorithmically. I think itâs more a problem that this can lead to changes in perception or culture shifts.
Hmm ya, that seems fair. It might generally encourage preaching to a minority of strong supporters who strong upvote, with the rest indifferent and abstaining from voting.
This is a good point itself.
This raises a new, pretty sanguine idea:
We could have a meta voting system that awards karma or adjusts upvoting power dependent on getting upvotes from different groups of people.
Examples motivating this vision:
If the two of us had a 15 comment long exchange, and we upvoted each other each time, we would gain a lot of karma. I donât think our karma gains should be worth hugely more than say, 4 âoutsideâ people upvoting both of us once for that exchange.
If you receive a strong upvote for the first time, from someone from another âfactionâ, or from a person who normally doesnât upvote you, that should be noted, rewarded and encouraged (but still anonymized[1]).
On the other hand, upvotes from the same group of people who upvoted you 50x in the past, and does it 5x a week, should be attenuated somewhat.
This is a little tricky:
Thereâs some care here, so we arenât getting the opposite problem of encouraging arbitrary upvotes from random people. We probably want to filter to get substantial opinions. (At the risk of being ideological, I think âdiversityâ, which is the idea here, is valuable but needs to be thoughtfully designed and not done blindly.)
Some of the ideas, like solving for âfactionsâ, involves âthe vote graphâ (a literal mathematical object, analogous to the friend graph on FB but for votes). This requires literal graph theory, e.g. I would probably consult a computer scientist or applied math lady.
I could also see using the view graph as useful.
This isnât exactly the same as your idea, but a lot of your idea can be folded in (while less direct, thereâs several ways we can bake in something like âpeople who get strong upvotes consistently are more rewardedâ).
Maybe there is a tension between notifying people of upvoting diversity, and giving away identity, which might be one reason why some actual graph theory probably needs to be used.
I think this is an interesting idea. I would probably recommend against trying to âgroupâ users, because it would be messy and hard to understand, and I am just generally worried about grouping users and treating them differently based on their groups. Just weakening your upvotes on users you often upvote seems practical and easy to understand.
Would minority views/âinterests become less visible this way, though?
I agree. Thereâs issues. It seems comple and not transparent at first[1]
I think the goal of using the graphs and building âfactionsâ (âfactionâ is not the best word I would use here, like if you were to put this on a grant proposal) is that it makes it visible and legible.
This might be more general and useful than it seems and can be used prosocially.
For example, once legible, you could identify minority views and give them credence (or just make this a âsliderâ for people to examine).
Like you said, this is hard to execute. I think this is hard in the sense that the designer needs to find the right patterns for both socialization and practical use. Once found, the patterns can ultimately can be relatively simple and transparent.
Misc comments:
To be clear, I think the ideas Iâm discussing in this post and reforms to the forum is at least a major project, up to a major set of useful interventions, maybe comparable to all of âprediction marketsâ (in the sense of the diversity of different projects it could support, investment, and potential impact that would justify it).
This isnât something I am actively pursuing (but these forum discussions are interesting and hard to resist).
This sounds somewhat like a search-engine eigenvector thing, e.g. PageRank.
Yes, I think thatâs right, I am guessing because both involve graph theory (and I guess in pagerank using the edges turns it into a linear algebra problem too?).
Note that I hardly know any more about graph theory.
PageRank mostly involved graph theory in the mere observation that thereâs a directed graph of pages linking to each other. It then immediately turns to linear algebra, where the idea is that you want a pageâs weight to correspond to the sum of the weights of the pages linking to itâand this exactly describes finding an eigenvector of the graph matrix.
On second thought I guess your idea for karma is more complicated, maybe Iâll look at some simple examples and see what comes up if I happen to have the time.
Thatâs interesting to know about pagerank. Itâs smart it just goes to linear algebra.
I think building the graph requires data that isnât publicly available like identity of votes and views. It might be hard to get a similar dataset to see if a method works or not. Some of the âclustering techniquesâ might not apply to other data.
Maybe there is a literature for this already.
Ya, thatâs right, total karma divided by the number of votes.
What I proposed could be what determines strong vote power alone, but strong vote power could be based on a combination of multiple things. What I meant by âgive weight to thisâ is that it could just be one of multiple determinants of strong vote power.
This is why I was thinking it would only be one factor. We could use both their total and average to determine the strong vote power. But also, maybe a bit of risk-aversion is good? The EA Forum is getting used more, so it would be good if the comments and posts people made were high quality rather than noise.
Also, again, it wouldnât penalize users for making comments that donât get voted on; it encourages them to chase strong upvotes and avoid downvotes (relative to regular upvotes or no votes).
I did have thought about a meta karma system, but where the voting effect or output differs by context (so a strong upvote or strong downvote has a different effect on comments or posts depending on the situation).
This would avoid the current situation of âI need to make sure the comment I like ranks highly or has the karma I wantâ and also prevent certain kinds of hysteresis.
While the voting effect or output can change radically in this system, based on context, IMO, itâs important make sure that the latent underlying voting power of the user accumulates in a fairly mundane/âsimple/âgradual way, otherwise you get these weird incentives.
Maybe this underweights many people appreciating a given comment or post and regular upvoting it, but few of them appreciating it enough to strong upvote it, relative to a post or comment getting both a few strong upvotes and at most a few regular votes?