We could give weight to the average vote per comment/post, e.g. a factor calculated by adding all weighted votes on someone’s comments and posts and then dividing by the number of votes on their comments (not the number of comments/posts, to avoid penalizing comments in threads that aren’t really read).
We could also use a prior on this factor, so that users with a small number of highly upvoted things don’t get too much power.
I don’t fully understand what you’re saying, but my guess is that you’re suggesting we should take a user’s total karma and divide that by votes. I don’t understand what it means to “give weight to this”—does the resulting calculation become their strong vote power? I am not being arch, I literally fully don’t understand, like I’m dumb.
I know someone who has some data and studied the forum voting realizations and weak/strong upvotes. They are are totally not a nerd, I swear!
Thoughts:
A proximate issue with the idea I think you are proposing is that currently, voting patterns and concentration of voting or strong upvotes differs in a systematic way by the “class of post/comment”:
There is a class of “workaday” comments/posts that no one finds problematic and gets just regular upvotes on.
Then there is a class of “I’m fighting the War in Heaven for the Lisan al Gaib” comments/posts that gets a large amount of strong upvotes. In my opinion, the karma gains here aren’t by merit or content.
I had an idea to filter for this (that is sort of exactly the opposite of yours) to downweight the karma of these comments by their environment, to get a “true measure” of the content. Also, the War in Heaven posts have a sort of frenzy to them. It’s not impossible that giving everyone a 2x − 10x multiplier on their karma might contribute to this frenzy, so moderating this algorithmically seems good.
A deeper issue is that people will be much more risk averse in a system like this that awards them for their average, not total karma.
In my opinion, people are already too risk averse, in a way that prevents confrontation, but at least that leads to a lot of generally positive comments/posts, which is a good thing.
Now, this sort of gives a new incentive, to aim for zingers or home runs. This seems pretty bad. I actually don’t think it is that dangerous/powerful in terms of actual karma gain, because as you mentioned, this can be moderated algorithmically. I think it’s more a problem that this can lead to changes in perception or culture shifts.
Now, this sort of gives a new incentive, to aim for zingers or home runs. This seems pretty bad.
Hmm ya, that seems fair. It might generally encourage preaching to a minority of strong supporters who strong upvote, with the rest indifferent and abstaining from voting.
We could have a meta voting system that awards karma or adjusts upvoting power dependent on getting upvotes from different groups of people.
Examples motivating this vision:
If the two of us had a 15 comment long exchange, and we upvoted each other each time, we would gain a lot of karma. I don’t think our karma gains should be worth hugely more than say, 4 “outside” people upvoting both of us once for that exchange.
If you receive a strong upvote for the first time, from someone from another “faction”, or from a person who normally doesn’t upvote you, that should be noted, rewarded and encouraged (but still anonymized[1]).
On the other hand, upvotes from the same group of people who upvoted you 50x in the past, and does it 5x a week, should be attenuated somewhat.
This is a little tricky:
There’s some care here, so we aren’t getting the opposite problem of encouraging arbitrary upvotes from random people. We probably want to filter to get substantial opinions. (At the risk of being ideological, I think “diversity”, which is the idea here, is valuable but needs to be thoughtfully designed and not done blindly.)
Some of the ideas, like solving for “factions”, involves “the vote graph” (a literal mathematical object, analogous to the friend graph on FB but for votes). This requires literal graph theory, e.g. I would probably consult a computer scientist or applied math lady.
I could also see using the view graph as useful.
This isn’t exactly the same as your idea, but a lot of your idea can be folded in (while less direct, there’s several ways we can bake in something like “people who get strong upvotes consistently are more rewarded”).
Maybe there is a tension between notifying people of upvoting diversity, and giving away identity, which might be one reason why some actual graph theory probably needs to be used.
I think this is an interesting idea. I would probably recommend against trying to “group” users, because it would be messy and hard to understand, and I am just generally worried about grouping users and treating them differently based on their groups. Just weakening your upvotes on users you often upvote seems practical and easy to understand.
Would minority views/interests become less visible this way, though?
I agree. There’s issues. It seems comple and not transparent at first[1]
I would probably recommend against trying to “group” users, because it would be messy and hard to understand, and I am just generally worried about grouping users and treating them differently based on their groups. Just weakening your upvotes on users you often upvote seems practical and easy to understand.
Would minority views/interests become less visible this way, though?
I think the goal of using the graphs and building “factions” (“faction” is not the best word I would use here, like if you were to put this on a grant proposal) is that it makes it visible and legible.
This might be more general and useful than it seems and can be used prosocially.
For example, once legible, you could identify minority views and give them credence (or just make this a “slider” for people to examine).
Like you said, this is hard to execute. I think this is hard in the sense that the designer needs to find the right patterns for both socialization and practical use. Once found, the patterns can ultimately can be relatively simple and transparent.
Misc comments:
To be clear, I think the ideas I’m discussing in this post and reforms to the forum is at least a major project, up to a major set of useful interventions, maybe comparable to all of “prediction markets” (in the sense of the diversity of different projects it could support, investment, and potential impact that would justify it).
This isn’t something I am actively pursuing (but these forum discussions are interesting and hard to resist).
Yes, I think that’s right, I am guessing because both involve graph theory (and I guess in pagerank using the edges turns it into a linear algebra problem too?).
Note that I hardly know any more about graph theory.
PageRank mostly involved graph theory in the mere observation that there’s a directed graph of pages linking to each other. It then immediately turns to linear algebra, where the idea is that you want a page’s weight to correspond to the sum of the weights of the pages linking to it—and this exactly describes finding an eigenvector of the graph matrix.
On second thought I guess your idea for karma is more complicated, maybe I’ll look at some simple examples and see what comes up if I happen to have the time.
That’s interesting to know about pagerank. It’s smart it just goes to linear algebra.
I think building the graph requires data that isn’t publicly available like identity of votes and views. It might be hard to get a similar dataset to see if a method works or not. Some of the “clustering techniques” might not apply to other data.
I don’t fully understand what you’re saying, but my guess is that you’re saying is like, that a user’s total karma and divide that by votes.
Ya, that’s right, total karma divided by the number of votes.
I don’t understand what it means to “give weight to this”—does the resulting calculation become their strong vote power?
What I proposed could be what determines strong vote power alone, but strong vote power could be based on a combination of multiple things. What I meant by “give weight to this” is that it could just be one of multiple determinants of strong vote power.
A deeper issue is that people will be much more risk averse in a system like this that awards them for their average, not total karma.
This is why I was thinking it would only be one factor. We could use both their total and average to determine the strong vote power. But also, maybe a bit of risk-aversion is good? The EA Forum is getting used more, so it would be good if the comments and posts people made were high quality rather than noise.
Also, again, it wouldn’t penalize users for making comments that don’t get voted on; it encourages them to chase strong upvotes and avoid downvotes (relative to regular upvotes or no votes).
I did have thought about a meta karma system, but where the voting effect or output differs by context (so a strong upvote or strong downvote has a different effect on comments or posts depending on the situation).
This would avoid the current situation of “I need to make sure the comment I like ranks highly or has the karma I want” and also prevent certain kinds of hysteresis.
While the voting effect or output can change radically in this system, based on context, IMO, it’s important make sure that the latent underlying voting power of the user accumulates in a fairly mundane/simple/gradual way, otherwise you get these weird incentives.
Maybe this underweights many people appreciating a given comment or post and regular upvoting it, but few of them appreciating it enough to strong upvote it, relative to a post or comment getting both a few strong upvotes and at most a few regular votes?
We could give weight to the average vote per comment/post, e.g. a factor calculated by adding all weighted votes on someone’s comments and posts and then dividing by the number of votes on their comments (not the number of comments/posts, to avoid penalizing comments in threads that aren’t really read).
We could also use a prior on this factor, so that users with a small number of highly upvoted things don’t get too much power.
I don’t fully understand what you’re saying, but my guess is that you’re suggesting we should take a user’s total karma and divide that by votes. I don’t understand what it means to “give weight to this”—does the resulting calculation become their strong vote power? I am not being arch, I literally fully don’t understand, like I’m dumb.
I know someone who has some data and studied the forum voting realizations and weak/strong upvotes. They are are totally not a nerd, I swear!
Thoughts:
A proximate issue with the idea I think you are proposing is that currently, voting patterns and concentration of voting or strong upvotes differs in a systematic way by the “class of post/comment”:
There is a class of “workaday” comments/posts that no one finds problematic and gets just regular upvotes on.
Then there is a class of “I’m fighting the War in Heaven for the Lisan al Gaib” comments/posts that gets a large amount of strong upvotes. In my opinion, the karma gains here aren’t by merit or content.
I had an idea to filter for this (that is sort of exactly the opposite of yours) to downweight the karma of these comments by their environment, to get a “true measure” of the content. Also, the War in Heaven posts have a sort of frenzy to them. It’s not impossible that giving everyone a 2x − 10x multiplier on their karma might contribute to this frenzy, so moderating this algorithmically seems good.
A deeper issue is that people will be much more risk averse in a system like this that awards them for their average, not total karma.
In my opinion, people are already too risk averse, in a way that prevents confrontation, but at least that leads to a lot of generally positive comments/posts, which is a good thing.
Now, this sort of gives a new incentive, to aim for zingers or home runs. This seems pretty bad. I actually don’t think it is that dangerous/powerful in terms of actual karma gain, because as you mentioned, this can be moderated algorithmically. I think it’s more a problem that this can lead to changes in perception or culture shifts.
Hmm ya, that seems fair. It might generally encourage preaching to a minority of strong supporters who strong upvote, with the rest indifferent and abstaining from voting.
This is a good point itself.
This raises a new, pretty sanguine idea:
We could have a meta voting system that awards karma or adjusts upvoting power dependent on getting upvotes from different groups of people.
Examples motivating this vision:
If the two of us had a 15 comment long exchange, and we upvoted each other each time, we would gain a lot of karma. I don’t think our karma gains should be worth hugely more than say, 4 “outside” people upvoting both of us once for that exchange.
If you receive a strong upvote for the first time, from someone from another “faction”, or from a person who normally doesn’t upvote you, that should be noted, rewarded and encouraged (but still anonymized[1]).
On the other hand, upvotes from the same group of people who upvoted you 50x in the past, and does it 5x a week, should be attenuated somewhat.
This is a little tricky:
There’s some care here, so we aren’t getting the opposite problem of encouraging arbitrary upvotes from random people. We probably want to filter to get substantial opinions. (At the risk of being ideological, I think “diversity”, which is the idea here, is valuable but needs to be thoughtfully designed and not done blindly.)
Some of the ideas, like solving for “factions”, involves “the vote graph” (a literal mathematical object, analogous to the friend graph on FB but for votes). This requires literal graph theory, e.g. I would probably consult a computer scientist or applied math lady.
I could also see using the view graph as useful.
This isn’t exactly the same as your idea, but a lot of your idea can be folded in (while less direct, there’s several ways we can bake in something like “people who get strong upvotes consistently are more rewarded”).
Maybe there is a tension between notifying people of upvoting diversity, and giving away identity, which might be one reason why some actual graph theory probably needs to be used.
I think this is an interesting idea. I would probably recommend against trying to “group” users, because it would be messy and hard to understand, and I am just generally worried about grouping users and treating them differently based on their groups. Just weakening your upvotes on users you often upvote seems practical and easy to understand.
Would minority views/interests become less visible this way, though?
I agree. There’s issues. It seems comple and not transparent at first[1]
I think the goal of using the graphs and building “factions” (“faction” is not the best word I would use here, like if you were to put this on a grant proposal) is that it makes it visible and legible.
This might be more general and useful than it seems and can be used prosocially.
For example, once legible, you could identify minority views and give them credence (or just make this a “slider” for people to examine).
Like you said, this is hard to execute. I think this is hard in the sense that the designer needs to find the right patterns for both socialization and practical use. Once found, the patterns can ultimately can be relatively simple and transparent.
Misc comments:
To be clear, I think the ideas I’m discussing in this post and reforms to the forum is at least a major project, up to a major set of useful interventions, maybe comparable to all of “prediction markets” (in the sense of the diversity of different projects it could support, investment, and potential impact that would justify it).
This isn’t something I am actively pursuing (but these forum discussions are interesting and hard to resist).
This sounds somewhat like a search-engine eigenvector thing, e.g. PageRank.
Yes, I think that’s right, I am guessing because both involve graph theory (and I guess in pagerank using the edges turns it into a linear algebra problem too?).
Note that I hardly know any more about graph theory.
PageRank mostly involved graph theory in the mere observation that there’s a directed graph of pages linking to each other. It then immediately turns to linear algebra, where the idea is that you want a page’s weight to correspond to the sum of the weights of the pages linking to it—and this exactly describes finding an eigenvector of the graph matrix.
On second thought I guess your idea for karma is more complicated, maybe I’ll look at some simple examples and see what comes up if I happen to have the time.
That’s interesting to know about pagerank. It’s smart it just goes to linear algebra.
I think building the graph requires data that isn’t publicly available like identity of votes and views. It might be hard to get a similar dataset to see if a method works or not. Some of the “clustering techniques” might not apply to other data.
Maybe there is a literature for this already.
Ya, that’s right, total karma divided by the number of votes.
What I proposed could be what determines strong vote power alone, but strong vote power could be based on a combination of multiple things. What I meant by “give weight to this” is that it could just be one of multiple determinants of strong vote power.
This is why I was thinking it would only be one factor. We could use both their total and average to determine the strong vote power. But also, maybe a bit of risk-aversion is good? The EA Forum is getting used more, so it would be good if the comments and posts people made were high quality rather than noise.
Also, again, it wouldn’t penalize users for making comments that don’t get voted on; it encourages them to chase strong upvotes and avoid downvotes (relative to regular upvotes or no votes).
I did have thought about a meta karma system, but where the voting effect or output differs by context (so a strong upvote or strong downvote has a different effect on comments or posts depending on the situation).
This would avoid the current situation of “I need to make sure the comment I like ranks highly or has the karma I want” and also prevent certain kinds of hysteresis.
While the voting effect or output can change radically in this system, based on context, IMO, it’s important make sure that the latent underlying voting power of the user accumulates in a fairly mundane/simple/gradual way, otherwise you get these weird incentives.
Maybe this underweights many people appreciating a given comment or post and regular upvoting it, but few of them appreciating it enough to strong upvote it, relative to a post or comment getting both a few strong upvotes and at most a few regular votes?