Reasons why upvotes on the EA forum and LW don’t correlate that well with impact .
More easily accessible content, or more introductory material gets upvoted more.
Material which gets shared more widely gets upvoted more.
Content which is more prone to bikeshedding gets upvoted more.
Posts which are beautifully written are more upvoted.
Posts written by better known authors are more upvoted (once you’ve seen this, you can’t unsee).
The time at which a post is published affects how many upvotes it gets.
Other random factors, such as whether other strong posts are published at the same time, also affect the number of upvotes.
Not all projects are conducive to having a post written about them.
The function from value to upvotes is concave (e.g., like a logarithm or like a square root), in that a project which results in a post with a 100 upvotes is probably more than 5 times as valuable as 5 posts with 20 upvotes each. This is what you’d expect if the supply of upvotes was limited.
Upvotes suffer from inflation as EA forum gets populated more, so that a post which would have gathered 50 upvotes two years might gather 100 upvotes now.
Upvotes may not take into account the relationship between projects, or other indirect effects. For example, projects which contribute to existing agendas are probably more valuable than otherwise equal standalone projects, but this might not be obvious from the text.
I agree that the correlation between number of upvotes on EA forum and LW posts/comments and impact isn’t very strong. (My sense is that it’s somewhere between weak and strong, but not very weak or very strong.) I also agree that most of the reasons you list are relevant.
But how I’d frame this is that—for example—a post being more accessible increases the post’s expected upvotes even more than it increases its expected impact. I wouldn’t say “Posts that are more accessible get more upvotes, therefore the correlation is weak”, because I think increased accessibility will indeed increase a post’s impact (holding other factor’s constant).
Same goes for many of the other factors you list.
E.g., more sharing tends to both increase a post’s impact (more readers means more opportunity to positively influence people) and signal that the post would have a positive impact on each reader (as that is one factor—among many—in whether people share things). So the mere fact that sharing probably tends to increase upvotes to some extent doesn’t necessarily weaken the correlation between upvotes and impact. (Though I’d guess that sharing does increase upvotes more than it increases/signals impact, so this comment is more like a nitpick than a very substantive disagreement.)
To make it clear, the claim is that the number karma for a forum post on a project does not correlate well with the project’s direct impact? Rather than, say, that a karma score of a post correlates well with the impact of the post itself on the community?
I’d say it also doesn’t correlate that well with its total (direct+indirect) impact either, but yes. And I was thinking more in contrast to the karma score being an ideal measure of total impact; I don’t have thoughts to share here on the impact of the post itself on the community.
I think that for me, I upvote according to how much I think a post itself is valuable for me or for the community as a whole. At least, that’s what I’m trying to do when I’m thinking about it logically.
Reasons why upvotes on the EA forum and LW don’t correlate that well with impact .
More easily accessible content, or more introductory material gets upvoted more.
Material which gets shared more widely gets upvoted more.
Content which is more prone to bikeshedding gets upvoted more.
Posts which are beautifully written are more upvoted.
Posts written by better known authors are more upvoted (once you’ve seen this, you can’t unsee).
The time at which a post is published affects how many upvotes it gets.
Other random factors, such as whether other strong posts are published at the same time, also affect the number of upvotes.
Not all projects are conducive to having a post written about them.
The function from value to upvotes is concave (e.g., like a logarithm or like a square root), in that a project which results in a post with a 100 upvotes is probably more than 5 times as valuable as 5 posts with 20 upvotes each. This is what you’d expect if the supply of upvotes was limited.
Upvotes suffer from inflation as EA forum gets populated more, so that a post which would have gathered 50 upvotes two years might gather 100 upvotes now.
Upvotes may not take into account the relationship between projects, or other indirect effects. For example, projects which contribute to existing agendas are probably more valuable than otherwise equal standalone projects, but this might not be obvious from the text.
...
I agree that the correlation between number of upvotes on EA forum and LW posts/comments and impact isn’t very strong. (My sense is that it’s somewhere between weak and strong, but not very weak or very strong.) I also agree that most of the reasons you list are relevant.
But how I’d frame this is that—for example—a post being more accessible increases the post’s expected upvotes even more than it increases its expected impact. I wouldn’t say “Posts that are more accessible get more upvotes, therefore the correlation is weak”, because I think increased accessibility will indeed increase a post’s impact (holding other factor’s constant).
Same goes for many of the other factors you list.
E.g., more sharing tends to both increase a post’s impact (more readers means more opportunity to positively influence people) and signal that the post would have a positive impact on each reader (as that is one factor—among many—in whether people share things). So the mere fact that sharing probably tends to increase upvotes to some extent doesn’t necessarily weaken the correlation between upvotes and impact. (Though I’d guess that sharing does increase upvotes more than it increases/signals impact, so this comment is more like a nitpick than a very substantive disagreement.)
To make it clear, the claim is that the number karma for a forum post on a project does not correlate well with the project’s direct impact? Rather than, say, that a karma score of a post correlates well with the impact of the post itself on the community?
I’d say it also doesn’t correlate that well with its total (direct+indirect) impact either, but yes. And I was thinking more in contrast to the karma score being an ideal measure of total impact; I don’t have thoughts to share here on the impact of the post itself on the community.
Thanks, that makes sense.
I think that for me, I upvote according to how much I think a post itself is valuable for me or for the community as a whole. At least, that’s what I’m trying to do when I’m thinking about it logically.