A post on a non-community topic that receives ~500 karma is roughly equivalent in impact to a high quality research paper in a peer reviewed journal
This makes sense assuming i) impact increases logarithmically with karma, ii) Nuño’s impact estimates of the EA Forum Prize winners are correct, and iii) the relationship between karma and impact among these posts generalises to other posts. However, I have so little confidence about these assumptions that I would not use karma as the only proxy for impact. At best, I would use the logarithm of karma as one component of a weighted factor model (WFM). I note in the post that:
The predictions above [by assuming i), ii) and iii)] are quite poor. For instance, they imply:
An unreasonably small difference of:
0.367 karma between the impact of “a good blog post, a particularly good comment” and “a thoughtful comment”.
3.67 karma between the impact of “an excellent blog post” and “a good blog post, a particularly good comment”.
“A particularly valuable paper” is worth 475 karma, whereas I think the right value is of the order of magnitude of 10 kkarma (with huge variation), i.e. 21.1 times as large.
In any case, one should certainly be mindful of Goodhart’s Law, and do not start optimising posts just for karma!
You are welcome!
This makes sense assuming i) impact increases logarithmically with karma, ii) Nuño’s impact estimates of the EA Forum Prize winners are correct, and iii) the relationship between karma and impact among these posts generalises to other posts. However, I have so little confidence about these assumptions that I would not use karma as the only proxy for impact. At best, I would use the logarithm of karma as one component of a weighted factor model (WFM). I note in the post that: