It would, for example, be super valuable to have something like “if you spend 5 minutes writing a comment that gets at least 10 in Karma, the is likely as cost effective as earning $200/hour and donating to AMF” or “if you spend 20 hours writing a post that gets at least 50 karma it is equivalent to earning $300/hour and donating all of that to AMF”.
When combined with the difficulty of assessing impact of Forum posts and comments, I think the relationship between karma and impact is too murky to make impact-per-karma a meaningful measure. Posts/comments of a specialized/technical nature will likely have a significantly higher IpK value than meta commentary.
Yeah perhaps, but I have no idea even why the forum is considered cost effective. Is it more because it advances work on causes? Or is it more because it builds and maintains the community? I think the answer to this question will go some way to understanding the strength of your claim. In general, I just wished this information was out there, just something about why we think the forum going is a good use of our time. Right now, I have not seen anything. I remember seeing more information about EAGs and their benefits and their cost effectiveness.
Thanks! That does seem super relevant. Is my understanding below roughly correct?
”A post on a non-community topic that receives ~500 karma is roughly equivalent in impact to a high quality research paper in a peer reviewed journal. And such a post receiving about 100 karma or a bit below is about 1-10% of the impact of such a journal article.”
A post on a non-community topic that receives ~500 karma is roughly equivalent in impact to a high quality research paper in a peer reviewed journal
This makes sense assuming i) impact increases logarithmically with karma, ii) Nuño’s impact estimates of the EA Forum Prize winners are correct, and iii) the relationship between karma and impact among these posts generalises to other posts. However, I have so little confidence about these assumptions that I would not use karma as the only proxy for impact. At best, I would use the logarithm of karma as one component of a weighted factor model (WFM). I note in the post that:
The predictions above [by assuming i), ii) and iii)] are quite poor. For instance, they imply:
An unreasonably small difference of:
0.367 karma between the impact of “a good blog post, a particularly good comment” and “a thoughtful comment”.
3.67 karma between the impact of “an excellent blog post” and “a good blog post, a particularly good comment”.
“A particularly valuable paper” is worth 475 karma, whereas I think the right value is of the order of magnitude of 10 kkarma (with huge variation), i.e. 21.1 times as large.
In any case, one should certainly be mindful of Goodhart’s Law, and do not start optimising posts just for karma!
It would, for example, be super valuable to have something like “if you spend 5 minutes writing a comment that gets at least 10 in Karma, the is likely as cost effective as earning $200/hour and donating to AMF” or “if you spend 20 hours writing a post that gets at least 50 karma it is equivalent to earning $300/hour and donating all of that to AMF”.
When combined with the difficulty of assessing impact of Forum posts and comments, I think the relationship between karma and impact is too murky to make impact-per-karma a meaningful measure. Posts/comments of a specialized/technical nature will likely have a significantly higher IpK value than meta commentary.
Yeah perhaps, but I have no idea even why the forum is considered cost effective. Is it more because it advances work on causes? Or is it more because it builds and maintains the community? I think the answer to this question will go some way to understanding the strength of your claim. In general, I just wished this information was out there, just something about why we think the forum going is a good use of our time. Right now, I have not seen anything. I remember seeing more information about EAGs and their benefits and their cost effectiveness.
Hi Ulrik,
You may want to check my post on What is the relationship between impact and EA Forum karma?.
Thanks! That does seem super relevant. Is my understanding below roughly correct?
”A post on a non-community topic that receives ~500 karma is roughly equivalent in impact to a high quality research paper in a peer reviewed journal. And such a post receiving about 100 karma or a bit below is about 1-10% of the impact of such a journal article.”
You are welcome!
This makes sense assuming i) impact increases logarithmically with karma, ii) Nuño’s impact estimates of the EA Forum Prize winners are correct, and iii) the relationship between karma and impact among these posts generalises to other posts. However, I have so little confidence about these assumptions that I would not use karma as the only proxy for impact. At best, I would use the logarithm of karma as one component of a weighted factor model (WFM). I note in the post that: