Thanks for this feedback! I was thinking about exactly the same issue as I counted the votes and wrote up this post.
--
Back when we were setting up initial rules for the Prize, I wasn’t sure whether to allow posts written on “org time” (that is, by employees of EA organizations who were paid by their employers for Forum work). Eventually, I decided to err on the side of making almost all posts eligible as a starting point, but to keep an eye on which types of posts were winning.
This is the first month (out of eight) that all winning posts have come from the employees of EA orgs; since the Prize began in November, roughly half of the winning posts have come from Forum contributors who (as far as I know) weren’t employed in direct work at the time, or were writing about subjects unrelated to their direct work. Some of the other half were written by org employees who drew on their work experience, but in cases where I’m not sure whether they were paid to do so (e.g. November’s winning post on EAF’s hiring process).
This doesn’t indicate that posts from employees of EA orgs should necessarily remain in the same category, but I did want to note that this month was anomalous. (We certainly don’t intend to be rating “the top serious EA organization documents.”)
---
Some thoughts on ways we could address this concern:
The comment prize, which we’ll be starting up next month, should help us highlight contributions that didn’t require as much time to make, and I could imagine scaling it up over time (in the sense of “amount awarded for comments relative to posts”). I noted this in my initial post:
We also hope that a “comment prize” will make it easier to recognize people who contribute their ideas without publishing full-fledged research posts.
2. Some organizations have been unusually thorough in posting on the Forum, and this is something we’d like to highlight and encourage (whether through a prize or some other means). For example, researchers from Rethink Priorities have spent a lot of additional time formatting posts and responding to comments, rather than only cross-posting research from their website.
3. It’s possible that posts produced by organizations should be in a separate category, though it’s tricky to define when this is the case. For example, Open Phil is a very different kind of research organization than a smaller org like ALLFED or AI Impacts, and I’m uncertain how to define people who are freelance researchers working off of a small grant or commission. It’s also hard to tell when something was or was not written on “paid time” by the employee of an EA organization.
Personally, I have a higher bar on voting for posts that come from org employees, but I’ll disclose that I did vote for each of the winning posts this month — I thought that the invertebrate sentience and nuclear risk series were especially outstanding, even by the standards of EA research organizations.
This is something I and the other judges will be discussing in future months, and if you have further thoughts, I’d appreciate hearing them!
I think that the main “organization” posts I’m thinking of are almost like a different class, like they are using the EA Forum as an academic journal as opposed to as a blog. There could be some self-selection then; like a separate category / website where people self-select for a different kind of feedback. I’m going to be chatting to people about this.
Ozzie,
Thanks for this feedback! I was thinking about exactly the same issue as I counted the votes and wrote up this post.
--
Back when we were setting up initial rules for the Prize, I wasn’t sure whether to allow posts written on “org time” (that is, by employees of EA organizations who were paid by their employers for Forum work). Eventually, I decided to err on the side of making almost all posts eligible as a starting point, but to keep an eye on which types of posts were winning.
This is the first month (out of eight) that all winning posts have come from the employees of EA orgs; since the Prize began in November, roughly half of the winning posts have come from Forum contributors who (as far as I know) weren’t employed in direct work at the time, or were writing about subjects unrelated to their direct work. Some of the other half were written by org employees who drew on their work experience, but in cases where I’m not sure whether they were paid to do so (e.g. November’s winning post on EAF’s hiring process).
This doesn’t indicate that posts from employees of EA orgs should necessarily remain in the same category, but I did want to note that this month was anomalous. (We certainly don’t intend to be rating “the top serious EA organization documents.”)
---
Some thoughts on ways we could address this concern:
The comment prize, which we’ll be starting up next month, should help us highlight contributions that didn’t require as much time to make, and I could imagine scaling it up over time (in the sense of “amount awarded for comments relative to posts”). I noted this in my initial post:
2. Some organizations have been unusually thorough in posting on the Forum, and this is something we’d like to highlight and encourage (whether through a prize or some other means). For example, researchers from Rethink Priorities have spent a lot of additional time formatting posts and responding to comments, rather than only cross-posting research from their website.
3. It’s possible that posts produced by organizations should be in a separate category, though it’s tricky to define when this is the case. For example, Open Phil is a very different kind of research organization than a smaller org like ALLFED or AI Impacts, and I’m uncertain how to define people who are freelance researchers working off of a small grant or commission. It’s also hard to tell when something was or was not written on “paid time” by the employee of an EA organization.
Personally, I have a higher bar on voting for posts that come from org employees, but I’ll disclose that I did vote for each of the winning posts this month — I thought that the invertebrate sentience and nuclear risk series were especially outstanding, even by the standards of EA research organizations.
This is something I and the other judges will be discussing in future months, and if you have further thoughts, I’d appreciate hearing them!
Makes sense. I’m excited for the comment prize.
I think that the main “organization” posts I’m thinking of are almost like a different class, like they are using the EA Forum as an academic journal as opposed to as a blog. There could be some self-selection then; like a separate category / website where people self-select for a different kind of feedback. I’m going to be chatting to people about this.