In general, I am in favour of having a forum prize of some kind. I personally find it motivating and I think it is a useful way for people to gain credibility within the EA movement. I think it also helps lend the forum an air of seriousness, which is sometimes good and sometimes bad but does probably raise the quality of the content.
From the “giving individuals credibility” perspective, having a few large prizes is good (a $750 prize is large enough to put on a CV for many people, a $50 one is clearly not). From the “motivating good content” perspective, having a moderately larger number of moderately smaller prizes would probably be better.
Perhaps I’m wrong about this, but I’ve sometimes felt like comment prizes are used to make up the difference on post prizes (“your post was very good but not top-3 this month, so we found a decent comment you made on that post and gave it to that!”). If so I think that is good evidence that we want more smaller prizes.
Comments on specific proposals
Themed prizes (e.g. setting aside one prize for “the best post on topic X”)
This seems like something you might want to do less frequently (annually?) in addition to more regular general prizes.
Giving more prizes, even if they end up being smaller on average
On the whole I think this would be a good idea (see above).
Selecting at least one judge to represent each major EA cause area
I don’t feel like any major cause area has been getting shafted here; I can readily recall winners on global health, animal welfare, long-term future, and meta topics. In the absence of a specific problem this seems like not a good idea; I feel like it would not provide much benefit and might encourage tribalism/factionalism.
The above comments assume topic-general prizes like the current system; I think themed prizes probably would want some specialist judges involved.
Including a community vote (not just upvotes, but a separate voting process). This would likely supplement judges’ votes, not replace them.
A lot of award ceremonies have some kind of “people’s choice” award. If you do this I again think it should probably be less frequent than monthly; perhaps quarterly or annually.
Having a special “first post” and/or “first comment” prize for people who make a really good first contribution from an account
Meh. This sounds cute but I don’t actually think it would be very valuable, and could actually be harmful on net. I think it’s good to set a high quality bar and incentivise people to work up to it, rather than agonising over making their first post perfect even more than they probably already do.
Having separate prizes for orgs/professional researchers and people who contribute to the Forum on more of an “amateur” basis
So I suppose the argument for this is that some people basically get paid to write on the Forum, and it’s not surprising that those people (coughSauliuscough) win lots of prizes, which crowds out good content from amateurs. I’m not sure to what extent this is true, and if true, I’m unconvinced that it’s bad.
Firstly, a lot of that content is just very good and I want there to be strong incentives to get that stuff on the forum. Secondly, a lot of prize-winning posts are from people who work for EA Orgs but are writing in their private capacity; indeed, you’d expect people who are especially committed to thinking about EA topics to both be more likely to win Forum prizes and be more likely to work for an EA Org. This seems like a good thing we want more of, no less so than contributions from non-Org-employees, so I don’t think they should all be put in a special category and forced to compete with each other.
There’s a similar argument which is something like “separate out content people write for work vs for pleasure”. I think this is more defensible but still probably wrong. If prizes incentivise orgs to post more of their work on the EA Forum, this seems like a good thing that benefits everybody. But I wouldn’t be extremely surprised to be argued out of this.
Having more flexible prize amounts (e.g. maybe one post should win all the money in some months if it’s especially good, or maybe money should be distributed according to vote ratios rather than just first/second/third place)
This sounds bad to me, but for fairly vague reasons. I feel like it gives the judges too many degrees of freedom, and that it’s probably good practice to have prize amounts be fairly predictable. But I’m not sure about this.
Having judges who are somewhat removed the community (or finding some other way to reduce the extent to which the Prize may reflect the biases of the community or of central orgs within the community)
This sounds like a great way to remove most of the value of the prize in return for highly dubious gains.
If I look at the winners of the EA Forum Prize, I expect to see exemplars of great EA content. The best people to judge great EA content are EAs, i.e. community members. We could bring in external expects from adjacent fields (global health, AI, etc), but why?
Firstly, those people are likely to be far more limited in the range of posts they can judge, which basically forces us into the “subject-specific prizes” model above; while I think those could be good to have occasionally I wouldn’t want them to completely take over the prize.
Secondly, those people are almost certain to be less well-aligned with the distinctive values of the EA community, which makes their opinions on which posts should get prizes much less valuable in this particular context.
Thirdly, if we do want domain experts to judge a domain-specific EA prize I’m pretty sure we can find them within the EA community, rather than bending over backwards to make them external.
And finally, all those fields already have their own fora, their own prizes, and their own way of sharing and evaluating information. It’s fine – indeed, very good – for the particular nexus known as EA to have their own systems and prizes too. It only becomes a problem if we think EA Forum prizes are the sole arbiters of truth and quality.
Getting rid of the Prize entirely without replacing it (one survey respondent, who has written many excellent posts and won at least one Prize, believes it to be “distracting and unnecessarily divisive”)
Has there been any evidence of the prize being divisive, i.e. actually causing conflict? Perhaps there is, but I’m not aware of it. And calling it distracting is confusing to me; is the claim that it incentivises people to write content based on a Keynesian beauty contest, rather than what they actually think it would be best for them to write?
Anyway, I don’t have much sympathy with that claim as stated, or for abolishing the prize entirely on that basis, but there might be an alternative interpretation that I’d be more sympathetic to.
So I suppose the argument for this is that some people basically get paid to write on the Forum, and it’s not surprising that those people (coughSauliuscough) win lots of prizes, which crowds out good content from amateurs. I’m not sure to what extent this is true, and if true, I’m unconvinced that it’s bad.
I went and checked and I think the people who have won the most money from the EA Forum Prize are actually Buck Shlegeris and Ben Kuhn (tied for first place). I don’t know if they were paid to write what they wrote but my guess is they were not.
General comments
In general, I am in favour of having a forum prize of some kind. I personally find it motivating and I think it is a useful way for people to gain credibility within the EA movement. I think it also helps lend the forum an air of seriousness, which is sometimes good and sometimes bad but does probably raise the quality of the content.
From the “giving individuals credibility” perspective, having a few large prizes is good (a $750 prize is large enough to put on a CV for many people, a $50 one is clearly not). From the “motivating good content” perspective, having a moderately larger number of moderately smaller prizes would probably be better.
Perhaps I’m wrong about this, but I’ve sometimes felt like comment prizes are used to make up the difference on post prizes (“your post was very good but not top-3 this month, so we found a decent comment you made on that post and gave it to that!”). If so I think that is good evidence that we want more smaller prizes.
Comments on specific proposals
This seems like something you might want to do less frequently (annually?) in addition to more regular general prizes.
On the whole I think this would be a good idea (see above).
I don’t feel like any major cause area has been getting shafted here; I can readily recall winners on global health, animal welfare, long-term future, and meta topics. In the absence of a specific problem this seems like not a good idea; I feel like it would not provide much benefit and might encourage tribalism/factionalism.
The above comments assume topic-general prizes like the current system; I think themed prizes probably would want some specialist judges involved.
A lot of award ceremonies have some kind of “people’s choice” award. If you do this I again think it should probably be less frequent than monthly; perhaps quarterly or annually.
Meh. This sounds cute but I don’t actually think it would be very valuable, and could actually be harmful on net. I think it’s good to set a high quality bar and incentivise people to work up to it, rather than agonising over making their first post perfect even more than they probably already do.
So I suppose the argument for this is that some people basically get paid to write on the Forum, and it’s not surprising that those people (coughSauliuscough) win lots of prizes, which crowds out good content from amateurs. I’m not sure to what extent this is true, and if true, I’m unconvinced that it’s bad.
Firstly, a lot of that content is just very good and I want there to be strong incentives to get that stuff on the forum. Secondly, a lot of prize-winning posts are from people who work for EA Orgs but are writing in their private capacity; indeed, you’d expect people who are especially committed to thinking about EA topics to both be more likely to win Forum prizes and be more likely to work for an EA Org. This seems like a good thing we want more of, no less so than contributions from non-Org-employees, so I don’t think they should all be put in a special category and forced to compete with each other.
There’s a similar argument which is something like “separate out content people write for work vs for pleasure”. I think this is more defensible but still probably wrong. If prizes incentivise orgs to post more of their work on the EA Forum, this seems like a good thing that benefits everybody. But I wouldn’t be extremely surprised to be argued out of this.
This sounds bad to me, but for fairly vague reasons. I feel like it gives the judges too many degrees of freedom, and that it’s probably good practice to have prize amounts be fairly predictable. But I’m not sure about this.
This sounds like a great way to remove most of the value of the prize in return for highly dubious gains.
If I look at the winners of the EA Forum Prize, I expect to see exemplars of great EA content. The best people to judge great EA content are EAs, i.e. community members. We could bring in external expects from adjacent fields (global health, AI, etc), but why?
Firstly, those people are likely to be far more limited in the range of posts they can judge, which basically forces us into the “subject-specific prizes” model above; while I think those could be good to have occasionally I wouldn’t want them to completely take over the prize.
Secondly, those people are almost certain to be less well-aligned with the distinctive values of the EA community, which makes their opinions on which posts should get prizes much less valuable in this particular context.
Thirdly, if we do want domain experts to judge a domain-specific EA prize I’m pretty sure we can find them within the EA community, rather than bending over backwards to make them external.
And finally, all those fields already have their own fora, their own prizes, and their own way of sharing and evaluating information. It’s fine – indeed, very good – for the particular nexus known as EA to have their own systems and prizes too. It only becomes a problem if we think EA Forum prizes are the sole arbiters of truth and quality.
Has there been any evidence of the prize being divisive, i.e. actually causing conflict? Perhaps there is, but I’m not aware of it. And calling it distracting is confusing to me; is the claim that it incentivises people to write content based on a Keynesian beauty contest, rather than what they actually think it would be best for them to write?
Anyway, I don’t have much sympathy with that claim as stated, or for abolishing the prize entirely on that basis, but there might be an alternative interpretation that I’d be more sympathetic to.
I went and checked and I think the people who have won the most money from the EA Forum Prize are actually Buck Shlegeris and Ben Kuhn (tied for first place). I don’t know if they were paid to write what they wrote but my guess is they were not.