Perhaps the most plausible way in which this could happen is that the authors of prize-winning posts are incentivized to post more frequently. We therefore examined whether prize-winning authors post more frequently in the six months following their prize than in the six months prior to it, relative to a control group.
I’m surprised this seemed the most plausible mechanism. Surely the incentive should have occurred prior to winning the prize? For my own case, I observed the existence of the prize, which encouraged me to put more work into making my post better, and the winning came later, presumably in part due to this extra effort. Is your idea that winning signals that you are high enough quality to be able to win, and hence its worth trying again?
In fact if winners suspected the Judges would be averse to letting them win ‘too often’ out of some egalitarian sentiment the effect might go in the opposite direction (though I think this would be very small, and I don’t think I used this as a judging criteria).
A year ago I suggested the use of randomization to estimate the causal effects of the prize on the parameters of interest. In general, I’m surprised that the EA community (including EA orgs such as CFAR or 80k) doesn’t use randomization more. One of Jeff Kaufman’s policy proposals is to “randomize everything”, and I think this should be an EA policy as much as a government policy.
Thanks Larks! I somewhat regularly encounter people who are hesitant to post on the forum, and can’t recall a time when telling them about the existence of the prize made them seem more likely to post. I can, however, think of people who have told me that they were more willing to post after having received the prize or other recognition for their work.
My guess is that something like imposter syndrome is more of a barrier to people posting than money is.
I’m surprised this seemed the most plausible mechanism. Surely the incentive should have occurred prior to winning the prize? For my own case, I observed the existence of the prize, which encouraged me to put more work into making my post better, and the winning came later, presumably in part due to this extra effort. Is your idea that winning signals that you are high enough quality to be able to win, and hence its worth trying again?
In fact if winners suspected the Judges would be averse to letting them win ‘too often’ out of some egalitarian sentiment the effect might go in the opposite direction (though I think this would be very small, and I don’t think I used this as a judging criteria).
A year ago I suggested the use of randomization to estimate the causal effects of the prize on the parameters of interest. In general, I’m surprised that the EA community (including EA orgs such as CFAR or 80k) doesn’t use randomization more. One of Jeff Kaufman’s policy proposals is to “randomize everything”, and I think this should be an EA policy as much as a government policy.
Thanks Larks! I somewhat regularly encounter people who are hesitant to post on the forum, and can’t recall a time when telling them about the existence of the prize made them seem more likely to post. I can, however, think of people who have told me that they were more willing to post after having received the prize or other recognition for their work.
My guess is that something like imposter syndrome is more of a barrier to people posting than money is.