while some users reported finding the Prize valuable or motivating, that number wasn’t quite as high as I had been hoping for
It seems like the instrumental thing here is whether users who won prizes found them motivating. Most users will not write prize-winning posts, but if the users who did were at least partially motivated by the prospect of winning one, then the world with the prize is almost certainly better than the counterfactual. More generally, if users who wrote original posts were likelier to endorse the prize than users in general, that is some indication that the prize is somewhat effective. Did you have enough data to determine whether either of these situations obtain?
That’s a good point; I should survey past winners to get their views on this, too (though I expect people to report that they were influenced a bit more than was actually the case).
Users who were more active on the Forum were more likely to endorse the Prize, but that seems like it results partly from their being more aware of the Prize in general/reading more of the posts. Given that the posts are pinned for at least a few days each month, and shared with the biggest EA Facebook group, awareness/readership numbers were lower than I’d have hoped for.
Thanks for the feedback, and for giving me an idea on data to collect!
It seems like the instrumental thing here is whether users who won prizes found them motivating. Most users will not write prize-winning posts, but if the users who did were at least partially motivated by the prospect of winning one, then the world with the prize is almost certainly better than the counterfactual. More generally, if users who wrote original posts were likelier to endorse the prize than users in general, that is some indication that the prize is somewhat effective. Did you have enough data to determine whether either of these situations obtain?
That’s a good point; I should survey past winners to get their views on this, too (though I expect people to report that they were influenced a bit more than was actually the case).
Users who were more active on the Forum were more likely to endorse the Prize, but that seems like it results partly from their being more aware of the Prize in general/reading more of the posts. Given that the posts are pinned for at least a few days each month, and shared with the biggest EA Facebook group, awareness/readership numbers were lower than I’d have hoped for.
Thanks for the feedback, and for giving me an idea on data to collect!