How would you recommend incorporating these ideas into the Forum prize?
Given the small amounts of money involved, it seems like “tokenizing” a post would be difficult (a lot of effort for not much reward), and I’m a little worried that public “prediction markets” around post prizes would create a strange atmosphere for judges (e.g. if we know which people will lose or gain money based on how we award prizes, it could create the appearance of collusion even if we never look at the market).
But I may have misunderstood which “aspects” you were thinking of, or something about the Impact Prize idea more generally, so I’m curious to hear your thoughts in more detail!
Agreed that tokenizing and markets would be difficult in the short term.
The main possible aspect would be evaluating many projects and estimating the impact of them, as opposed to just giving an ordering for the very top projects. A rubric can be used for evaluations.
For example, say you have some rubric where every project was scored on “importance”, “novelty”, and “quanity” or similar. Then you divy up the prize money proportional to those things on the rubric and make the results public.
That could be an uncomfortable level of transparency for some people, but it would help foster a discussion of which projects are the most valuable.
I like the idea of trying to be more granular with evaluation, though I don’t like the idea of making judges do a lot more work. Right now, I’d estimate that the value of the time it takes for judges to vote + CEA to administrate the prize is more than half the cost of the prize itself.
I could see something like “divide up winnings by number of votes”, since we have approval voting already, though that won’t track impact very precisely (a post with one vote is probably less than 1⁄6 as “valuable” as a post that gets a unanimous vote from all 6 judges). I’ll keep thinking about different systems, though I think the current amounts will be kept stable for at least another few months.
How would you recommend incorporating these ideas into the Forum prize?
Given the small amounts of money involved, it seems like “tokenizing” a post would be difficult (a lot of effort for not much reward), and I’m a little worried that public “prediction markets” around post prizes would create a strange atmosphere for judges (e.g. if we know which people will lose or gain money based on how we award prizes, it could create the appearance of collusion even if we never look at the market).
But I may have misunderstood which “aspects” you were thinking of, or something about the Impact Prize idea more generally, so I’m curious to hear your thoughts in more detail!
Agreed that tokenizing and markets would be difficult in the short term.
The main possible aspect would be evaluating many projects and estimating the impact of them, as opposed to just giving an ordering for the very top projects. A rubric can be used for evaluations.
For example, say you have some rubric where every project was scored on “importance”, “novelty”, and “quanity” or similar. Then you divy up the prize money proportional to those things on the rubric and make the results public.
That could be an uncomfortable level of transparency for some people, but it would help foster a discussion of which projects are the most valuable.
I like the idea of trying to be more granular with evaluation, though I don’t like the idea of making judges do a lot more work. Right now, I’d estimate that the value of the time it takes for judges to vote + CEA to administrate the prize is more than half the cost of the prize itself.
I could see something like “divide up winnings by number of votes”, since we have approval voting already, though that won’t track impact very precisely (a post with one vote is probably less than 1⁄6 as “valuable” as a post that gets a unanimous vote from all 6 judges). I’ll keep thinking about different systems, though I think the current amounts will be kept stable for at least another few months.