Thanks for writing this! While I’ve long been a big fan of impact prize-like-mechanisms, the ideas about memetics and reinforcement learning were nice and novel—or at least if I’d thought about them properly before I’d forgotten them since. I also found the quote from Wei Dai very interesting.
I think you somewhat over-hedged in your openning paragraph however:
Global markets are currently only (somewhat) efficient in incentivising problem-solving in areas where the benefits can be internalised, such as by earning a profit from the product one has built. ^[Ignoring various market failures such as long-time horizons, large coordination problems, high initial costs, and more.]
Within this circumscribed range of problems (those whose benefits can be internalised, without long-term horizons, little coordination required, low capital needs), markets seem to be one of the most efficient mechanisms we have in motivating problem-solving. While we can think of some others that are also very effective, like intellectual curiosity for scientific discovery or parental love for childraising, there do not seem to be many which are dramatically more effective. If markets are indeed a top-tier incentivisation mechanism, calling them only ‘somewhat’ efficient, rather than ‘quite’ or ‘often’ seems to rather understate their potential for market mechanisms to incentivise useful charitable work.
Secondly, I don’t think issues like long time horizons or high initial costs, and many coordination problems, are commonly considered market failures:
There are many examples of markets dealing correctly with issues a long way in the future. For example, Amazon has operated on a very long-term horizon for many years, forgoing current profitability for the sake of growth, and in financial markets there are liquid and rationally priced markets for bonds and derivatives maturing 50 years in the future.
Market mechanisms have incentivised investments in many large projects—a good example is semicondictor fabs, which take around a billion dollars of upfront capital investment before any return can be made.
Although you’re right that there are some classes of coordination problems for which markets can struggle (e.g. holdouts refusing to sell land needed for a large project), there are others at which it excels, like coordinating supply chains across huge numbers of people.
A quick check on some online lists here and here confirms than these three are not generally considered examples of market failure.
As such I think if anything you are underselling the potential here. The fact that markets can deal with these sorts of problems gives me hope for very long-term impact prizes—things like financing projects with tradable impact certificates that ultimately pay off years in the future when the ultimate impact is clear.
This is a great writeup! As the person who runs this Forum and tries to incentivize cognitive work* from participants, I found it useful and will return to it.
The memetics idea was particularly novel to me, but I think I intuitively understood the point—hence my always capitalizing “the EA Forum Prize” to make it a concrete, recognized proper noun.
If you look over the most recent Prize post, do you notice any places where you see an opportunity to improve the program? Or cases where you aren’t sure if we do X, but think that X would be particularly valuable to do if we weren’t yet doing it?
Also:
...e.g. awarding a prize for responses to “Does God exist?” and only having theologians put in the work, almost all of whom answer “Yes”.
I’m deeply amused by the thought of a theologian who sets out to win the “G-Prize”, only to discover that he’s no longer convinced by the arguments that sustain his faith once money is on the line. That would be quite the r/rational story prompt.
--
*Though I prefer to think of it as “cognitive fun” for those who like writing and helping the community and getting compliments from your friendly neighborhood moderator!
The prize has been going on for a while, which seems important, and I think the transparency of the Prize post is really important for making common knowledge of what kind of work there is demand for. So overall it’s pretty great.
The structure of feedback looks to me like: “here’s the object-level content of the post, and here are 2-3 reasons we liked it”. I think you could be more clear about what you want to incentivise. More precisely, the current structure doesn’t answer:
How strong were the reasons relative to each other? (e.g. maybe removing Reason A would make the person win 2nd prize instead of 1st, but removing Reason B might make them win no prize)
Were the reasons only jointly sufficient to merit the prize, or might accomplishing only one of them have worked?
What other properties did the post display, which did not merit the prize? For example, maybe prize-meriting posts tend to be quite long—even though length is not something you want to incentivise on the margin.
Why did the posts end up ordered the way they did? Beyond “the black-box voting process gave that verdict” :) Currently I don’t know why SHOW was judged as deserving 4x the prize money of “The Case for the Hotel”, for example.
Adding a bit of critique where appropriate seems good, since I’m the person who writes the summaries and can ask myself to put in more time.
Your other questions all come back to the black-box voting criterion, and I don’t push judges to share their individual reasoning. That’s largely because doing so makes being a judge more difficult; right now, voting is relatively little work, and given the small amounts of money at stake, I’m reluctant to ask for many more EA hours going toward the project).
We may consider splitting up the prize funding differently in the future; saying “piece X deserves four times as much money as piece Y” isn’t a good way to think about how the Prize works, but there’s still an argument for trying to more closely match grant funding to judges’ enthusiasm levels.
Thanks for writing this! While I’ve long been a big fan of impact prize-like-mechanisms, the ideas about memetics and reinforcement learning were nice and novel—or at least if I’d thought about them properly before I’d forgotten them since. I also found the quote from Wei Dai very interesting.
I think you somewhat over-hedged in your openning paragraph however:
Within this circumscribed range of problems (those whose benefits can be internalised, without long-term horizons, little coordination required, low capital needs), markets seem to be one of the most efficient mechanisms we have in motivating problem-solving. While we can think of some others that are also very effective, like intellectual curiosity for scientific discovery or parental love for childraising, there do not seem to be many which are dramatically more effective. If markets are indeed a top-tier incentivisation mechanism, calling them only ‘somewhat’ efficient, rather than ‘quite’ or ‘often’ seems to rather understate their potential for market mechanisms to incentivise useful charitable work.
Secondly, I don’t think issues like long time horizons or high initial costs, and many coordination problems, are commonly considered market failures:
There are many examples of markets dealing correctly with issues a long way in the future. For example, Amazon has operated on a very long-term horizon for many years, forgoing current profitability for the sake of growth, and in financial markets there are liquid and rationally priced markets for bonds and derivatives maturing 50 years in the future.
Market mechanisms have incentivised investments in many large projects—a good example is semicondictor fabs, which take around a billion dollars of upfront capital investment before any return can be made.
Although you’re right that there are some classes of coordination problems for which markets can struggle (e.g. holdouts refusing to sell land needed for a large project), there are others at which it excels, like coordinating supply chains across huge numbers of people.
A quick check on some online lists here and here confirms than these three are not generally considered examples of market failure.
As such I think if anything you are underselling the potential here. The fact that markets can deal with these sorts of problems gives me hope for very long-term impact prizes—things like financing projects with tradable impact certificates that ultimately pay off years in the future when the ultimate impact is clear.
This is a great writeup! As the person who runs this Forum and tries to incentivize cognitive work* from participants, I found it useful and will return to it.
The memetics idea was particularly novel to me, but I think I intuitively understood the point—hence my always capitalizing “the EA Forum Prize” to make it a concrete, recognized proper noun.
If you look over the most recent Prize post, do you notice any places where you see an opportunity to improve the program? Or cases where you aren’t sure if we do X, but think that X would be particularly valuable to do if we weren’t yet doing it?
Also:
I’m deeply amused by the thought of a theologian who sets out to win the “G-Prize”, only to discover that he’s no longer convinced by the arguments that sustain his faith once money is on the line. That would be quite the r/rational story prompt.
--
*Though I prefer to think of it as “cognitive fun” for those who like writing and helping the community and getting compliments from your friendly neighborhood moderator!
Thanks, that’s great to hear.
The prize has been going on for a while, which seems important, and I think the transparency of the Prize post is really important for making common knowledge of what kind of work there is demand for. So overall it’s pretty great.
The structure of feedback looks to me like: “here’s the object-level content of the post, and here are 2-3 reasons we liked it”. I think you could be more clear about what you want to incentivise. More precisely, the current structure doesn’t answer:
How strong were the reasons relative to each other? (e.g. maybe removing Reason A would make the person win 2nd prize instead of 1st, but removing Reason B might make them win no prize)
Were the reasons only jointly sufficient to merit the prize, or might accomplishing only one of them have worked?
What other properties did the post display, which did not merit the prize? For example, maybe prize-meriting posts tend to be quite long—even though length is not something you want to incentivise on the margin.
Why did the posts end up ordered the way they did? Beyond “the black-box voting process gave that verdict” :) Currently I don’t know why SHOW was judged as deserving 4x the prize money of “The Case for the Hotel”, for example.
Adding a bit of critique where appropriate seems good, since I’m the person who writes the summaries and can ask myself to put in more time.
Your other questions all come back to the black-box voting criterion, and I don’t push judges to share their individual reasoning. That’s largely because doing so makes being a judge more difficult; right now, voting is relatively little work, and given the small amounts of money at stake, I’m reluctant to ask for many more EA hours going toward the project).
We may consider splitting up the prize funding differently in the future; saying “piece X deserves four times as much money as piece Y” isn’t a good way to think about how the Prize works, but there’s still an argument for trying to more closely match grant funding to judges’ enthusiasm levels.