1. I’ve been doing a decent amount of thinking & experimentation in similar work recently. I’m personally optimistic about non-market applications like GJP and Metaculus. I think that the path for similar groups to pay forecasters is much more straightforward than similar in prediction markets. I think there could be a lot more good work in this area.
2. GJP charges several thousand per question, but Metaculus is free, assuming they accept your questions. I think the answer to this is very complicated; there are many variables at play. That said, I think that with a powerful system, $50k-500k per year in predictions could get a pretty significant informational return.
3. This is also a very vague question, it’s not obvious what metrics to use to best answer it. That said, if a good prediction system is made, it could help answer this question in specific quantitative ways. It seems to me that a robust prediction system should be roughly at least as accurate as a non-predictive system with the same people. Long-term predictions are tricky, but I think we could have some basic estimates of bias.
4. This is also a huge question. I think there’s a lot of experimentation yet to be done here on many different kinds of questions. If we could have meta-predictions on things like, “How important will we have found this predictable item was to have in the system”, then we may be able to use the system to answer and optimize here.
5. I’m not very optimistic about prediction markets. This is of course something that would be nice to formally predict in the next 1-3 years.
On your questions:
1. I’ve been doing a decent amount of thinking & experimentation in similar work recently. I’m personally optimistic about non-market applications like GJP and Metaculus. I think that the path for similar groups to pay forecasters is much more straightforward than similar in prediction markets. I think there could be a lot more good work in this area.
2. GJP charges several thousand per question, but Metaculus is free, assuming they accept your questions. I think the answer to this is very complicated; there are many variables at play. That said, I think that with a powerful system, $50k-500k per year in predictions could get a pretty significant informational return.
3. This is also a very vague question, it’s not obvious what metrics to use to best answer it. That said, if a good prediction system is made, it could help answer this question in specific quantitative ways. It seems to me that a robust prediction system should be roughly at least as accurate as a non-predictive system with the same people. Long-term predictions are tricky, but I think we could have some basic estimates of bias.
4. This is also a huge question. I think there’s a lot of experimentation yet to be done here on many different kinds of questions. If we could have meta-predictions on things like, “How important will we have found this predictable item was to have in the system”, then we may be able to use the system to answer and optimize here.
5. I’m not very optimistic about prediction markets. This is of course something that would be nice to formally predict in the next 1-3 years.