Very cool idea, would love to see this implemented. Some thoughts while reading:
the government is happy because it effectively harnesses a lot of inputs about what to fund
don’t they have plenty of that already and further pressures are actually negative if they think they know best?
Non-populist politicians are happy because they can tell their voters “your opinion matters” and now it’s believable
FWIW I feel like I rarely observe non-populist politicians running on a direct democracy platform, and the one time I remember they weren’t successful (the German Piratenpartei).
Who are the experts? I expect this to cause controversy. Even without this mechanism I already perceive that on fairly clearcut questions there is a lot of controversy, also in less polarized countries relative to the US, like Germany (e.g. on nuclear energy, GMOs, rent control). Maybe this could be circumvented by letting the population decide it? Or at least their elected representatives? I‘ve stumbled upon the „Bayesian Truth Serum“ mechanism that might useful for eliciting non-measurable outcomes.
https://www.overcomingbias.com/2017/01/surprising-popularity.html
My intuition is, that around 70% of citizens in the western societies, who decide to participate, would go for the “Activist” strategy and 30% for the “Forecaster” strategy, but there is no evidence.
Maybe this could be a useful use case of integrating an Elicit forecast like described here. https://forum.effectivealtruism.org/posts/YgbPSyTvft6EcKnWm/forum-update-new-features-december-2020#Elicit_predictions
FWIW I’d expect even more „Activists“, more like 95% maybe? Only being able to predict which of ~20 categories some group of „experts“ will find most important feels demotivating to me, even though I do forecasts regularly. This might change with the amount of monetary compensation, though I expect this wouldn’t motivate too many people, maybe 1 in 100?
Since you can comment only while allocating credit, you can write hoaxes to actively cause harm, but you actually have to fund (give some of your credit to) the causes that you want to harm.
If the granularity is on the level of „ longevity, corruption, legalization of drugs, mental health, better roads, etc“ I think I don’t expect people to get hoaxy on those. I assume they’d get filtered beforehand to be kind of common-sensical and maybe even boring to think about. Speaking of granularity, I wonder if this would even be minimally enough to distinguish good from lucky forecasters, especially when there are so many participants.
ETA: Right, and congrats on getting the grants approved! Do they come from the Czech government?
don’t they have plenty of that already and further pressures are actually negative if they think they know best?
Yes, but a lot of it seems to be the inputs from the lobby, interest groups, or people who are mostly virtue signaling to their peers, honest citizen participation (citizen assemblies etc.) is not that common… In this case, the government pre-commits to only allocate a small part of the budget accordingly, apart from that, politicians can still do what they think they know best.
Who are the experts? I expect this to cause controversy. … Maybe this could be circumvented by letting the population decide it? Or at least their elected representatives? I‘ve stumbled upon the „Bayesian Truth Serum“ mechanism …
Thanks for sharing! The Nature study that Robin Hanson talks about is pretty relevant. But in our mechanism, the participants are predicting expert consensus (not their own consensus), so we don’t need to make it harder for them to coordinate their answers, we just have to make sure they don’t know who the experts in DELPHI 3-5 years later will be, so that they can´t influence them.
Also, unlike in the Surprising Popularity mechanism, in case you are confident that only you and a few others know the truth, your incentive is not to keep going with the contemporary consensus but actually go with your contrarian opinion, especially when it is likely to become accepted 3-5 years later (and experts should more likely to accept it earlier than the majority of the public).
I’d expect even more „Activists“, more like 95% maybe?
If “what do I support” becomes a socially useful topic to mention to your friends, this social incentive might be more important than the financial incentive for choosing the forecaster strategy. But probably you´re right, there would be less the 30% forecasters.
I assume they’d get filtered beforehand to be kind of common-sensical and maybe even boring to think about. Speaking of granularity, I wonder if this would even be minimally enough to distinguish good from lucky forecasters, especially when there are so many participants.
Right, we need to find the level of granularity between “boring to most” and “too difficult to most”. I think there are already pretty good setups and scoring mechanisms to eliminate luck—like if you forecast in a probability distribution and are rewarded based on how much you have improved the current aggregate. But yes, this needs more research.
Do the grants come from the Czech government?
Yes, the Technological Agency of the Czech Republic.
Thanks, interesting points. Yes, predicting what unknown experts will conclude seems reasonable to me (though NunoSempere’s points also seem sensible). Looking forward to read your next update!
Very cool idea, would love to see this implemented. Some thoughts while reading:
don’t they have plenty of that already and further pressures are actually negative if they think they know best?
FWIW I feel like I rarely observe non-populist politicians running on a direct democracy platform, and the one time I remember they weren’t successful (the German Piratenpartei).
Who are the experts? I expect this to cause controversy. Even without this mechanism I already perceive that on fairly clearcut questions there is a lot of controversy, also in less polarized countries relative to the US, like Germany (e.g. on nuclear energy, GMOs, rent control). Maybe this could be circumvented by letting the population decide it? Or at least their elected representatives? I‘ve stumbled upon the „Bayesian Truth Serum“ mechanism that might useful for eliciting non-measurable outcomes. https://www.overcomingbias.com/2017/01/surprising-popularity.html
Maybe this could be a useful use case of integrating an Elicit forecast like described here. https://forum.effectivealtruism.org/posts/YgbPSyTvft6EcKnWm/forum-update-new-features-december-2020#Elicit_predictions FWIW I’d expect even more „Activists“, more like 95% maybe? Only being able to predict which of ~20 categories some group of „experts“ will find most important feels demotivating to me, even though I do forecasts regularly. This might change with the amount of monetary compensation, though I expect this wouldn’t motivate too many people, maybe 1 in 100?
If the granularity is on the level of „ longevity, corruption, legalization of drugs, mental health, better roads, etc“ I think I don’t expect people to get hoaxy on those. I assume they’d get filtered beforehand to be kind of common-sensical and maybe even boring to think about. Speaking of granularity, I wonder if this would even be minimally enough to distinguish good from lucky forecasters, especially when there are so many participants.
ETA: Right, and congrats on getting the grants approved! Do they come from the Czech government?
Yes, but a lot of it seems to be the inputs from the lobby, interest groups, or people who are mostly virtue signaling to their peers, honest citizen participation (citizen assemblies etc.) is not that common… In this case, the government pre-commits to only allocate a small part of the budget accordingly, apart from that, politicians can still do what they think they know best.
Thanks for sharing! The Nature study that Robin Hanson talks about is pretty relevant. But in our mechanism, the participants are predicting expert consensus (not their own consensus), so we don’t need to make it harder for them to coordinate their answers, we just have to make sure they don’t know who the experts in DELPHI 3-5 years later will be, so that they can´t influence them.
Also, unlike in the Surprising Popularity mechanism, in case you are confident that only you and a few others know the truth, your incentive is not to keep going with the contemporary consensus but actually go with your contrarian opinion, especially when it is likely to become accepted 3-5 years later (and experts should more likely to accept it earlier than the majority of the public).
If “what do I support” becomes a socially useful topic to mention to your friends, this social incentive might be more important than the financial incentive for choosing the forecaster strategy. But probably you´re right, there would be less the 30% forecasters.
Right, we need to find the level of granularity between “boring to most” and “too difficult to most”. I think there are already pretty good setups and scoring mechanisms to eliminate luck—like if you forecast in a probability distribution and are rewarded based on how much you have improved the current aggregate. But yes, this needs more research.
Yes, the Technological Agency of the Czech Republic.
Thanks, interesting points. Yes, predicting what unknown experts will conclude seems reasonable to me (though NunoSempere’s points also seem sensible). Looking forward to read your next update!