As someone who is one of the most active users of Metaculus (particularly 2 months ago), I find a lot of the Metaculus estimates on existential risk pretty suspicious:
There’s no incentive to do well on those questions.
The feedback loops are horrible
Indeed, some people have actually joked betting low on the more existential questions since they won’t get a score if we’re all dead (at least, I hope they’re joking)
At the object-level, I just think people are really poorly calibrated about x-risk questions
My comment here arguably changed the community’s estimates by ~10%
It wasn’t a very insightful comment by EA standards
People are not well-trained on low probabilities
Both that they literally have no way of doing so on Metaculus (can’t go <1% or >99%) and that they haven’t predicted enough questions to be calibrated at those levels.
My inside view is that people on Metaculus are more well-read than most people on existential risk matters, but probably on average less well-read than, eg, your typical organizer of a EA local or college group.
My guess is that as of summer 2020, there’s no strong reason to think Metaculus will do better than a group of well-read EAs on existential risk questions.
As another very active user of metaculus recently, who’s come from an EA background, I basically agree with all of the above, except that I don’t think all the users are joking about betting low on the existential questions.
“Indeed, some people have actually joked betting low on the more existential questions since they won’t get a score if we’re all dead (at least, I hope they’re joking)”
I think several of them aren’t joking, they care more about feeling “smart” because they’ve gamed the system than the long-term potential consequences of inaccurate forecasts.
I do, like you, really hope that I’m wrong and they are in fact joking.
Thanks, this is quite useful. I hadn’t considered the issue of incentives sufficiently before, and the OP and your comment make me put less weight on the Metaculus x-risk forecasts than I did previously.
(Though I didn’t put a lot of absolute weight on them, and I can’t think of any decision or downstream discussion that would be significantly affected by the update on Metaculus.)
Will MacAskill mentioned in this comment that he’d ‘expect that, say, a panel of superforecasters, after being exposed to all the arguments, would be closer to my view than to the median FHI view.’
You’re a good forecaster right? Does it seem right to you that a panel of good forecasters would come to something like Will’s view, rather than the median FHI view?
I’m not sure. I mentioned as a reply to that comment that I was unimpressed with the ability of existing “good” forecasters to think about low-probability and otherwise out-of-distribution problems. My guess is that they’d change their minds if “exposed” to all the arguments, and specifically have views very close to the median FHI view, if “exposed” → reading the existing arguments very carefully and put lots of careful thought into them. However, I think this is a very tough judgement call, and does seem like the type of thing that’d be really bad if we get it wrong!
My beliefs here are also tightly linked to me thinking that the median FHI view is more likely to be correct than Will’s view, and it is a well-known bias that people think their views are more common/correct than they actually are.
As someone who is one of the most active users of Metaculus (particularly 2 months ago), I find a lot of the Metaculus estimates on existential risk pretty suspicious:
There’s no incentive to do well on those questions.
The feedback loops are horrible
Indeed, some people have actually joked betting low on the more existential questions since they won’t get a score if we’re all dead (at least, I hope they’re joking)
At the object-level, I just think people are really poorly calibrated about x-risk questions
My comment here arguably changed the community’s estimates by ~10%
It wasn’t a very insightful comment by EA standards
People are not well-trained on low probabilities
Both that they literally have no way of doing so on Metaculus (can’t go <1% or >99%) and that they haven’t predicted enough questions to be calibrated at those levels.
My inside view is that people on Metaculus are more well-read than most people on existential risk matters, but probably on average less well-read than, eg, your typical organizer of a EA local or college group.
My guess is that as of summer 2020, there’s no strong reason to think Metaculus will do better than a group of well-read EAs on existential risk questions.
As another very active user of metaculus recently, who’s come from an EA background, I basically agree with all of the above, except that I don’t think all the users are joking about betting low on the existential questions.
I find it hard to parse where our disagreement is. Can you point me to what quote you disagree with?
“Indeed, some people have actually joked betting low on the more existential questions since they won’t get a score if we’re all dead (at least, I hope they’re joking)”
I think several of them aren’t joking, they care more about feeling “smart” because they’ve gamed the system than the long-term potential consequences of inaccurate forecasts.
I do, like you, really hope that I’m wrong and they are in fact joking.
Thanks, this is quite useful. I hadn’t considered the issue of incentives sufficiently before, and the OP and your comment make me put less weight on the Metaculus x-risk forecasts than I did previously.
(Though I didn’t put a lot of absolute weight on them, and I can’t think of any decision or downstream discussion that would be significantly affected by the update on Metaculus.)
Thanks for the answer.
Will MacAskill mentioned in this comment that he’d ‘expect that, say, a panel of superforecasters, after being exposed to all the arguments, would be closer to my view than to the median FHI view.’
You’re a good forecaster right? Does it seem right to you that a panel of good forecasters would come to something like Will’s view, rather than the median FHI view?
I’m not sure. I mentioned as a reply to that comment that I was unimpressed with the ability of existing “good” forecasters to think about low-probability and otherwise out-of-distribution problems. My guess is that they’d change their minds if “exposed” to all the arguments, and specifically have views very close to the median FHI view, if “exposed” → reading the existing arguments very carefully and put lots of careful thought into them. However, I think this is a very tough judgement call, and does seem like the type of thing that’d be really bad if we get it wrong!
My beliefs here are also tightly linked to me thinking that the median FHI view is more likely to be correct than Will’s view, and it is a well-known bias that people think their views are more common/correct than they actually are.