(I’m an FRI employee, but responding here in my personal capacity.)
Yeah, in general we thought about various types of framing effects a lot in designing the tournament, but this was one we hadn’t devoted much time to. I think we were all pretty surprised by the magnitude of the effect in the public survey.
Personally, I think this likely affected our normal tournament participants less than it did members of the public. Our “expert” sample mostly had considered pre-existing views on the topics, so there was less room for the elicitation of their probabilities to affect things. And superforecasters should be more fluent in probabilistic reasoning than educated members of the public, so should be less caught out by probability vs. odds.
In any case, forecasting low probabilities is very little studied, and an FRI project to remedy that is currently underway.
(I’m an FRI employee, but responding here in my personal capacity.)
Yeah, in general we thought about various types of framing effects a lot in designing the tournament, but this was one we hadn’t devoted much time to. I think we were all pretty surprised by the magnitude of the effect in the public survey.
Personally, I think this likely affected our normal tournament participants less than it did members of the public. Our “expert” sample mostly had considered pre-existing views on the topics, so there was less room for the elicitation of their probabilities to affect things. And superforecasters should be more fluent in probabilistic reasoning than educated members of the public, so should be less caught out by probability vs. odds.
In any case, forecasting low probabilities is very little studied, and an FRI project to remedy that is currently underway.