Theres a very interesting passage in here, showing that asking the public extinction questions in terms of odds rather than percentages made the estimates of risk six orders of magnitude lower:
Participants in our public survey of 912 college graduates estimated a higher median probability of extinction by 2100 (5%) than superforecasters (1%) but lower than that of experts (6%). A similar pattern also emerged for AI-caused extinction (public survey participants gave a 2% probability, and superforecasters and domain experts gave 0.38% and 3%, respectively).
However, respondents of the same sample estimated much lower chances of both extinction and catastrophe by 2100 when presented with an alternative elicitation method. In a follow-up survey, we gave participants examples of low probability events—for example, that there is a 1-in-300,000 chance of being killed by lightning. We then asked them to fill in a value for “X” such that there was a “1-in-X” chance of a given risk—like human extinction by 2100.69 69 The set of reference classes we gave to participants had ten examples, including: 1 in 2: Probability a flip of a fair coin will be Tails 1 in 300,000: Lifetime probability of dying from lightning 1 in 10,000,000: Probability a random newborn becomes a U.S. president
Using that method, the median probability of humanity’s extinction before 2100 was 1 in 15 million. The median probability of AI-caused extinction before 2100 was 1 in 30 million.
This seems like incredibly strong evidence of anchoring bias in answers to this question, and it goes both ways (if you see percentages, you anchor to the 1-99 range). What I want to know is whether this effect would show up for superforecasters and domain experts as well. Are the high risk estimates of AI extinction merely an anchoring bias artifact?
1 in 2: Probability a flip of a fair coin will be Tails 1 in 300,000: Lifetime probability of dying from lightning 1 in 10,000,000: Probability a random newborn becomes a U.S. president
Hmm, naively the anchoring bias from having many very low-probability examples would be larger than the anchoring bias from using odds vs probabilities.
Happy to make a small bet here in case FRI or others run followup studies.
I would expect the effect to persist even with minimal examples. In everyday life when, we encounter probability in terms of odds, it’s in the context of low probabilities ( like the chances of winning the lottery being 1 in a million), whereas when we encounter percentage probabilities it’s usually regarding events in the 5-95% probability range, like whether one party will win an election.
Just intuitively, saying there is a 0.1% chance of extinction feels like a “low” estimate, whereas saying there is a 1 in a thousand chance of extinction feels like a high estimate, even though they both refer to the exact same probability. I think there a subset of people who want to say “AI extinction is possible, but extremely unlikely”, and are expressing this opinion with wildly different numbers depending on whether asked in terms of odds or percentages.
(I’m an FRI employee, but responding here in my personal capacity.)
Yeah, in general we thought about various types of framing effects a lot in designing the tournament, but this was one we hadn’t devoted much time to. I think we were all pretty surprised by the magnitude of the effect in the public survey.
Personally, I think this likely affected our normal tournament participants less than it did members of the public. Our “expert” sample mostly had considered pre-existing views on the topics, so there was less room for the elicitation of their probabilities to affect things. And superforecasters should be more fluent in probabilistic reasoning than educated members of the public, so should be less caught out by probability vs. odds.
In any case, forecasting low probabilities is very little studied, and an FRI project to remedy that is currently underway.
I agree, in that I predict that the effect would be lessened for experts and lessened still more for superforecasters.
However, that doesn’t tell us how much less. A six order of magnitude discrepancy leaves a lot of room! If switching to odds only dropped superforecasters by three orders of magnitude and experts by four orders of magnitude, everything you said above would be true, but it would still make a massive difference to risk estimates. The people in EA may already have a (P|Doom) before going in, but everyone else won’t. Being an AI expert does not make one immune to anchoring bias.
I think it’s very important to follow up on this for domain experts. I often see “median AI expert thinks thinks there is 2% chance AI x-risk” used as evidence to take AI risk seriously, but is there an alternate universe where the factoid is “median AI expert thinks there is 1 in 50,000 odds of AI x-risk” ? We really need to find out.
Theres a very interesting passage in here, showing that asking the public extinction questions in terms of odds rather than percentages made the estimates of risk six orders of magnitude lower:
This seems like incredibly strong evidence of anchoring bias in answers to this question, and it goes both ways (if you see percentages, you anchor to the 1-99 range). What I want to know is whether this effect would show up for superforecasters and domain experts as well. Are the high risk estimates of AI extinction merely an anchoring bias artifact?
Hmm, naively the anchoring bias from having many very low-probability examples would be larger than the anchoring bias from using odds vs probabilities.
Happy to make a small bet here in case FRI or others run followup studies.
I would expect the effect to persist even with minimal examples. In everyday life when, we encounter probability in terms of odds, it’s in the context of low probabilities ( like the chances of winning the lottery being 1 in a million), whereas when we encounter percentage probabilities it’s usually regarding events in the 5-95% probability range, like whether one party will win an election.
Just intuitively, saying there is a 0.1% chance of extinction feels like a “low” estimate, whereas saying there is a 1 in a thousand chance of extinction feels like a high estimate, even though they both refer to the exact same probability. I think there a subset of people who want to say “AI extinction is possible, but extremely unlikely”, and are expressing this opinion with wildly different numbers depending on whether asked in terms of odds or percentages.
Yeah this is plausible but my intuitions go the other way. Would be interested in a replication that looks like
50% Probability a flip of a fair coin will be Tails
0.0003%: Lifetime probability of dying from lightning
0.00001%: Probability a random newborn becomes a U.S. president
vs
1 in 2: Probability a flip of a fair coin will be Tails
1 in 6: Probability a fair die will land on 5
1 in 9: Probability that in a room with 10 people, 2 of them have the same birthday.
1 in 14: Probability a randomly selected adult in America self-identifies as lesbian, gay, bisexual, or transgender
1 in 100: lifetime risk of dying from a car accident
I would also find this experiment interesting!
(I’m an FRI employee, but responding here in my personal capacity.)
Yeah, in general we thought about various types of framing effects a lot in designing the tournament, but this was one we hadn’t devoted much time to. I think we were all pretty surprised by the magnitude of the effect in the public survey.
Personally, I think this likely affected our normal tournament participants less than it did members of the public. Our “expert” sample mostly had considered pre-existing views on the topics, so there was less room for the elicitation of their probabilities to affect things. And superforecasters should be more fluent in probabilistic reasoning than educated members of the public, so should be less caught out by probability vs. odds.
In any case, forecasting low probabilities is very little studied, and an FRI project to remedy that is currently underway.
I agree, in that I predict that the effect would be lessened for experts and lessened still more for superforecasters.
However, that doesn’t tell us how much less. A six order of magnitude discrepancy leaves a lot of room! If switching to odds only dropped superforecasters by three orders of magnitude and experts by four orders of magnitude, everything you said above would be true, but it would still make a massive difference to risk estimates. The people in EA may already have a (P|Doom) before going in, but everyone else won’t. Being an AI expert does not make one immune to anchoring bias.
I think it’s very important to follow up on this for domain experts. I often see “median AI expert thinks thinks there is 2% chance AI x-risk” used as evidence to take AI risk seriously, but is there an alternate universe where the factoid is “median AI expert thinks there is 1 in 50,000 odds of AI x-risk” ? We really need to find out.