The Parable of the Boy Who Cried 5% Chance of Wolf

[Kat Woods 🔶 ⏸️]

Epistemic status: a parable making a moderately strong claim about statistics

Once upon a time, there was a boy who cried, “there’s a 5% chance there’s a wolf!”

The villagers came running, saw no wolf, and said “He said there was a wolf and there was not. Thus his probabilities are wrong and he’s an alarmist.”

On the second day, the boy heard some rustling in the bushes and cried “there’s a 5% chance there’s a wolf!”

Some villagers ran out and some did not.

There was no wolf.

The wolf-skeptics who stayed in bed felt smug.

“That boy is always saying there is a wolf, but there isn’t.”

“I didn’t say there was a wolf!” cried the boy. “I was estimating the probability at low, but high enough. A false alarm is much less costly than a missed detection when it comes to dying! The expected value is good!”

The villagers didn’t understand the boy and ignored him.

On the third day, the boy heard some sounds he couldn’t identify but seemed wolf-y. “There’s a 5% chance there’s a wolf!” he cried.

No villagers came.

It was a wolf.

They were all eaten.

Because the villagers did not think probabilistically.

The moral of the story is that we should expect to have a large number of false alarms before a catastrophe hits and that is not strong evidence against impending but improbable catastrophe.

Each time somebody put a low but high enough probability on a pandemic being about to start, they weren’t wrong when it didn’t pan out. H1N1 and SARS and so forth didn’t become global pandemics. But they could have. They had a low probability, but high enough to raise alarms.

The problem is that people then thought to themselves “Look! People freaked out about those last ones and it was fine, so people are terrible at predictions and alarmist and we shouldn’t worry about pandemics”

And then COVID-19 happened.

This will happen again for other things.

People will be raising the alarm about something, and in the media, the nuanced thinking about probabilities will be washed out.

You’ll hear people saying that X will definitely fuck everything up very soon.

And it doesn’t.

And when the catastrophe doesn’t happen, don’t over-update.

Don’t say, “They cried wolf before and nothing happened, thus they are no longer credible.”

Say “I wonder what probability they or I should put on it? Is that high enough to set up the proper precautions?”

When somebody says that nuclear war hasn’t happened yet despite all the scares, when somebody reminds you about the AI winter where nothing was happening in it despite all the hype, remember the boy who cried a 5% chance of wolf.

Originally posted on my Twitter and personal blog.

Reminder that if this reaches 25 upvotes, you can listen to this post on your podcast player using the Nonlinear Library.

[weeatquince🔸]

As I understand it, counter-terrorism risk management trends to work by having ongoing monitoring of potential theats combined with a pre-agreed escalation (alert levels) and response protocol to ensure propotionate action to prevent (or mitigate) those threats.

This seems to provide a template solution to this issue.

I’d be keen for more policy advocacy for governments to implement similar protocol mechanisms for bio threats (and also for EA researchers to work with risk managers and try mapping risks along these lines).

[christian.r]

Thanks for writing this! I think it’s great. Reminds me of another wild animal metaphor about high-stakes decision-making under uncertainty—Reagan’s 1984 “Bear in the Woods” campaign ad:

There is a bear in the woods. For some people, the bear is easy to see. Others don’t see it at all. Some people say the bear is tame. Others say it’s vicious and dangerous. Since no one can really be sure who’s right, isn’t it smart to be as strong as the bear—if there is a bear?

I think that kind of reasoning is helpful when communicating about GCRs and X-risks.

[Jeroen Willems🔸]

What is the reasoning behind having the probability increase each time? It might be more interesting if the probability stayed at 5% each time. Because now you might get the conclusion “only once the odds are significantly high, say at 15%, should we start worrying”.

[Kat Woods 🔶 ⏸️]

Great point! I didn’t give it much thought, honestly. I think you’re right and saying 5% each time is better. Gonna update it now.

Thanks for the suggestion!

[MichaelPlant]

Not sure if this is essential to the parable, but wouldn’t it be useful to distinguish between the following cases?:

(1) the boy says every day there’s 5% chance the wolf could come every evening, but isn’t saying the wolf is there right now

and

(2) the boy says there is a wolf in the village whenever he thinks there’s a 5+% chance this is true.

If the boy is doing (1) and the villagers panic now then they’ve just misunderstood what he’s saying. If the boy is doing (2), then you’d understand why the villagers would start ignoring the boy (just like everyone ignores car alarms because they are so oversensitive).

I’m not sure either is neatly analogous to the X-risk case, which is that different people give different estimates and these range from negligible to doom being apparently virtually certain. I guess that’s a bit like there being many different boys in the village, each of whom assigns a different percentage chance to the wolf appearing at some point (but none of whom are claiming it’s literally here now).

[Karthik Tadepalli]

Related: A Failure, But Not of Prediction. The best case for x-risk reduction I’ve ever read, and it doesn’t even mention x-risks once.

[Lizka]

Thanks for linking the post — I think it’s really great.

Note also, a comment from the post:

There once was a shepherd boy who lived near a small town. One day, he heard rustling in the trees and thought it had a 10% chance of being a wolf, and did nothing, and nothing came of it. On another day, he once again heard rustling in the trees and thought it had a 10% chance of being a wolf, and did nothing, and nothing came of it. On a third day, he once again heard rustling in the trees and thought it had a 10% chance of being a wolf, and did nothing, and he and all of his flock were eaten by the wolf.

There once was a shepherd boy who lived near a small town. One day, he heard rustling in the trees and thought it had a 10% chance of being a wolf, and cried out “Wolf!,” and all the neighbors came to help but found no wolf. On another day, he heard rustling in the trees and thought it had a 10% chance of being a wolf, and cried out “Wolf!,” and all the neighbors came to help but found no wolf. On a third day, he once again heard rustling in the trees and thought it had a 10% chance of being a wolf, and cried out “Wolf!,” but none of his neighbors came to help because they thought there would be no wolf, and he and all of his flock were eaten by the wolf.

[Jack R]

Relatedly: Heuristics That Almost Always Work