To figure out whether it tends to help or hurt in general, it would be better to get a lot more examples. It turns out, though, that this is a very common tool for people to use when estimating the efficacy of a conversion funnel. You figure out what the steps are, get estimates for each step, and that gives you an overall conversion rate. These aren’t perfect, but they do pretty well, and they do a lot better than trying to estimate a conversion rate without breaking it down.
Are there other examples of people using this method, yielding success or failure?
In the discussion people gave a few other examples of people using this sort of model:
Still, this isn’t very much. Before deciding whether this is a method that tends to lead people to give bad estimates (relative to whatever they would use instead) I’d like to see more examples?
I think canonicalizing this as a Fallacy was very premature: Yudkowsky wrote his post based on two examples:
Nate Silver’s Trump’s Six Stages of Doom (which got a very-clearly-wrong-in-hindsight answer)
My Breaking Down Cryonics Probabilities (which is harder to evaluate since it’s about something farther in the future)
I wrote a response at the time, ending with:
In the discussion people gave a few other examples of people using this sort of model:
Robin Hanson in Break Cryonics Down
Forecasters in the Good Judgement project (comment)
Animal Charity Evaluators trying to predict the power of a planned behavior change study.
Still, this isn’t very much. Before deciding whether this is a method that tends to lead people to give bad estimates (relative to whatever they would use instead) I’d like to see more examples?