I think this description is incomplete. By ~late April/May, in direct comparisons, generalist predictors on Metaculus and GJ Open generally outpredicted epidemiologist’s surveys for forecasts 1 week out. This is relevant because there was no way for changes due to new gov’t interventions or behavioral changes in the future to show up in case statistics in less than 7 days.
It takes several days for infections to get symptoms, a few days for people to get tested, and another X days (decreases as pandemic progressed) for tests to resolve.
So a question like “how many cases/deaths in 6 days” can almost be entirely decomposed to your distributions on
- how many infected *now*,
- how testing will change in a week
- probability of data updates.
The latter two does incorporate some elements of politics and behaviorial change. However, “amateur forecasters better at predicting political trends” can’t be the full story, because in a lot of weeks data updates/testing didn’t change *that much*, so amateur predictors essentially had a better internal model of how many ppl currently infected.
I messaged Khorton on Twitter, but just paraphrasing what I said here: