I think there’s some evidence that Metaculus, while a group of fairly smart and well-informed people, are nowhere near as knowledgeable as a fairly informed EA (perhaps including a typical user of this forum?) for the specific questions around existential and global catastrophic risks.
One example I can point to is that for this question on climate change and GCR before 2100 (that has been around since October 2018), a single not-very-informative comment from me was enough to change the community median from 24% to 10%. This suggests to me that Metaculus users did not previously have strong evidence or careful reasoning on this question, or perhaps GCR-related thinking in general.
Now you might think that actual superforecasters are better, but based on the comments given so far for COVID-19, I’m unimpressed. In particular the selected comments point to use of reference classes that EAs and avid Metaculus users have known to be flawed for over a week before the report came out (eg, using China’s low deaths as evidence that this can be easily replicated in other countries as the default scenario).
Now COVID-19 is not an existential risk or GCR, but it is an “out of distribution” problem showing clear and fast exponential growth that seems unusual for most questions superforecasters are known to excel at.
I think there’s some evidence that Metaculus, while a group of fairly smart and well-informed people, are nowhere near as knowledgeable as a fairly informed EA (perhaps including a typical user of this forum?) for the specific questions around existential and global catastrophic risks.
One example I can point to is that for this question on climate change and GCR before 2100 (that has been around since October 2018), a single not-very-informative comment from me was enough to change the community median from 24% to 10%. This suggests to me that Metaculus users did not previously have strong evidence or careful reasoning on this question, or perhaps GCR-related thinking in general.
Now you might think that actual superforecasters are better, but based on the comments given so far for COVID-19, I’m unimpressed. In particular the selected comments point to use of reference classes that EAs and avid Metaculus users have known to be flawed for over a week before the report came out (eg, using China’s low deaths as evidence that this can be easily replicated in other countries as the default scenario).
Now COVID-19 is not an existential risk or GCR, but it is an “out of distribution” problem showing clear and fast exponential growth that seems unusual for most questions superforecasters are known to excel at.