Error

Unrecognized LW server error:

Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?

Vasco Grilo🔸Mar 28, 2024, 3:17 PM
2 points
0 ∶ 0

Hi Rose,
The values for the standard deviation of the AI extinction risk seem to high. For example, the median and maximum AI extinction risk until the end of 2030 by superforecasters are 10^-6 and 10 % (pp. 269 and 272), and therefore the standard deviation has to be lower than 10 % (= 0.1 − 10^-6), but you report a value of 2.6 (p. 269). Maybe 2.6 is the standard deviation as a fraction of the mean (i.e. the coefficient of variation)?
- Molly Hickman Mar 29, 2024, 8:20 PM
  3 points
  0 ∶ 0
  Parent
  
  Hi @Vasco Gril, thanks for the question. That is the standard deviation in percentage points. The distribution is decidedly un-Gaussian so the standard deviation is a little misleading.
  We limited the y axis range on the box-and-dot plots like that one on page 272 -- they’re all truncated at the 95th percentile of tournament participants + a 5% cushion (footnote on page 18) -- so the max for Stage 1 for supers was actually 21.9%.
  Here are a couple more summary stats for the superforecasters, for the 2030 question. The raw data are available here if you want to explore in more detail!
```
  stage count   mean    sd   min  median   max
  <int> <int>  <dbl> <dbl> <dbl>   <dbl> <dbl>
1     1    88 0.510  2.60      0 0.0001   21.9
2     2    57 0.378  1.74      0 0.0001   12  
3     3    16 0.0392 0.125     0 0.00075   0.5
4     4    69 0.180  1.20      0 0.0001   10  
```
  - Vasco Grilo🔸Mar 29, 2024, 9:04 PM
    2 points
    0 ∶ 0
    Parent
    
    Thanks for clarifying, and sharing the data, Molly!
    That is the standard deviation in percentage points.
    I thought it was something else because you have “%” after the medians, but no “pp” after the standard deviations. For future occasions, you could add “pp” either after the standard deviations or in the headers.
    Here are a couple more summary stats for the superforecasters, for the 2030 question.
    I am surprised to see the minimum AI extinction risk until the end of 2030 is 0 for all stages. I wonder whether the values were rounded, or you had discrete options for the values which could be inputted and some forecasters selected 0 as the closest value (in a linear scale) to their best guess. I think superforecasters predicting an astronomically low extinction risk would be fine, but guessing a value of exactly 0 would be a pretty bad sign, as one cannot be infinitely confident humans will not go extinct.
spreadlove5683 Jul 25, 2023, 12:56 PM
2 points
0 ∶ 0

Can someone give me the TLDR on the implications of these results in light of the fact that Samotsvety’s group seemingly/perhaps had much higher odds for AI catastrophe? I didn’t read the exact definitions they used for catastrophe, but:

Samotsvety’s group (n=13) gave “What’s your probability of misaligned AI takeover by 2100, barring pre-APS-AI catastrophe?” at 25%
(source https://forum.effectivealtruism.org/posts/EG9xDM8YRz4JN4wMN/samotsvety-s-ai-risk-forecasts)

Whereas XPT gave
”AI Catastrophic risk (>10% of humans die within 5 years)” for year 2100 at 2.13%

Without having read the exact definitions for “misaligned AI takeover” and still knowing that Samotsvety’s prediction was conditional on pre-APS-AI catastrophe not happening, this still seems like a very large discrepancy. I know that Samotsvety’s group was a much smaller n. n=13 vs n=88. How much weight should we give to Samotsvety’s group’s other predictions on AI timelines given the discrepancy in the risk prediction likelihoods?
- rosehadshar Jul 25, 2023, 2:29 PM
  4 points
  1 ∶ 0
  Parent
  
  Good question.
  There’s a little bit on how to think about the XPT results in relation to other forecasts here (not much). Extrapolating from there to Samotsvety in particular:
  - Reasons to favour XPT (superforecaster) forecasts:
    Larger sample size
    The forecasts were incentivised (via reciprocal scoring, a bit more detail here)
    The most accurate XPT forecasters in terms of reciprocal scoring also gave the lowest probabilities on AI risk (and reciprocal scoring accuracy may correlate with actual accuracy)
  - Speculative reasons to favour Samotsvety forecasts:
    (Guessing) They’ve spent longer on average thinking about it
    (Guessing) They have deeper technical expertise than the XPT superforecasters
  I also haven’t looked in detail at the respective resolution criteria, but at first glance the forecasts also seem relatively hard to compare directly. (I agree with you though that the discrepancy is large enough that it suggests a large disagreement were the two groups to forecast the same question—just expect that it will be hard to work out how large.)