FRI has informed decisions on frontier AI companiesā capability scaling policies
Their scaling policies are not very good (or are ignored in favor of profits and increased scaling) so I donāt see how this is a win for forecasting. Unless youāre saying they would be even worse without FRI, which I donāt think is true (theyād probably behave the same regardless).
Superforecasters tend to believe x-risk isnāt a big deal. Regardless of whether theyāre using reasonable procedures, theyāre getting the wrong object-level answer in this case. FRIās consulting plausibly made the scaling policies worse. Hard to say without knowing more details.
(Iām thinking particular of XPT, which is from 2023 so it may be outdated at this point. But that tournament had superforecasters predicting only a 1% chance of AI extinction, which is ludicrous and should not be used as the basis for decisions.)
Sharing what context Iām able to: Our work in this space so far has mostly been around assessing both risks and effective safeguards for AI-biorisk and AI-cybersecurity risk.
Superforecasters and domain experts tend to be relatively aligned on these topics so far (e.g., see this study as one example, and a later update here). (Weāve completed private research on AI-cybersecurity risk and will be publishing some of it soon.)
Their scaling policies are not very good (or are ignored in favor of profits and increased scaling) so I donāt see how this is a win for forecasting. Unless youāre saying they would be even worse without FRI, which I donāt think is true (theyād probably behave the same regardless).
Superforecasters tend to believe x-risk isnāt a big deal. Regardless of whether theyāre using reasonable procedures, theyāre getting the wrong object-level answer in this case. FRIās consulting plausibly made the scaling policies worse. Hard to say without knowing more details.
(Iām thinking particular of XPT, which is from 2023 so it may be outdated at this point. But that tournament had superforecasters predicting only a 1% chance of AI extinction, which is ludicrous and should not be used as the basis for decisions.)
Sharing what context Iām able to: Our work in this space so far has mostly been around assessing both risks and effective safeguards for AI-biorisk and AI-cybersecurity risk.
Superforecasters and domain experts tend to be relatively aligned on these topics so far (e.g., see this study as one example, and a later update here). (Weāve completed private research on AI-cybersecurity risk and will be publishing some of it soon.)