It’s indeed the case that GJP was 34.7% better than the ICPM. But it’s not the case that GJP participants were 34.7% better than intelligence analysts. The intelligent analyst used prediction markets that are generally worse than prediction pools (see Appendix A), so we are not comparing apples to apples.
It would be fair to judge IC for using prediction markets rather than prediction pools after seeing research coming out of GJP. But we don’t know how an intelligence analyst prediction pool would perform compared to the GJP prediction pool. We have reasons to believe that difference might not be that impressive based on ICPM vs GJP PM and based on Sell et al (2021).
It’s indeed the case that GJP was 34.7% better than the ICPM. But it’s not the case that GJP participants were 34.7% better than intelligence analysts. The intelligent analyst used prediction markets that are generally worse than prediction pools (see Appendix A), so we are not comparing apples to apples.
It would be fair to judge IC for using prediction markets rather than prediction pools after seeing research coming out of GJP. But we don’t know how an intelligence analyst prediction pool would perform compared to the GJP prediction pool. We have reasons to believe that difference might not be that impressive based on ICPM vs GJP PM and based on Sell et al (2021).