Training data: I agree that the stuff you’re pointing to seems worthwhile. But I feel like you’ve latched onto a particular type of training data, and you’re missing important categories, e.g.:
Epistemics stuff—there are lots of super smart people earnestly trying to figure out very hard questions, and I think that if you could access their thinking, there would be a lot there which would compare favourably to a lot of the data that would be collected from people in this community. It wouldn’t be so targeted in terms of the questions it addressed (e.g. “AI strategy”, but learning good epistemics may be valuable and transfer over)
International negotiation, and high-stakes bargaining in general—potentially very important, but not something I think our community has any particular advantage at
Synthetic data—a bunch of things may be unlocked more by working out how to enable “self-play” (or the appropriate analogue), rather than just collecting more data the hard way
Yeah I think I agree with all this; I suppose since ‘we’ have the AI policy/strategy training data anyway that seems relatively low effort and high value to do, but yes if we could somehow get access to the private notes of a bunch of international negotiators that also seems very valuable! Perhaps actually asking top forecasters to record their working and meetings to use as training data later would be valuable, and I assume many people already do this by default (tagging @NunoSempere). Although of course having better forecasting AIs seems more dual-use than some of the other AI tools.
Training data: I agree that the stuff you’re pointing to seems worthwhile. But I feel like you’ve latched onto a particular type of training data, and you’re missing important categories, e.g.:
Epistemics stuff—there are lots of super smart people earnestly trying to figure out very hard questions, and I think that if you could access their thinking, there would be a lot there which would compare favourably to a lot of the data that would be collected from people in this community. It wouldn’t be so targeted in terms of the questions it addressed (e.g. “AI strategy”, but learning good epistemics may be valuable and transfer over)
International negotiation, and high-stakes bargaining in general—potentially very important, but not something I think our community has any particular advantage at
Synthetic data—a bunch of things may be unlocked more by working out how to enable “self-play” (or the appropriate analogue), rather than just collecting more data the hard way
Yeah I think I agree with all this; I suppose since ‘we’ have the AI policy/strategy training data anyway that seems relatively low effort and high value to do, but yes if we could somehow get access to the private notes of a bunch of international negotiators that also seems very valuable! Perhaps actually asking top forecasters to record their working and meetings to use as training data later would be valuable, and I assume many people already do this by default (tagging @NunoSempere). Although of course having better forecasting AIs seems more dual-use than some of the other AI tools.