I have been considering writing a somewhat technical post arguing that “large Transformer models are shortcut finders” is a more clarifying abstraction for these sorts of artifacts than considering them to be simulators, shoggoths, utility maximizers, etc. Empirical research on the challenges of out-of-distribution generalization, path dependency in training, lottery tickets/winning subnetworks, training set memorization, and other areas appear to lend credence to this as a more reasonable abstraction.
Beyond allowing for a more accurate conception of their function, I believe seeing Transformer models through this lens naturally leads to another conclusion: that the existential risk posed by AI in the near-term, at least as presented within EA and adjacent communities, is likely overblown.
I have been considering writing a somewhat technical post arguing that “large Transformer models are shortcut finders” is a more clarifying abstraction for these sorts of artifacts than considering them to be simulators, shoggoths, utility maximizers, etc. Empirical research on the challenges of out-of-distribution generalization, path dependency in training, lottery tickets/winning subnetworks, training set memorization, and other areas appear to lend credence to this as a more reasonable abstraction.
Beyond allowing for a more accurate conception of their function, I believe seeing Transformer models through this lens naturally leads to another conclusion: that the existential risk posed by AI in the near-term, at least as presented within EA and adjacent communities, is likely overblown.