Yeah, difficult question… some random thoughts, not sure how helpful:
for paths where a computational understanding of human cognition is especially useful, expertise with Bayesian modeling of cognition might be promising (and seems already popular)
relatedly, I have the impression that human cognition does things that would require current approaches to become more Bayesian… e.g. like the things that are worked on under active learning in ML, and models of perception related to active inference in neuroscience
this might imply, that we’ll (want to) build AI systems that go more into this direction
this might also imply, that there are approximations of Bayesian cognition that scale?
something something developing AI systems that make explicit predictions about explicit future scenarios might be one of the key applications of highly advanced AI systems and the more “formal” this is, the easier to align?
you likely talked to him already, but I understood that David Lindner at ETH Zürich does something related to value learning using approaches analogous to Bayesian optimization(?)
I also wondered how dependend the final design of TAI will be on the current architectures we have today. Even assuming we’ll see TAI in less than 20 years, maybe what ends up happening is that TAI will already be largely designed by AI systems? And then maybe more Bayesian architectures are totally on the table if we think it would be helpful?
maybe a somewhat useful question to ask: In case AI Safety researchers had a headstart that amounts to multiple years, would making the AI more Bayesian be among the most promising ways to use the headstart?
Yeah, difficult question… some random thoughts, not sure how helpful:
for paths where a computational understanding of human cognition is especially useful, expertise with Bayesian modeling of cognition might be promising (and seems already popular)
relatedly, I have the impression that human cognition does things that would require current approaches to become more Bayesian… e.g. like the things that are worked on under active learning in ML, and models of perception related to active inference in neuroscience
this might imply, that we’ll (want to) build AI systems that go more into this direction
this might also imply, that there are approximations of Bayesian cognition that scale?
something something developing AI systems that make explicit predictions about explicit future scenarios might be one of the key applications of highly advanced AI systems and the more “formal” this is, the easier to align?
you likely talked to him already, but I understood that David Lindner at ETH Zürich does something related to value learning using approaches analogous to Bayesian optimization(?)
I also wondered how dependend the final design of TAI will be on the current architectures we have today. Even assuming we’ll see TAI in less than 20 years, maybe what ends up happening is that TAI will already be largely designed by AI systems? And then maybe more Bayesian architectures are totally on the table if we think it would be helpful?
maybe a somewhat useful question to ask: In case AI Safety researchers had a headstart that amounts to multiple years, would making the AI more Bayesian be among the most promising ways to use the headstart?