My understanding of these “reasoning” approaches is that they seem to work very well on problems where there is a well-defined correct answer, and where that can be automatically verified. And it seems reasonable to expect much progress in that area.
What is the thinking of how much of human reasoning work is to do with problems like these?
As a counter-example, in my own particular work on climate prediction, we do not get rapid feedback about what works well, and it is contested what methods and frameworks we should even use i.e. it’s not possible presently to say “getting a good answer just requires solving [list of well-defined problems]” (except making computers so fast that we can do pretty much exact simulations of physics). So it doesn’t seem clear to me that these reasoning models will get a lot better at that kind of thing. But this is perhaps towards the far end of the spectrum of complex problems.
I can see these reasoning models becoming very good at things like writing code where requirements to be met can be precisely specified and automatically verified, and improving performance of devices (such as computer chips) according to well-specified benchmarks. How much difference would it make to make fast progress on problems similar to these?
There doesn’t look to me to be a reason to think that systems trained this way will yield impressive performance at solving messier problems without clear right answers, like predicting complex systems (that can’t be observed experimentally or simulated very well), selecting amongst decision options with different strengths on multiple criteria, dealing with organisational politics etc. Does that seem fair?
These are genuine questions—I don’t feel I have a good grasp of what kinds of work most of our economy is engaged in...
This is my understanding too – some crucial questions going forward:
How useful are AIs that are mainly good at these verifiable tasks?
How much does getting better at reasoning on these verifiable tasks generalise to other domains? (It seems like at least a bit e.g. o1 improved at law)
How well will reinforcement learning work when applied at scale to areas with weaker reward signals?
My understanding of these “reasoning” approaches is that they seem to work very well on problems where there is a well-defined correct answer, and where that can be automatically verified. And it seems reasonable to expect much progress in that area.
What is the thinking of how much of human reasoning work is to do with problems like these?
As a counter-example, in my own particular work on climate prediction, we do not get rapid feedback about what works well, and it is contested what methods and frameworks we should even use i.e. it’s not possible presently to say “getting a good answer just requires solving [list of well-defined problems]” (except making computers so fast that we can do pretty much exact simulations of physics). So it doesn’t seem clear to me that these reasoning models will get a lot better at that kind of thing. But this is perhaps towards the far end of the spectrum of complex problems.
I can see these reasoning models becoming very good at things like writing code where requirements to be met can be precisely specified and automatically verified, and improving performance of devices (such as computer chips) according to well-specified benchmarks. How much difference would it make to make fast progress on problems similar to these?
There doesn’t look to me to be a reason to think that systems trained this way will yield impressive performance at solving messier problems without clear right answers, like predicting complex systems (that can’t be observed experimentally or simulated very well), selecting amongst decision options with different strengths on multiple criteria, dealing with organisational politics etc. Does that seem fair?
These are genuine questions—I don’t feel I have a good grasp of what kinds of work most of our economy is engaged in...
This is my understanding too – some crucial questions going forward:
How useful are AIs that are mainly good at these verifiable tasks?
How much does getting better at reasoning on these verifiable tasks generalise to other domains? (It seems like at least a bit e.g. o1 improved at law)
How well will reinforcement learning work when applied at scale to areas with weaker reward signals?