I wonder if there’s anything useful we can learn from artificial neural networks, especially those trained through reinforcement learning and/or evolved in complex environments. In particular, for reinforcement learning, we get to choose the “learning rate”, i.e. the strength of the reward/punishment signal (or an evolutionary algorithm could choose this for us). One important pattern we use when training neural networks (although I don’t know how common this is with RL) is decreasing the learning rate over time (and now, sometimes cyclical learning rates), and the usual effect of this is to first quickly get the behaviour roughly correct with a large learning rate, and then refine it with a lower learning rate. Continuing to train with a large learning rate can create large swings in behaviour, possibly going too far in one direction at one step and then going too far in the opposite direction in the next, and never really approaching locally optimal behaviour.
And then we could also look into how learning and signal strength interacts with neural architecture, neural network complexity and neural network size.
There are some important disanalogies in the way artificial neural networks work and are typically trained today, e.g., from my understanding,
gradient descent, the favoured algorithm for updating weights of a neural network, is not how we learn at all, and
biological neurons seem functionally more complicated than artificial ones.
There are more realistic attempts at simulating brains, though.
I think this is an interesting idea. Unfortunately, I’m woefully ignorant about the relevant details, so it’s unclear to me if the differences between artificial neural networks and actual brains makes the analogy basically useless. Still, I think it would probably be worthwhile for someone with more specialized knowledge than myself to think through the analogy roughly along the lines you’ve outlined and see what comes of it. I’d be happy to collaborate if anyone (including yourself) wants to take up the task.
I wonder if there’s anything useful we can learn from artificial neural networks, especially those trained through reinforcement learning and/or evolved in complex environments. In particular, for reinforcement learning, we get to choose the “learning rate”, i.e. the strength of the reward/punishment signal (or an evolutionary algorithm could choose this for us). One important pattern we use when training neural networks (although I don’t know how common this is with RL) is decreasing the learning rate over time (and now, sometimes cyclical learning rates), and the usual effect of this is to first quickly get the behaviour roughly correct with a large learning rate, and then refine it with a lower learning rate. Continuing to train with a large learning rate can create large swings in behaviour, possibly going too far in one direction at one step and then going too far in the opposite direction in the next, and never really approaching locally optimal behaviour.
And then we could also look into how learning and signal strength interacts with neural architecture, neural network complexity and neural network size.
There are some important disanalogies in the way artificial neural networks work and are typically trained today, e.g., from my understanding,
gradient descent, the favoured algorithm for updating weights of a neural network, is not how we learn at all, and
biological neurons seem functionally more complicated than artificial ones.
There are more realistic attempts at simulating brains, though.
Hey Michael,
I think this is an interesting idea. Unfortunately, I’m woefully ignorant about the relevant details, so it’s unclear to me if the differences between artificial neural networks and actual brains makes the analogy basically useless. Still, I think it would probably be worthwhile for someone with more specialized knowledge than myself to think through the analogy roughly along the lines you’ve outlined and see what comes of it. I’d be happy to collaborate if anyone (including yourself) wants to take up the task.