Ok, I’ll discuss operant conditioning a bit here. I may have discounted this a too easily in my comment on part 1 (which was related to sensorimotor control) - I don’t think all aspects of operant conditioning necessarily require valanced experience, but it probably does at least require a predictive world model (or efferent copy) which, in itself, seems to be quite cognitively sophisticated.
I’ve thought a fair bit about operant conditioning in the context of adaptive sensorimotor control that ‘fine-tunes’ reflexive behaviours (see chapters 3 and 4 of my thesis). I mentioned fixation in my other comment, which is where an insect (say a bee) centers a visual object frontally—the bee has an intrinsically desired world state (object in front) and acts to realize that state, but I do not believe that positioning the object frontally really counts as positive valance for her. However, the bee is able to learn to change how it responds to discrepancies in the desired world state (say the polarity coupling her yaw torque to the world is inverted, she will learn to turn right to make the object turn right, instead of the normal situation of turning left to make an object on her left move rightwards) and one hypothesis is that the bee can make this adaption because it not only has both a desired world state and motor control program it would normally use to achieve that state, but also makes a prediction of how it’s actions will affect the world. In the event that the bee observes the results of its actions no longer match its predictions, and before she reverses the polarity of its yaw control, the bee may first update her world model to reflect the fact that it should now expect the world to turn in the opposite direction, from which the new predictions can be used to update the world model. Predictive models are an old idea from psychology that were explored in Drosophila using behavioral experiments before a neural circuit implementing an efferent copy was identified in them. The whole world model thing sounds rather abstract but has some real world examples, such as a growing animal learns that adapts its gait to longer legs, or an insect that adapts its flight muscle output to compensate for wing wear, and it also seems to have a relatively simple neural implementation in Drosophila that doesn’t really code for much information about about the world. Of course the appearance of adaptive sensory motor control may not necessarily require a predictive world model, and it is possible that a robust motor control scheme could pre-code responses to enough conditions to appear adaptive, but given that insects have small brains I’d suggest a basic adaptive process is involved. See Section 6.2.4 in my thesis for a more in depth discussion on adaptive control and extra literature references.
I agree that the classic case of operant conditioning (like learning to do something for a reward) does imply valanced experience (I don’t think that showing a conditioned reflex like salivating necessarily reduces the strength of this evidence). However, I don’t entirely agree with how you are phrasing learning new or unfamiliar actions—it would help to be more specific. In most cases what is being learnt is the use of known actions in new contexts or novel combinations of known actions (in known or novel contexts) - I think that learning a new action is quite rare. Let me elaborate—an adult rat (post development) probably knows how to push things in general (and indeed make most other motor actions its body is capable of) but it needs to form the association that applying the known pushing action to a lever gives it a reward. Likewise, the soccer playing bumblebees knew how to walk and probably more or less knew how to push things, but they had to learn to do this in sequence to get the ball to the reward point. Teach a rat (or bumblebee) to handstand an I will agree you’ve taught it a new action. Why make this distinction? The cognitive flexibility associated with learning to use known actions in new contexts seems different from learning new motor skills, and what should be assessed be assessed is the novelty of the context in the former case and the novelty of the action in the later case. Learning new combinations of known movements seems somewhere in between contextual and motor learning. Most organisms do motor learning during development when they have an intrinsic motivation to learn how to use their bodies (and the learning probably involves changing spinal cord type circuits), so I’d suggest that contextual learning provides stronger evidence for cognitive flexibility (I don’t know if any literature supports this, this is a distinction just became apparent to me when reading this post).
As an aside, I don’t think you’ve mentioned novelty seeking behaviour yet? I was peripherally involved in a study that shows honeybee choose to look a novel stimulus over a recently experienced stimulus in the absence of any specific reward. I’m not really sure how this fits into the framework of this study, but learning could be a good place to consider this.
I hope to get to navigation in the next of my mini-series of comments..
Edited a bit for clarity and grammar (without breaking the formatting as I did in my other comment).
Ok, I’ll discuss operant conditioning a bit here. I may have discounted this a too easily in my comment on part 1 (which was related to sensorimotor control) - I don’t think all aspects of operant conditioning necessarily require valanced experience, but it probably does at least require a predictive world model (or efferent copy) which, in itself, seems to be quite cognitively sophisticated.
I’ve thought a fair bit about operant conditioning in the context of adaptive sensorimotor control that ‘fine-tunes’ reflexive behaviours (see chapters 3 and 4 of my thesis). I mentioned fixation in my other comment, which is where an insect (say a bee) centers a visual object frontally—the bee has an intrinsically desired world state (object in front) and acts to realize that state, but I do not believe that positioning the object frontally really counts as positive valance for her. However, the bee is able to learn to change how it responds to discrepancies in the desired world state (say the polarity coupling her yaw torque to the world is inverted, she will learn to turn right to make the object turn right, instead of the normal situation of turning left to make an object on her left move rightwards) and one hypothesis is that the bee can make this adaption because it not only has both a desired world state and motor control program it would normally use to achieve that state, but also makes a prediction of how it’s actions will affect the world. In the event that the bee observes the results of its actions no longer match its predictions, and before she reverses the polarity of its yaw control, the bee may first update her world model to reflect the fact that it should now expect the world to turn in the opposite direction, from which the new predictions can be used to update the world model. Predictive models are an old idea from psychology that were explored in Drosophila using behavioral experiments before a neural circuit implementing an efferent copy was identified in them. The whole world model thing sounds rather abstract but has some real world examples, such as a growing animal learns that adapts its gait to longer legs, or an insect that adapts its flight muscle output to compensate for wing wear, and it also seems to have a relatively simple neural implementation in Drosophila that doesn’t really code for much information about about the world. Of course the appearance of adaptive sensory motor control may not necessarily require a predictive world model, and it is possible that a robust motor control scheme could pre-code responses to enough conditions to appear adaptive, but given that insects have small brains I’d suggest a basic adaptive process is involved. See Section 6.2.4 in my thesis for a more in depth discussion on adaptive control and extra literature references.
I agree that the classic case of operant conditioning (like learning to do something for a reward) does imply valanced experience (I don’t think that showing a conditioned reflex like salivating necessarily reduces the strength of this evidence). However, I don’t entirely agree with how you are phrasing learning new or unfamiliar actions—it would help to be more specific. In most cases what is being learnt is the use of known actions in new contexts or novel combinations of known actions (in known or novel contexts) - I think that learning a new action is quite rare. Let me elaborate—an adult rat (post development) probably knows how to push things in general (and indeed make most other motor actions its body is capable of) but it needs to form the association that applying the known pushing action to a lever gives it a reward. Likewise, the soccer playing bumblebees knew how to walk and probably more or less knew how to push things, but they had to learn to do this in sequence to get the ball to the reward point. Teach a rat (or bumblebee) to handstand an I will agree you’ve taught it a new action. Why make this distinction? The cognitive flexibility associated with learning to use known actions in new contexts seems different from learning new motor skills, and what should be assessed be assessed is the novelty of the context in the former case and the novelty of the action in the later case. Learning new combinations of known movements seems somewhere in between contextual and motor learning. Most organisms do motor learning during development when they have an intrinsic motivation to learn how to use their bodies (and the learning probably involves changing spinal cord type circuits), so I’d suggest that contextual learning provides stronger evidence for cognitive flexibility (I don’t know if any literature supports this, this is a distinction just became apparent to me when reading this post).
As an aside, I don’t think you’ve mentioned novelty seeking behaviour yet? I was peripherally involved in a study that shows honeybee choose to look a novel stimulus over a recently experienced stimulus in the absence of any specific reward. I’m not really sure how this fits into the framework of this study, but learning could be a good place to consider this.
I hope to get to navigation in the next of my mini-series of comments..
Edited a bit for clarity and grammar (without breaking the formatting as I did in my other comment).