Hi Michael, thanks for your skepticism of the empirical findings which is helpful. Khorton makes a similar point to you so I think that it is something worth taking seriously. As I mention in my epistemic status note I don’t have any experience in education and so can really benefit from insight from those that do and comments on the general state of educational research (e.g. that it struggles to come to firm conclusions on things) are useful.
In response to your surprise at not having heard of this evidence before, I don’t actually think the evidence I have cited is that incredible. For example the Trickey and Topping (2004) review finds ‘moderate’ effects, not large. The Trickey and Topping follow-up which finds evidence of long-term benefits does sound very impressive, but ultimately that is just one study and I take on board the comment that it might be that such a preliminary study may involve the very best, most highly-motivated teachers. Also, as I mention in the post, the research on moral development is slightly sparse and I am looking forward to seeing further evidence in this area.
There are a few relevant quotes from Trickey and Topping (2004) (which is pretty well-cited with 379 citations FYI). Perhaps I could have included some of these in the post:
“Many of the studies could be criticized on grounds of methodological rigour, but the quality and quantity of evidence nevertheless bears favourable comparison with that on many other methods in education.”
“It is not possible to assert that any use of the P4C process will always lead to positive outcomes, since implementation integrity may be highly variable. However, a wide range of evidence has been reported suggesting that, given certain conditions, children can gain significantly in measurable terms both academically and socially through this type of interactive process.”
“further investigation is needed of wider generalization within and beyond school, and of longer term maintenance of gains”
Overall these comments seem to imply that the evidence for P4C is highly promising building a strong case for further research. I wouldn’t say that subsequent research has definitively put the question to rest and the comments from Khorton and yourself have led me to uprate the expected value of further academic research in this area. It is worth noting that expansion of philosophical education would make such further empirical research easier, so I still see this as a worthy aim.
Thanks for that extra info—that all makes sense to me :)
And perhaps I interpreted your statements as suggesting stronger and more durable effects (or stronger evidence of that) than you actually intended to suggest.
Though “moderate” effects on “logical reasoning” and “reading ability” still sounds like a fairly big deal (though I say this without having checked precisely what’s meant by any of those three terms, in this context). Although maybe it’s the sort of pattern I and Khorton allude to, where a wide range of interventions will all cause benefits compared to business-as-usual, due to additional attention, passion, etc. when an intervention is being delivered and studied.
In any case, there could indeed be value in:
Looking into the existing evidence more closely and exhaustively (or, if you’ve already done that, additional people doing that)
Gathering additional evidence on this sort of intervention
Perhaps rolling this sort of intervention out even if we get no additional, or additional and less promising, evidence about outcomes such as logical reasoning and reading ability. We might do this for the other potential benefits of this sort of intervention, e.g. for moral circle expansion or EA movement-building.
My wording may have been slightly over-positive, but I agree that the evidence seems good enough to warrant further investigation. I wouldn’t say I have done an exhaustive review by any means.
I agree with your suggestions! I probably need a little bit of time to reflect on feedback but I may reach out to some people (e.g. SHIC folks) to see if there is anyone who is interested in investigating further and who has more specialised expertise in education than I do.
(Just want to note that it’s possible your wording was as positive as I initially thought and that that was appropriate, as I haven’t actually read the studies, so I can’t rule out that the effects really are very good and very well-evidenced.)
Hi Michael, thanks for your skepticism of the empirical findings which is helpful. Khorton makes a similar point to you so I think that it is something worth taking seriously. As I mention in my epistemic status note I don’t have any experience in education and so can really benefit from insight from those that do and comments on the general state of educational research (e.g. that it struggles to come to firm conclusions on things) are useful.
In response to your surprise at not having heard of this evidence before, I don’t actually think the evidence I have cited is that incredible. For example the Trickey and Topping (2004) review finds ‘moderate’ effects, not large. The Trickey and Topping follow-up which finds evidence of long-term benefits does sound very impressive, but ultimately that is just one study and I take on board the comment that it might be that such a preliminary study may involve the very best, most highly-motivated teachers. Also, as I mention in the post, the research on moral development is slightly sparse and I am looking forward to seeing further evidence in this area.
There are a few relevant quotes from Trickey and Topping (2004) (which is pretty well-cited with 379 citations FYI). Perhaps I could have included some of these in the post:
“Many of the studies could be criticized on grounds of methodological rigour, but the quality and quantity of evidence nevertheless bears favourable comparison with that on many other methods in education.”
“It is not possible to assert that any use of the P4C process will always lead to positive outcomes, since implementation integrity may be highly variable. However, a wide range of evidence has been reported suggesting that, given certain conditions, children can gain significantly in measurable terms both academically and socially through this type of interactive process.”
“further investigation is needed of wider generalization within and beyond school, and of longer term maintenance of gains”
Overall these comments seem to imply that the evidence for P4C is highly promising building a strong case for further research. I wouldn’t say that subsequent research has definitively put the question to rest and the comments from Khorton and yourself have led me to uprate the expected value of further academic research in this area. It is worth noting that expansion of philosophical education would make such further empirical research easier, so I still see this as a worthy aim.
Thanks for that extra info—that all makes sense to me :)
And perhaps I interpreted your statements as suggesting stronger and more durable effects (or stronger evidence of that) than you actually intended to suggest.
Though “moderate” effects on “logical reasoning” and “reading ability” still sounds like a fairly big deal (though I say this without having checked precisely what’s meant by any of those three terms, in this context). Although maybe it’s the sort of pattern I and Khorton allude to, where a wide range of interventions will all cause benefits compared to business-as-usual, due to additional attention, passion, etc. when an intervention is being delivered and studied.
In any case, there could indeed be value in:
Looking into the existing evidence more closely and exhaustively (or, if you’ve already done that, additional people doing that)
Gathering additional evidence on this sort of intervention
Perhaps rolling this sort of intervention out even if we get no additional, or additional and less promising, evidence about outcomes such as logical reasoning and reading ability. We might do this for the other potential benefits of this sort of intervention, e.g. for moral circle expansion or EA movement-building.
My wording may have been slightly over-positive, but I agree that the evidence seems good enough to warrant further investigation. I wouldn’t say I have done an exhaustive review by any means.
I agree with your suggestions! I probably need a little bit of time to reflect on feedback but I may reach out to some people (e.g. SHIC folks) to see if there is anyone who is interested in investigating further and who has more specialised expertise in education than I do.
(Just want to note that it’s possible your wording was as positive as I initially thought and that that was appropriate, as I haven’t actually read the studies, so I can’t rule out that the effects really are very good and very well-evidenced.)