Thank you so much for this research. Is there a more intuitive way to interpret SMD values? For example, how many standard deviations is an average vegetarian away from the average person in the general population?
putting SMDs into sensible terms is a continual struggle. I don’t think it’ll be easy to put vegetarians and meat eaters on a common scale because if vegetarians are all clustered around zero meat consumption, then the distance between vegs and meat eaters is just entirely telling you how much meat the meat eater group eats, and that changes a lot between populations.
Also, different disciplines have different ideas about what a ‘big’ effect size is. Andrew Gelman writes something I like about this:
the first problem I noticed with that meta-analysis was an estimated average effect size of 0.45 standard deviations. That’s an absolutely huge effect, and, yes, there could be some nudges that have such a large effect, but there’s no way the average of hundreds would be that large. It’s easy, though, to get such a large estimate by just averaging hundreds of estimates that are subject to massive selection bias. So it’s no surprise that they got an estimate of 0.45, but we shouldn’t take this as an estimate of treatment effects.
A different approach: here are a few studies, their main findings in normal terms, the SMD that translates to, and whether subjectively that’s considered big or small
Jalil et al. (2023) | 5.6% reduction in meat eaten |SMD = 0.11 | small (but well-measured and has lasting effects)
Camp * Lawrence (2019) |self-reported decrease on meat items FFQ of .28 points | SMD = 0.4 | moderate/​large
Andersson & Nelander (2021 | 1% decrease in vegetarian meals sold | SMD = 0.16 | small (but, again, well measured)
So, for instance, the absolute change in the third study is a lot smaller than the absolute change in the first but has a bigger SMD because there’s less variation in the dependent variable in that setting.
So anyway this is another hard problem. But in general, nothing here is equating to the kind of radical transformation that animal advocates might hope for.
Thank you so much for this research. Is there a more intuitive way to interpret SMD values? For example, how many standard deviations is an average vegetarian away from the average person in the general population?
Thank you for your kind words!
putting SMDs into sensible terms is a continual struggle. I don’t think it’ll be easy to put vegetarians and meat eaters on a common scale because if vegetarians are all clustered around zero meat consumption, then the distance between vegs and meat eaters is just entirely telling you how much meat the meat eater group eats, and that changes a lot between populations.
Also, different disciplines have different ideas about what a ‘big’ effect size is. Andrew Gelman writes something I like about this:
But by convention, an SMD of 0.5 is typically just considered a ‘medium’ effect. I tend to agree with Gelman that changing people’s behavior by half a standard deviation on average is huge.
A different approach: here are a few studies, their main findings in normal terms, the SMD that translates to, and whether subjectively that’s considered big or small
Jalil et al. (2023) | 5.6% reduction in meat eaten |SMD = 0.11 | small (but well-measured and has lasting effects)
Camp * Lawrence (2019) |self-reported decrease on meat items FFQ of .28 points | SMD = 0.4 | moderate/​large
Andersson & Nelander (2021 | 1% decrease in vegetarian meals sold | SMD = 0.16 | small (but, again, well measured)
So, for instance, the absolute change in the third study is a lot smaller than the absolute change in the first but has a bigger SMD because there’s less variation in the dependent variable in that setting.
So anyway this is another hard problem. But in general, nothing here is equating to the kind of radical transformation that animal advocates might hope for.