Thanks for sharing the data for the paper. Based on a fairly quick re-analysis, it appears that a decline in selection of chicken can be detected—but as you note, choice of alternative red meats tends to increase slightly, and some of the increase in selecting the new plant-based meat is driven by fewer people selecting the other veg-based options.
A few considerations as a result:
Could it still be good to reduce purchasing of chicken, given the relatively poorer conditions of chickens and greater animals per kg of meat slughtered to get chicken vs. beef?
Given some people who seemingly would have eaten the chicken switch to beef when a chicken alternative is on the menu, might some moral self deception be at play? People say they’ll eat veggie if only there is a replacement for X. When there is a replacement for X, they can’t continue eating X in good conscience but switch to Y, for which they can then say they would eat plant-based if only there were a plant-based Y alternative
I wonder if one could see if chicken increases when an explicit replacement for beef and steak were made (or is sofritas already explicitly intended as a replacement for one of these?)
I wonder if meat might ultimately decline if there were yet more plant-based explicit alternatives—what do you think?
It would be particularly interesting to know the ‘readiness to change’ or ‘intention/sympathy’ towards vegetarianism—I might expect more sympathetic/genuinely ready to change people really would try the alternative out as opposed to just switching to beef
Finally, what are your thoughts on the ecological validity of the experiment? Naturally, people can’t see or smell the options, as you note in the paper, but the experiment also takes place in the absence of repeat visits, advertising, and hearsay about how X or Y tastes really good, has to be tried etc. I’m not totally surprised that just adding an option to the end of a virtual menu doesn’t greatly shift behavior, but that also seems a slightly reductionistic model of what plant-based advocates might expect would happen in such a situation, which might be more to do with norm shifting, decreasing difficulty enabling people to switch over time etc. I hope that does not come across as overly critical, because I’m also really pleased to see a rigorous attempt to put ideas to the test and challenge assumptions. [edit: I haven’t thoroughly read you paper, so apologies if you already discuss this]
Hi James, neat visualizations, and very validating that you were able to extend our work like this! We worked hard to make our materials legible but you don’t really know how well that went until someone actually tries to use them 😃 So this is great to see.
Yes, a switch away from chicken meat towards beef could be good under some circumstances/assumptions. But the goal of our experiment was to come up with an effect size large enough to take to Chipotle, and we don’t think we found one. My guess is that the interspecies tradeoffs also would not be very persuasive to a fast casual chain relative to beef’s larger climate impact.
I’m not sure. Sofritas are more or less an analogue to ground beef, but I’m not sure people make that connection. Our thinking for this experiment was that chicken typically has the fewest analogues widely available, so we should try to focus on that. But I am no longer sure that I have a good sense of how introducing PMAs would impact meat consumption. Yes, we find some evidence that chickn’itas absorbs demand from chicken specifically, but it’s not a slam dunk by any means. Maybe another PMA or two would have larger effects. I doubt it.
I agree that proto-vegetarians might be more actiely exploring alternatives...but how many people are in this category? I’d venture less than 1% of people are seriously considering it. Probably a much larger category are looking to “cut back” in some sense, but that might mean many things to them.
I think our experiment has high ecological validity for the thing we are testing, which is the introduction of PMAs to an online, Chipotle-like menu. That’s a real environment in which people encounter PMAs, and because it’s online, IMHO it may lack promotion, buzz, etc. Perhaps a more elaborate test of a more fleshed out, multi-component theory would find different effects. On the other hand, our intervention is easily scaled up.
For tests of “hearsay about how X or Y tastes really good, has to be tried etc” see, Sparkman et al. (2020, e.g. figure 2) and Piester et al. (2020). We review some of those studies here. I think broadly speaking you are talking about norms-based approaches, see here for a general review and here for a review specific to eating meat.
The cleaned data set was very nice to have access to and clear—the only thing that wasn’t clear to me was whether any exclusions were actually applied on the basis of the attention checks, and what the correct answers to the attention question were, but this may be in your documentation already, I just had a fairly quick look and downloaded the csv and got going.
Thanks for the thorough response -
indeed if the goal is about doing something to shift Chipotle/similar chains then the chicken reduction angle is unlikely to be persuasive.
Is there any way of finding out from real data whether people who literally wouldn’t go to chipotle started doing so when sofritas became a thing?
Fair enough
Agree with this. I could imagine that over the scale of something like Chipotle, it could be that satisfying the ‘every so often’ plant-based purchase, as opposed to meat, of reducetarians could be impactful and affect how much meat gets purchased overall, but it’s far from clear and not something we’d be powered to detect with all but the most elaborate experiments, most likely
That’s a good point, I actually had not considered people ordering online somehow...to the extent that the study was intending to represent an online experience then yes I consider it more ecologically than I first perceived it to be
Those studies don’t really convince me as I don’t think it’s possible to actually change what people perceive to be real norms (or basically their schemas of how the world is), or what their friends are saying and getting excited about, or the media is reporting on, which percolates organically and affects ones worldview, with small experimental manipulations, so to this extent I think the sorts of stuff I’m talking about are very hard experimentally. Even with an online menu I think people still arrive there having already been influenced by all sorts of things, I’m not referring to explicit advertising that would in the menu or in the store specifically. [edit: but I don’t think it is necessary to discuss further]
Thanks for sharing the data for the paper. Based on a fairly quick re-analysis, it appears that a decline in selection of chicken can be detected—but as you note, choice of alternative red meats tends to increase slightly, and some of the increase in selecting the new plant-based meat is driven by fewer people selecting the other veg-based options.
A few considerations as a result:
Could it still be good to reduce purchasing of chicken, given the relatively poorer conditions of chickens and greater animals per kg of meat slughtered to get chicken vs. beef?
Given some people who seemingly would have eaten the chicken switch to beef when a chicken alternative is on the menu, might some moral self deception be at play? People say they’ll eat veggie if only there is a replacement for X. When there is a replacement for X, they can’t continue eating X in good conscience but switch to Y, for which they can then say they would eat plant-based if only there were a plant-based Y alternative
I wonder if one could see if chicken increases when an explicit replacement for beef and steak were made (or is sofritas already explicitly intended as a replacement for one of these?)
I wonder if meat might ultimately decline if there were yet more plant-based explicit alternatives—what do you think?
It would be particularly interesting to know the ‘readiness to change’ or ‘intention/sympathy’ towards vegetarianism—I might expect more sympathetic/genuinely ready to change people really would try the alternative out as opposed to just switching to beef
Finally, what are your thoughts on the ecological validity of the experiment? Naturally, people can’t see or smell the options, as you note in the paper, but the experiment also takes place in the absence of repeat visits, advertising, and hearsay about how X or Y tastes really good, has to be tried etc. I’m not totally surprised that just adding an option to the end of a virtual menu doesn’t greatly shift behavior, but that also seems a slightly reductionistic model of what plant-based advocates might expect would happen in such a situation, which might be more to do with norm shifting, decreasing difficulty enabling people to switch over time etc. I hope that does not come across as overly critical, because I’m also really pleased to see a rigorous attempt to put ideas to the test and challenge assumptions. [edit: I haven’t thoroughly read you paper, so apologies if you already discuss this]
Hi James, neat visualizations, and very validating that you were able to extend our work like this! We worked hard to make our materials legible but you don’t really know how well that went until someone actually tries to use them 😃 So this is great to see.
Yes, a switch away from chicken meat towards beef could be good under some circumstances/assumptions. But the goal of our experiment was to come up with an effect size large enough to take to Chipotle, and we don’t think we found one. My guess is that the interspecies tradeoffs also would not be very persuasive to a fast casual chain relative to beef’s larger climate impact.
I’m not sure. Sofritas are more or less an analogue to ground beef, but I’m not sure people make that connection. Our thinking for this experiment was that chicken typically has the fewest analogues widely available, so we should try to focus on that. But I am no longer sure that I have a good sense of how introducing PMAs would impact meat consumption. Yes, we find some evidence that chickn’itas absorbs demand from chicken specifically, but it’s not a slam dunk by any means. Maybe another PMA or two would have larger effects. I doubt it.
I agree that proto-vegetarians might be more actiely exploring alternatives...but how many people are in this category? I’d venture less than 1% of people are seriously considering it. Probably a much larger category are looking to “cut back” in some sense, but that might mean many things to them.
I think our experiment has high ecological validity for the thing we are testing, which is the introduction of PMAs to an online, Chipotle-like menu. That’s a real environment in which people encounter PMAs, and because it’s online, IMHO it may lack promotion, buzz, etc. Perhaps a more elaborate test of a more fleshed out, multi-component theory would find different effects. On the other hand, our intervention is easily scaled up.
For tests of “hearsay about how X or Y tastes really good, has to be tried etc” see, Sparkman et al. (2020, e.g. figure 2) and Piester et al. (2020). We review some of those studies here. I think broadly speaking you are talking about norms-based approaches, see here for a general review and here for a review specific to eating meat.
The cleaned data set was very nice to have access to and clear—the only thing that wasn’t clear to me was whether any exclusions were actually applied on the basis of the attention checks, and what the correct answers to the attention question were, but this may be in your documentation already, I just had a fairly quick look and downloaded the csv and got going.
Thanks for the thorough response -
indeed if the goal is about doing something to shift Chipotle/similar chains then the chicken reduction angle is unlikely to be persuasive.
Is there any way of finding out from real data whether people who literally wouldn’t go to chipotle started doing so when sofritas became a thing?
Fair enough
Agree with this. I could imagine that over the scale of something like Chipotle, it could be that satisfying the ‘every so often’ plant-based purchase, as opposed to meat, of reducetarians could be impactful and affect how much meat gets purchased overall, but it’s far from clear and not something we’d be powered to detect with all but the most elaborate experiments, most likely
That’s a good point, I actually had not considered people ordering online somehow...to the extent that the study was intending to represent an online experience then yes I consider it more ecologically than I first perceived it to be
Those studies don’t really convince me as I don’t think it’s possible to actually change what people perceive to be real norms (or basically their schemas of how the world is), or what their friends are saying and getting excited about, or the media is reporting on, which percolates organically and affects ones worldview, with small experimental manipulations, so to this extent I think the sorts of stuff I’m talking about are very hard experimentally. Even with an online menu I think people still arrive there having already been influenced by all sorts of things, I’m not referring to explicit advertising that would in the menu or in the store specifically. [edit: but I don’t think it is necessary to discuss further]