Non-technical summary
We examined whether respondents selected more meat-free meals from certain types of menu, using data from an existing online hypothetical choice study (Brachem et al., 2019).
Respondents selected more meat-free meals from hypothetical menus with more meat-free options and fewer fish/poultry options. But we didn’t find a strong association for menus containing meat-analogue options.
There are lots of important limitations to our analysis (e.g. not a randomized experiment, hypothetical choices – not actual behavior; meat-analogues in study not very appealing).
Despite these limitations, we think the results point to:
the need for more research on the cost-effectiveness of advocating for meat-analogues compared to more meat-free options of any kind;
the potential harm to animal welfare if food-service providers include more fish and poultry dishes on their menus.
Summary
Increasing consumption of meat-free meals can help reduce demand for factory farmed animal products and anthropogenic greenhouse gas emissions. But relatively little research has been done on how meat-free meal selection is influenced by menu options, such as the availability of meat-analogue options or different types of meat.
We conducted a preregistered reanalysis of data from a series of hypothetical discrete choice experiments from Brachem et al. (2019). We explored how meat-free meal selection by 1348 respondents (mostly German students) varied across 26 different menus, depending on the number of meat-free options and whether any options contained fish/poultry meat or meat-analogues. Menus consisted of five options (of which, two or three were meat-free) and were composed using images and descriptions of actual dishes available at restaurants at the University of Göttingen.
While our work was motivated by causal hypotheses, our reanalysis was limited to detecting correlations and not causal effects. Specific limitations include:
Examining hypotheses that the original study was not designed to evaluate.
De facto observational design, despite blinded randomization in the original study.
Possible non-random correlations between the presence of poultry/fish or meat-analogue menu options and the appealingness of other dishes.
Analysis of self-reported, hypothetical meal preferences, rather than actual behavior.
Meat-analogues in menus not reflecting prominent products attracting significant financial investment.
Notwithstanding, our reanalysis found meat-free meal selection odds were:
higher among menus with an extra meat-free option (odds ratio of 2.3, 90% CI [1.8 to 3.0]).
lower among menus featuring poultry or fish options (odds ratio of 0.7, 90% CI [0.6 to 0.9]).
not significantly associated with the presence of meat-analogues on a menu (odds ratio of 1.2 (90% CI [0.9 to 1.6])) in our preregistered meat-analogue definition. Estimates varied across analogue definitions, but were never significantly different from 1.
Despite the many limitations, these findings might slightly update our beliefs to the extent we believe correlations would be expected if causation were occurring.
The poultry/fish option correlation highlights the potential for welfare losses from substitution towards small-bodied animals from menu changes as well as shifts in consumer preferences.
Given the study didn’t feature very prominent meat analogues, the absence of a correlation in this reanalysis cannot credibly be used to refute a belief that high-quality analogues play an important role in reducing meat consumption. But when coupled with the strong correlation on an additional meat-free option, we think the reanalysis highlights the need for further research on the most effective ways to encourage selection of meat-free meals. It remains an open question whether, at the margin, it would be more cost-effective to advocate for more menu options featuring meat-analogues specifically, or for more meat-free options of any kind.
You can read the full post on the Rethink Priorities website, and also see the pre-print and code via the Open Science Framework..
Hey, I wonder if there is a way to make this post, or posts like this, easier to read in the future. My friend suggested Chat GPT3, and it worked quite well, helping me understand what is written here. I wonder if maybe in future publications you can trial it out, and then do a little summary for people that don’t work with scientific studies/ statistics/ whatever it is, that makes it hard to understand with a quick read. It’s not a criticism, it’s just a suggestion, but I think from now on I will use chat GPT3 as a helpful tool :)
Thanks for the helpful comment @Ula Zarosa . I’ve now added a shorter non-technical summary.
Looks amazing. Thank you so much!
I’m not sure how to interpret the scale of this effect. To put this in context, what if we imagine people chose randomly from among the five options? Then we would expect 40% of people to choose meat-free options when it was two of five choices, and 60% when it was three of five. Did your reanalysis see a larger or smaller change than that?
Thanks for the question @Jeff Kaufman .
The short answer is that our headline odds ratio (OR) on the number of meat-free meals was pretty close to one associated with your random choice example. But the absolute increase in meat-free meal selections arising from an extra veg option (around 12pp) was lower than in the random choice example (20pp). That inconsistency reflects the high number of meat-free meal selections made by participants in the study even when there were only two meat-free options.
Detail
In the random selection example you provided (meat-free selections increasing from 40% to 60%), the OR would be 2.25:
Odds Ratio=Odds_afterOdds_before=60%/40%40%/60%=2.25
That’s just below our estimate for the OR (2.33) on the additional meat-free meal coefficient, but well within the 90% CI (1.81 to 2.99).
When interpreting results presented in odds ratio space, its important to bear in mind that odds ratios don’t translate linearly into pp changes—the starting odds/probability matters. An OR of 2.25 is associated with a 20pp increase for a baseline probability of 40%, but only a 10pp increase for a baseline probability of 80%.
The last column of Table 10 in the full write-up on the RP website provides the results of a linear regression model, which is easier to interpret in probability space. That model suggests 56% of meal selections were meat-free in a theoretical menu containing two (non-analogue) meat-free options, and three (red) meat-based dishes. Adding an additional (non-analogue) meat-free option would increase meat-free meal selection by 12pp (90% CI: 0.08 to 0.16). That’s quite a bit lower than the absolute increase in your random selection example (20pp).
So even though the results in odds ratio results were similar to one that might be expected under random selection, the absolute increase in number of meat-free meal selections was quite a bit lower. That inconsistency arises because participants in the study were opting for a high number of meat-free meals even when there were only two meat-free options.
Thanks! Would interpreting this as “here was a group of people who, on average, preferred non-meat options, so giving them more non-meat options decreased how much meat they ate” be right?
I don’t think I would agree with that as a general explanation of the results.… in order for us to get the results we did, you’d need some people who were selecting meat when there two veg options to select meat-free when there are three veg options. The folks who always selected a meat-free dish (around 20% of respondents) don’t drive variation in meat-free meal selection across different menu types, and so can’t explain our results. Same applies to those who always selected meat-based dishes. Indeed the regression results on menu characteristics were identical when we excluded those respondents (see Table 10 of full-writeup).
But I think your quote above might be able to explain why the pp change in meat dishes chosen is lower than the change of menu options. Would probably make a few changes though (completeness at the expense of brevity).
“When offered more non-meat menu options (20pp increase), survey respondents selected fewer meat-based dishes (12pp decrease). Respondents, on average, selected a relatively low share of meat-based meals across the experiment. This is one reason why the fall in the share of meat-dishes is smaller than the change in the share of menu options.”
Maybe the odds ratio in my situation is 60%/40% = 1.5? If so, this is well below 2.3 and outside your 90% CI. That’s quite a large effect!
One piece of context for background: what’s the base rate for vegans in Germany? Or if you collected it, for the survey group?
This source suggests the rate of self-identified veganism in Germany is about 3% in 2022. (We did not do any data collection ourselves; this report is a re-analysis of existing data collected by Brachem et al.)