We are grateful to Vasco for sharing this analysis with us prior to publication, for delaying in order to give us time to respond, and also for using his time to quantitatively explore the most impactful way to help animals.
While we appreciate Vasco’s intentions in producing this analysis, we believe there are reasons to be much more optimistic about Veganuary’s impact. We feel Vasco’s results reflect arbitrary assumptions based on his priors, rather than high-quality evidence about our campaign’s effectiveness.
Vasco’s calculations imply that Veganuary counterfactually reduced meat consumption by an extraordinarily small amount – the equivalent of just 1.3K metric tonnes. Around half of this is assumed to come from 25M direct pledge participants – who he assumes counterfactually reduce lifetime meat consumption by the equivalent of just 25g each (around 0.03% of annual meat consumption in the UK in 2021). The other half comes from all Veganuary’s other work, including our corporate engagement work, which we believe reaches many more people.
We regrettably think Vasco made very big adjustments that reflect his pessimism about our impact, but neither the magnitude of these judgements nor the degree of underlying uncertainty is communicated very clearly in his post. Crucially, Vasco arbitrarily assumed that Veganuary’s impact on pledge participants is 10x lower than the average effect size from the Green et al (2024) meta-analysis. Vasco’s justification of this is that our spend per pledge participant (c.$0.13) is probably a lot lower than the interventions in the Green et al (2024) meta-analysis.
We believe Veganuary might be a victim of its own success here. Being able to reach millions of people at low cost by creatively leveraging media and inspiring improved retail and food service offerings is a potential signal of effectiveness – but Vasco appears to have used this as a negative signal about our impact.
Vasco also assumed that the counterfactual impact of our non-pledge work is about the same as our work to inspire pledge participation. While this is hard to know for sure, we believe the counterfactual impact of our corporate engagement work to inspire more, better, and highly visible plant-based offerings in retailers and food service could be several times higher than arising from pledge participants.
We accept that properly measuring the counterfactual impact of interventions on animal product consumption is exceptionally hard, even more so with campaigns like Veganuary that gain national media coverage and influence product offerings at major retailers and food service outlets. We’d love to be able to allocate more resources towards measuring our impact. We aren’t yet in a position to provide quantitative estimates we feel we can stand by, but the following results make us optimistic our impact is likely to be much higher than what Vasco set out in his analysis:
Meat reduction data from Germany’s Statistical Office: The German Federal Statistical Office (Destatis) published that based on scanner data from German food retailers, 29.4% less meat was purchased in January 2024 than in December 2023. Compared to the 2023 annual average, meat sales in January 2024 were 12.5% lower than the previous 12 months. Compared to the 2022 annual average, meat sales in January 2023 were 14.3% lower than the previous 12 months.
New Research from the University of Exeter suggests that taking part in Veganuary leads to sustained reductions in meat consumption and can also produce fundamental shifts in people’s attitudes towards meat and their own self-identity as a meat-eater. Full study here.
Kantar data from the UK shows that, for 832K people who gave up animal products for the first time in January 2019, sustained reductions in animal product purchases over the following six months totalled 4,452,603 kg – equivalent to sparing 3.6 million animals by our calculations. While Veganuary cannot take all the counterfactual credit for these diet changes, it is striking how the estimated aggregate reduction in animal product consumption from pledge participants in just one country in 2019 is considerably higher than Vasco’s assumptions.
Participant surveys consistently show long-term dietary changes, with over 80% of surveyed email participants reducing animal product consumption by at least half, creating compounding benefits over time. We do of course recognise the selection bias in participant surveys, which is why we commissioned the Kantar study in 2019 and have switched to nationally representative surveys to better measure our reach.
While we are grateful for Vasco’s commitment to seeking the most cost-effective way to help animals, we think he has made arbitrary judgements that make Veganuary look considerably less cost-effective than we believe it is and has not clearly explained these judgements in a way is helpful for readers.
While we accept it is exceptionally difficult to measure the impact of interventions like Veganuary, which use the diet change element of its work to drive progress through corporate engagement, we welcome ideas from readers on ideas of how we might be able to do this (and resources to help run such studies). We also welcome serious attempts to measure our effectiveness that legibly explain key cruxes/judgements/uncertainties, and credible suggestions on how we can improve our effectiveness (acknowledging we operate under constraints given our name, branding, and supporter base).
Toni Vernelli, International Head of Communications, Veganuary
Vasco’s calculations imply that Veganuary counterfactually reduced meat consumption by an extraordinarily small amount – the equivalent of just 1.3K metric tonnes. Around half of this is assumed to come from 25M direct pledge participants – who he assumes counterfactually reduce lifetime meat consumption by the equivalent of just 25g each (around 0.03% of annual meat consumption in the UK in 2021). The other half comes from all Veganuary’s other work, including our corporate engagement work, which we believe reaches many more people.
I agree these numbers follow from my assumptions, but they do not seem super low to me. I do not have strong expectations about the mean reduction in meat consumption in kg among the people who reported participating in Veganuary. Note “My effect size refers to people who reported participating in Veganuary. The number of people who actually participated may be significantly smaller. Veganuary concluded 25 M participated multiplying small fractions of participants in the surveyed countries by their populations. However, such fractions may be significantly explained by a few people mistakenly reporting their participation due to lack of attention or social desirability bias”.
We regrettably think Vasco made very big adjustments that reflect his pessimism about our impact, but neither the magnitude of these judgements nor the degree of underlying uncertainty is communicated very clearly in his post. For example, Vasco arbitrarily assumed that Veganuary’s impact on pledge participants is 10x lower than the average effect size from the Green et al (2024) meta-analysis. Vasco’s justification of this is that our spend per pledge participant (c.$0.13) is probably a lot lower than the interventions in the Green et al (2024) meta-analysis.
Nitpick. Using “For example” may suggest you think there are many other important assumptions I made which you think are “arbitrary”. However, I guess my assumptions about the effect per person who reported participating in the pledge, and the effect on people who did not participate in the pledge, which you mentioned in your comment, are the most important, and the ones you disagree with the most. Are there others? If not, I think “Crucially” or “In particular” would communicate your views more faithfully than “For example”.
We believe Veganuary might be a victim of its own success here. Being able to reach millions of people at low cost by creatively leveraging media and inspiring improved retail and food service offerings is a potential signal of effectiveness – but Vasco appears to have used this as a negative signal about our impact.
My prior is that the effect per person decreases with the number of people affected. So more people being affected updates me towards a smaller effect per person. However, it does not necessarily update me towards a lower impact.
Meat reduction data from Germany’s Statistical Office: The German Federal Statistical Office (Destatis) published that based on scanner data from German food retailers, 29.4% less meat was purchased in January 2024 than in December 2023. Compared to the 2023 annual average, meat sales in January 2024 were 12.5% lower than the previous 12 months. Compared to the 2022 annual average, meat sales in January 2023 were 14.3% lower than the previous 12 months.
I do not know whether that was caused by Veganuary. The meat consumption per capita excluding fish and seafood in Germany peaked in 1987, well before Veganuary started in the UK in 2014, and has trended downwards unevenly since then.
From the point of view of animal welfare, it is much more important to look into the number of poultry birds and aquatic animals per capita. In Germany, the number of poultry birds per capita increased from 1.67 (= 135*10^6/(81.0*10^6)) in 2014 (when Veganuary started in the UK) to 2.07 (= 173*10^6/(83.6*10^6)) in 2020 (the last year for which there is data about the number of poultry birds on Our World in Data (OWID)), i.e. 24.0 % (= 2.07/1.67 − 1). The vast majority of this increased happened from 2014 to 2016.
New Research from the University of Exeter suggests that taking part in Veganuary leads to sustained reductions in meat consumption and can also produce fundamental shifts in people’s attitudes towards meat and their own self-identity as a meat-eater. Full study here.
The study looked into just 40 highly selected people[1], and there was no control group or measures to mitigate social desirability bias.
Kantar data from the UK shows that, for 832K people who gave up animal products for the first time in January 2019, sustained reductions in animal product purchases over the following six months totalled 4,452,603 kg – equivalent to sparing 3.6 million animals by our calculations. While Veganuary cannot take all the counterfactual credit for these diet changes, it is striking how the estimated aggregate reduction in animal product consumption from pledge participants in just one country in 2019 is considerably higher than Vasco’s assumptions.
Could you share a link? I would not be surprised if only 1 % of that meat reduction had been caused by Veganuary, in which case there would be little tension with my results. I would also like to know about the changes in the consumption of specific types of animal-based foods.
I worry about replacements of red meat with white meat, eggs, and farmed aquatic animals, which respect a greater suffering per kg. Veganuary started in the United Kingdom in 2014, and the production of broilers per person in there in January increased 4.27 times as fast from 2014 to 2024 as from 1994 to 2013. Data about consumption (production plus net imports) would be more informative, and the production of broilers per person could have increased faster without Veganuary, but the correlation is still concerning.
While we are grateful for Vasco’s commitment to seeking the most cost-effective way to help animals, we think he has made arbitrary judgements that make Veganuary look considerably less cost-effective than we believe it is and has not clearly explained these judgements in a way is helpful for readers.
I estimated Veganuary in 2024 was 1.20 % as cost-effective as cage-free campaigns, and that these are 0.718 % as cost-effective as SWP. So I think more optimistic assumptions about Veganuary would still leave it much less cost-effective that SWP.
The pre-screen survey was completed by 1,125 people on Prolific. Of those, 60 were eligible participants (who were either omnivores or flexitarians and intended to take part in Veganuary or a different meat-free January challenge) and were invited to the baseline survey. 48 participants completed this survey without failing more than one of three attention check questions. All of these were invited to the follow-up survey in February which was completed without more than one failed attention check by all of the 40 people who participated in it.
My understanding of the argument regarding “Meat reduction data from Germany’s Statistical Office” is that significantly less meat is consumed in January than in the other months (I don’t understand the argument in the sense that Veganuary has contributed to the downward trend over the years). Perhaps it would be better to compare the value for January 2023 with the monthly average for 2023 than with the monthly average for 2022, because this would rule out the possibility that the comparatively low consumption is due to the downward trend over the years. However, since the annual decline is less than 12.5% or 14.3%, consumption in January is also lower than the average month of the corresponding year. The fact that the consumption of poultry meat in Germany has risen in recent years is very regrettable, but I don’t believe that this is due to Veganuary. However, the German Statistical Office should also have figures for different animal species, so you should be able to see whether the consumption of poultry meat is higher, lower or similar in January compared to other months. I don’t think comparing January with December is very helpful because December is an atypical month due to Christmas.
We are grateful to Vasco for sharing this analysis with us prior to publication, for delaying in order to give us time to respond, and also for using his time to quantitatively explore the most impactful way to help animals.
While we appreciate Vasco’s intentions in producing this analysis, we believe there are reasons to be much more optimistic about Veganuary’s impact. We feel Vasco’s results reflect arbitrary assumptions based on his priors, rather than high-quality evidence about our campaign’s effectiveness.
Vasco’s calculations imply that Veganuary counterfactually reduced meat consumption by an extraordinarily small amount – the equivalent of just 1.3K metric tonnes. Around half of this is assumed to come from 25M direct pledge participants – who he assumes counterfactually reduce lifetime meat consumption by the equivalent of just 25g each (around 0.03% of annual meat consumption in the UK in 2021). The other half comes from all Veganuary’s other work, including our corporate engagement work, which we believe reaches many more people.
We regrettably think Vasco made very big adjustments that reflect his pessimism about our impact, but neither the magnitude of these judgements nor the degree of underlying uncertainty is communicated very clearly in his post. Crucially, Vasco arbitrarily assumed that Veganuary’s impact on pledge participants is 10x lower than the average effect size from the Green et al (2024) meta-analysis. Vasco’s justification of this is that our spend per pledge participant (c.$0.13) is probably a lot lower than the interventions in the Green et al (2024) meta-analysis.
We believe Veganuary might be a victim of its own success here. Being able to reach millions of people at low cost by creatively leveraging media and inspiring improved retail and food service offerings is a potential signal of effectiveness – but Vasco appears to have used this as a negative signal about our impact.
Vasco also assumed that the counterfactual impact of our non-pledge work is about the same as our work to inspire pledge participation. While this is hard to know for sure, we believe the counterfactual impact of our corporate engagement work to inspire more, better, and highly visible plant-based offerings in retailers and food service could be several times higher than arising from pledge participants.
We accept that properly measuring the counterfactual impact of interventions on animal product consumption is exceptionally hard, even more so with campaigns like Veganuary that gain national media coverage and influence product offerings at major retailers and food service outlets. We’d love to be able to allocate more resources towards measuring our impact. We aren’t yet in a position to provide quantitative estimates we feel we can stand by, but the following results make us optimistic our impact is likely to be much higher than what Vasco set out in his analysis:
Meat reduction data from Germany’s Statistical Office: The German Federal Statistical Office (Destatis) published that based on scanner data from German food retailers, 29.4% less meat was purchased in January 2024 than in December 2023. Compared to the 2023 annual average, meat sales in January 2024 were 12.5% lower than the previous 12 months. Compared to the 2022 annual average, meat sales in January 2023 were 14.3% lower than the previous 12 months.
New Research from the University of Exeter suggests that taking part in Veganuary leads to sustained reductions in meat consumption and can also produce fundamental shifts in people’s attitudes towards meat and their own self-identity as a meat-eater. Full study here.
Kantar data from the UK shows that, for 832K people who gave up animal products for the first time in January 2019, sustained reductions in animal product purchases over the following six months totalled 4,452,603 kg – equivalent to sparing 3.6 million animals by our calculations. While Veganuary cannot take all the counterfactual credit for these diet changes, it is striking how the estimated aggregate reduction in animal product consumption from pledge participants in just one country in 2019 is considerably higher than Vasco’s assumptions.
Participant surveys consistently show long-term dietary changes, with over 80% of surveyed email participants reducing animal product consumption by at least half, creating compounding benefits over time. We do of course recognise the selection bias in participant surveys, which is why we commissioned the Kantar study in 2019 and have switched to nationally representative surveys to better measure our reach.
While we are grateful for Vasco’s commitment to seeking the most cost-effective way to help animals, we think he has made arbitrary judgements that make Veganuary look considerably less cost-effective than we believe it is and has not clearly explained these judgements in a way is helpful for readers.
While we accept it is exceptionally difficult to measure the impact of interventions like Veganuary, which use the diet change element of its work to drive progress through corporate engagement, we welcome ideas from readers on ideas of how we might be able to do this (and resources to help run such studies). We also welcome serious attempts to measure our effectiveness that legibly explain key cruxes/judgements/uncertainties, and credible suggestions on how we can improve our effectiveness (acknowledging we operate under constraints given our name, branding, and supporter base).
Toni Vernelli, International Head of Communications, Veganuary
Thanks, Toni!
I agree these numbers follow from my assumptions, but they do not seem super low to me. I do not have strong expectations about the mean reduction in meat consumption in kg among the people who reported participating in Veganuary. Note “My effect size refers to people who reported participating in Veganuary. The number of people who actually participated may be significantly smaller. Veganuary concluded 25 M participated multiplying small fractions of participants in the surveyed countries by their populations. However, such fractions may be significantly explained by a few people mistakenly reporting their participation due to lack of attention or social desirability bias”.
Nitpick. Using “For example” may suggest you think there are many other important assumptions I made which you think are “arbitrary”. However, I guess my assumptions about the effect per person who reported participating in the pledge, and the effect on people who did not participate in the pledge, which you mentioned in your comment, are the most important, and the ones you disagree with the most. Are there others? If not, I think “Crucially” or “In particular” would communicate your views more faithfully than “For example”.
My prior is that the effect per person decreases with the number of people affected. So more people being affected updates me towards a smaller effect per person. However, it does not necessarily update me towards a lower impact.
I do not know whether that was caused by Veganuary. The meat consumption per capita excluding fish and seafood in Germany peaked in 1987, well before Veganuary started in the UK in 2014, and has trended downwards unevenly since then.
From the point of view of animal welfare, it is much more important to look into the number of poultry birds and aquatic animals per capita. In Germany, the number of poultry birds per capita increased from 1.67 (= 135*10^6/(81.0*10^6)) in 2014 (when Veganuary started in the UK) to 2.07 (= 173*10^6/(83.6*10^6)) in 2020 (the last year for which there is data about the number of poultry birds on Our World in Data (OWID)), i.e. 24.0 % (= 2.07/1.67 − 1). The vast majority of this increased happened from 2014 to 2016.
The study looked into just 40 highly selected people[1], and there was no control group or measures to mitigate social desirability bias.
Could you share a link? I would not be surprised if only 1 % of that meat reduction had been caused by Veganuary, in which case there would be little tension with my results. I would also like to know about the changes in the consumption of specific types of animal-based foods.
I estimated Veganuary in 2024 was 1.20 % as cost-effective as cage-free campaigns, and that these are 0.718 % as cost-effective as SWP. So I think more optimistic assumptions about Veganuary would still leave it much less cost-effective that SWP.
My understanding of the argument regarding “Meat reduction data from Germany’s Statistical Office” is that significantly less meat is consumed in January than in the other months (I don’t understand the argument in the sense that Veganuary has contributed to the downward trend over the years). Perhaps it would be better to compare the value for January 2023 with the monthly average for 2023 than with the monthly average for 2022, because this would rule out the possibility that the comparatively low consumption is due to the downward trend over the years. However, since the annual decline is less than 12.5% or 14.3%, consumption in January is also lower than the average month of the corresponding year.
The fact that the consumption of poultry meat in Germany has risen in recent years is very regrettable, but I don’t believe that this is due to Veganuary. However, the German Statistical Office should also have figures for different animal species, so you should be able to see whether the consumption of poultry meat is higher, lower or similar in January compared to other months.
I don’t think comparing January with December is very helpful because December is an atypical month due to Christmas.