I am a Research Scientist at the Humane and Sustainable Food Lab at Stanford and a nonresident fellow at the Kahneman-Treisman Center at Princeton. By trade, I am a meta-analyst.
Here is my date-me doc.
Seth Ariel Green šø
Karma: 1,141
Anyone else get a pig butchering scam attempt lately via DM on the forun?
I just got the following message> Happy day to you, I am [X] i saw your profile today and i like it very much,which makes me to write to you to let you know that i am interested in you,therefore i will like you to write me back so that i will tell you further about myself and send you also my picture for you to know me physically.
[EMAIL]
I reported the user on their profile and opened a support request but just FYI
I think thatās a good ideaāor just post as yourself (?)
(Ofc I think I and others understand that things are in flux and this is all NBD)
š Looks interesting! What do you think about having the title reflect its origins, e.g. ālinkpost: Climate Change Is Worse Than Factory Farmingā, or āsuggested reading: [X]ā or something like that?
At a glance right now, the UX here looks like the EA Forum Team is itself endorsing this pretty radical position. (FWIW I appreciate the drive to cross-post interesting material/āthe broader drive to improve the forum experience, I have been thinking about your other post a bit lately and hope to respond soon)
Hi Ruben, I am not expert on that strand of research, but here a few papers that may be of interest (lead author/āyear/ātitle):
Rosenfeld 2018 The psychology of vegetarianism: Recent advances and future directions Dagevos 2021 Finding flexitarians: Current studies on meat eaters and meat reducers Salehi 2023 Forty-five years of research on vegetarianism and veganism: A systematic and comprehensive literature review of quantitative studies Cramer 2017 Characteristics of Americans Choosing Vegetarian and Vegan Diets for Health Reasons Hielkema 2022 A āvegetarian curry stewā or just a ācurry stewā? ā The effect of neutral labeling of vegetarian dishes on food choice among meat-reducers and non-reducers Barr 2002 Perceptions and practices of self-defined current vegetarian, former vegetarian, and nonvegetarian women
My implicit knowledge on the topic of knowledge production (rather than of veganuary) is that rosy results like the one you are citing often do not stand up to scrutiny. Maya raised one very salient objection to a gap between the headline interpretation and the data of a past iteration of this work here.
I believe that if I dig into it, Iāll find other, similar issues.
Sorry for such a meta answerā¦
No meaningful relationship! (see code below.) However, big caveat here that we had to guess on some of the samples because many studies do not report how many subjects or meals were treated (e.g. they report how many restaurants or days were assigned to treatment and control but didnāt count how many people participated)
> summary(lm(d ~ total_sample, data = dat)) Call: lm(formula = d ~ total_sample, data = dat) Residuals: Min 1Q Median 3Q Max -0.59897 -0.13702 -0.01868 0.12322 0.75767 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.06330835 0.02664964 2.376 0.0193 * total_sample -0.00002876 0.00004690 -0.613 0.5410 --- Signif. codes: 0 ā***ā 0.001 ā**ā 0.01 ā*ā 0.05 ā.ā 0.1 ā ā 1 Residual standard error: 0.2474 on 110 degrees of freedom Multiple R-squared: 0.003407, Adjusted R-squared: -0.005653 F-statistic: 0.376 on 1 and 110 DF, p-value: 0.541
Delay indicates the number of days that have elapsed between the beginning of treatment and the final outcome measure. How outcomes are measured varies from study to study, so in some cases itās a 24 hour food recall X number of days after treatment is administered (the last part of it), in others itās a continuous outcome measurement in a cafeteria (the entire period of delay).
I donāt know this, sorry, and not every study reports enough location data to begin to estimate this (e.g. studies that recruit an online sample from multiple countries)
This I can say more about!
The median delay, in days, is 14, and the mean is 52 (we have a few studies with long delays, the longest is 3 years (Jalil et al. 2023).
So Iād say, think āabout 2 weeks on average with some lengthy outliersā. Also thereās basically no relationship between delay and effect size.
to replicate in R (from the root directory of our project):
> source('./scripts/libraries.R') > source('./scripts/load-data.R') > summary(dat$delay) Min. 1st Qu. Median Mean 3rd Qu. Max. 4.00 11.50 14.00 52.05 60.00 1095.00 > source('./functions/sum-lm.R') # this is a little function we wrote that puts summary(lm()) into a dplyr-friendly pipe > dat |> sum_lm(y = d, x = delay) Estimate Std. Error t value Pr(>|t|) (Intercept) 0.05312 0.02552 2.08181 0.03968 delay 0.00005 0.00019 0.23166 0.81723
Hi Vasco, Iām afraid not, sorry. The diversity of outcome measures makes this all but impossible, e..g one study measures āservings of meat per weekā, others it by the gram, others count how many meals are served in a given time period, etc.
Thank you David! We will post any updates to https://āādoi.org/āā10.31219/āāosf.io/āāq6xyr
The paper is currently under submission at a journal and we likely wonāt modify it until we get some feedback.
$3500 to Animal Charity Evaluators
$1000 to GiveWell
$500 to Direct Action Everywhere
$ 480 to GiveDirectly
Definitely! When I went vegan, I prompted someone I know to look up how dairy cows are treated (not well), and they changed their diet quite a bit in light of that. So I have seen downstream effects personally. Caveat that I am annoying and prone to evangelize.
And if i were going to promote one definitely-not-scalable intervention to one very-hard-to-reach-population, I would take a bunch of die-hard meat eaters to Han Dynasty on the upper west side of Manhattan and order 1) DanDan noodles without pork 2) pea leaves with garlic 3) cumin tofu 4) kung pao tofu and 5) eggplant in garlic sauce for the table, and then ask āhello is this not delicious??ā every 30 seconds š
I donāt know, sorry. There would be a lot of additional assumptions needed to extrapolate from the RCTs we analyze to this.
That sounds very interesting!
Making things more pleasant for vegetarians and vegans is a good thing to do, even if it does not change other peopleās behavior too much.
In the long-run, we want to make vegetarianism seem just as ānice, natural, and normalā (https://āāwww.sciencedirect.com/āāscience/āāarticle/āāabs/āāpii/āāS0195666315001518) as eating meat.
I think things like a Meatless Monday Lunch are very helpful for that.
Hi there,
-
Delays run the gamut. Jalil et al (2023) measure three years worth of dining choices, Weingarten et al. a few weeks; other studies are measuring whatās eaten at a dining hall during treatment and control but with no individual outcomes; and other studies are structured recall tasks like 3/ā7/ā30 days after treatment that ask people to say what they ate in a 24 hour period or over a given week. We did a bit of exploratory work on the relationship between length of delay and outcome size and didnāt find anything interesting.
-
Iām afraid we donāt know that overall. A few studies did moderator analysis where they found that people who scored high on some scale or personality factor tended to reduce their MAP consumption more, but no moderator stood out to us as a solid predictor here. Some studies found that women seem more amenable to messaging interventions, based on the results of Piester et al. 2020 and a few others, but some studies that exclusively targeted women found very little. I think gendered differences are interesting here but we didnāt find anything conclusive.
-
Hi Wayne,
Great questions, Iāll try to give them the thoughtful treatment they deserve.
We donāt place much (any?) credence in the statistical significance of the overall result, and I recognize that a lot of work is being done by the word āmeaningfullyā in āmeaningfully reducing.ā For us, changes on the order of a few percentage pointsāespecially given relatively small samples & vast heterogeneity of designs and contexts (hence our point about āwell-validatedāāalmost nothing is directly replicated out of sample in our database) -- are not the kinds of transformational change that others in this literature have touted. Another way to slice this, if you were looking to evaluate results based on significance, is to look at how many results are, according to their own papers, statistical nulls: 95 out of 112, or about 85%. (On the other hand, many of these studies might be finding small but real effects but not be sufficiently powered to identify them: If you plan for d > 0.4, an effect of d = 0.04 is going to look like a null, even if real changes are happening). So my basic conclusion is that marginal changes probably are possible, so in that sense, yes, many of these interventions probably āwork,ā but I wouldnāt call the changes transformative. I think the proliferation of GLP-1 drugs is much more likely to be transformative.
Itās true that cost-effectiveness estimates might still be very good even if the results are small. If there was a way to scale up the Jalil et al. intervention, Iād probably recommend it right away. But I donāt know of any such opportunity. (It requires getting professors to substitute out a normal economics lecture for one focused on meat consumption, and weād probably want at least a few other schools to do measurement to validate the effect, and my impression from talking to the authors is that measurement was a huge lift). I also think that choice architecture approaches are promising and awaiting a new era of evaluation. My lab is working on some of these; for someone interested in supporting the evaluation side of things, donating to the lab might be a good fit.
This is in the supplement rather than the paper, but one of our depressing results is that rigorous evaluations published by nonprofits, such as The Humane League, Mercy For Animals, and Faunalytics, produce a small backlash on average (see table below). But itās also my impression that a lot of these groups have changed gears a lot, and are now focusing less on (e.g.) leafletting and direct persuasion efforts and more on corporate campaigns, undercover investigations, and policy work. I donāt know if they have moved this direction specifically because a lot of their prior work was showing null/ābacklash results, but in general I think this shift is a good idea given the current research landscape.
4. Pursuant to that, economists working on this sometimes talk about the consumer-citizen gap, where people will support policies that ban practices whose products theyāll happily consume. (People are weird!) For my money, if I were a significant EA donor on this space, I might focus here: message testing ballot initiatives, preparing for lengthy legal battles, etc. But as always with these things, the details matter. If you ban factory farms in California and lead Californians to source more of their meat from (e.g.) Brazil, and therefore cause more of the rainforest to be clearcutāwell thatās not obviously good either.
5. Almost all interventions in our database targeted meat rather than other animal products (one looked at fish sauce and a couple also measured consumption of eggs and dairy). Also a lot of studies just say the choice was between a meat dish and a vegetarian dish, and whether that vegetarian dish contained eggs or milk is sometimes omitted. But in general, Iād think of these as āless meatā interventions.
Sorry I canāt offer anything more definitive here about what works and where people should donate. An economist I like says his dadās first rule of social science research was: āSometimes itās this way, and sometimes itās that way,ā and I suppose I hew to that š
š Great questions!
Most studies in our dataset donāt report these kinds of fine-grained results, but in general my impression from the texts is that the typical study gets a lot of people to change their behavior a little. (In part because if they got people to go vegan I expect they would say that.)
Some studies deliberately exclude vegetarians as part of their recruitment process, but most just draw from whatever population at large. Somewhere between 2 and 5% of people identify as vegetarians (and many of them eat meat sometimes), so I donāt personally worry too much about this curtailing results. A few studies specifically recruit people who are motivated to change their diets and/āor help animals, e.g. Cooney (2016) recruited people who wanted to help Mercy for Animals evaluate its materials.
I think this is a fair mental model, but I think one of the main open questions of our paper is about how do we recruit people to cut back on meat in general vs. just cutting back on a few categories, e.g. red and processed meat. So I guess my mental model is that most people have heard that raising cows is bad for the environment and those who are cutting back are substituting partly to plant-based substitutes (reps from Impossible Foods noted at a recent meeting that most of their customers also eat meat) and partly to chicken and fish, e.g. the Mayo Clinicās page on heart-healthy diets suggests āLean meat, poultry and fish; low-fat or fat-free dairy products; and eggs are some of the best sources of protein...Fish is healthier than high-fat meatsā, although it also says that āEating plant protein instead of animal protein lowers the amounts of fat and cholesterol you take in.ā
So Iād say we still have a lot of open questions...
š Our pleasure!
To the best of my recollection, the only paper in our dataset that provides a cost-benefit estimation is Jalil et al. (2023)
Calculations indicate a high return on investment even under conservative assumptions (~US$14 per metric ton CO2eq). Our findings show that informational interventions can be cost effective and generate long-lasting shifts towards more sustainable food options.
Thereās also a red/āprocessed meat studyāEmmons et al. (2005) --- that does some cost-effectiveness analyses, but itās almost 20 years old and its reporting is really sparse: changes to the eating environment āwere not reported in detail, precluding more detailed analyses of this intervention.ā So Iād stick with Jalil et al. to get a sense of ballpark estimates.
What is the average charity? I donāt have a good intuition for what it looks like, is, how big it is, what it works on etc.[1] I think pinning this down will help make the comparison clearer. Will, how do you think about this?
Sidenote: At least in the US, I would be open to the argument that the average charityādefined as being the midpoint of some multidimensional array of size, cause area, staffing, location, etc. -- produces literally zero charitable benefit on net, and might even be doing harm. You might not share this intuition, but we have a long list of mostly null effects for pro-social interventions once theyāre evaluated rigorously (enterprise zones in California, medicaid enrollment in oregon, head start, etc. -- any of which you might take issue with but I think the broader point is defensible that on average, interventions donāt work.) If the average social utility gain of a given nonprofit America is zero, then I donāt know how weāre going to say some other cause is X or Y times ābetterā than that. The seeing eye dog vs curing blindness comparison is a lot more coherent, I think.