This is great data to have! Thanks for collecting and sharing it. I think the Sioux Falls (Metaculus underestimate of the 48% ban support) and Swiss (Metaculus overestimate of the 37% ban support) factory farming ban proposals are particularly interesting opportunities to connect this survey data to policy results. I’ll share a few scattered, preliminary thoughts to spark discussion, and I hope to see more work on this topic in the future.
These 2022 results seem to be in line with the very similar surveys conducted by Rethink Priorities in 2019, which I found very useful, but I don’t know if those results have been shared publicly. Will you be sharing that data too? I know it’s been eagerly anticipated, and Sentience Institute has held off on similar work while waiting for it. I’m not sure if that 2019 data is now seen as just a pilot for this 2022 data collection?
In 2017, we preregistered credible intervals and informally solicited estimates from many others. The data was surprisingly ban-supporting relative to priors, which may be a more important takeaway than any post-hoc explanation. I didn’t preregister any CIs for the 2019 or 2022 RP results. I think these drops in ban support are around what I’d expect, but it’s very hard to say in hindsight, especially with other variation (e.g., different outcome scales, presumably different samples).
The Sentience Institute AFT survey also has questions with pro/con information, e.g., “Some people think that we should ban all animal farming and transition to plant-based and cultured foods, to reduce harm to humans and animals. Others think that we should keep using animals for food, to provide the conventional meat consumers are used to eating. Where would you place yourself on this scale?” (wording based on the GSS). That wording seems to elicit much stronger ban support than this new wording (though take with a large grain of salt due to other variation in the surveys), which seems to make sense as it is much more ban-supporting than the ban-opposing “it is wrong to kill animals” and “right to eat meat if they choose” wordings. Concretely, on a 1–6 support scale, we found a mean of 4.12 (95% CI: 4.04–4.21) for “ban all animal farming” with our nationally representative sample in 2021. I think it’s fair to say that’s much higher despite also having pro/con information, and I think it’s important qualification for interpreting the 2022 RP results that people may miss out on in this post.
Social scientists have long asked “Is there really any such thing as public opinion?” (Lewis 1939), and I think the majority answer has been some version of “public opinion does not exist” (e.g., Blumer 1948, Bourdieu 1972): There are many interesting wordings to consider: simple vs complex, ban-supporting vs ban-opposing, socially desirable and acquiescing vs socially undesirable and anti-acquiescing, politically left vs right favored, financially incentivized, politically engaged, etc. All question wordings matter, and none are objectively correct or objectively biased. I think we may disagree on this point because you say some question wordings “are biased towards answering “Yes”...”, though you may mean some subjective standard of bias, such as distance from likely counterfactual ballot measure results. Some wordings more naturally jibe with what people have in mind when they see survey results—I prioritize simple wordings in part for this reason—but ideally we share the exact survey wording alongside percentages or scores whenever possible to ensure that clarity.
If you have time, what was the sample (M Turk, Prolific, Civis Analytics, Ipsos Omnibus, Knowledge Panel, etc.), what were the demographics, and was it weighted for representativeness, and if so, how?
What exactly do you mean by “strong” in “strong basis for more radical action”? One operationalization I like is: All things considered, I think these survey and ballot results should update the marginal farmed animal advocate towards more radical approaches relative to their prior. I’d love to know if you agree.
Well-done again on this very interesting work! [minor edits made to this comment for clarity and fixing typos]
In 2019 we conducted some exploratory small-N, low-confidence studies on this topic that informed these high-quality, larger N studies. We feel comfortable presenting these recent results as we want to promote the norm of advocates choosing strategies and messages based on the best quality evidence, so would rather the community update based on the results of high-quality studies rather than low-confidence small-N studies where the wrong inferences may be drawn.
[Update 2022-Nov-18: I added a methods section to make it clearer for readers. Thanks Jacy for flagging this issue]. Respondents were recruited via Prolific and surveyed using Qualtrics and were weighted for representativeness using 5-year 2019 American community survey data and general social survey data. More details and data are available in the OSF project, but let us know if it’s not clear or something is missing since the 2nd survey was part of a larger survey that is described here which we have not finished publication of yet. (minor update- the pre-registration for the experimental study was still, unintentionally, embargoed at the time of publication so that information wasn’t easily accessible for readers, but it has been lifted and should be available here by Nov 12)
I don’t think one should take a survey showing 16% support for banning slaughterhouses when animal welfare frames are used as reason for the marginal farmed animal advocate to update towards more radical approaches relative to their prior, unless that farmed advocate assumed there would be basically 0% support. In practice, I imagine many advocates have been relying on surveys showing much higher support as their benchmark so this survey would update them toward more pessimism.
More generally, given the clear variation in support results from well-done surveys from different organizations on this specific item it doesn’t seem advisable for advocates to update their beliefs strongly on how support for other specific or the whole class of radical policies will be. As we wrote, it would be better to test the specific policies in question with a variety of messages to get closer to understanding how the public might actually act if given the chance to vote. And of course compare these to more moderate asks that the movement thinks are also worthwhile in expectation. I don’t think we should take attitudes towards a specific policy as very indicative of attitudes to all of animal agriculture or other anti-factory farming policies. I also don’t think we should take attitudes towards all of animal agriculture as indicative of support for specific policy proposals advocated for under specific messages. It was the claim that support for banning slaughterhouses should encourage advocates to push stronger messages and policies that we wanted to raise more questions about.
Thanks for the reminder about the 2021 AFT survey!- which if I’m correct also showed ~43% support for banning slaughterhouses, when including No Opinion. I’ll add it to the report.
We should also note that Norwood (one of the authors who replicated SI’s original 2017 study) this year ran a new slaughterhouse ban survey experiment ([Britton & Norwood 2022](https://doi.org/10.1017/aae.2022.17)) and found lower support. (I only just received the data from them so I couldn’t include it in the post).
Here is my summary from just skimming the article and quickly aggregating the data.
They test a hypothesis that the question ordering in the 2017 SI study cued respondents’ ideal self (like whether voting is a moral virtue) rather than their common self (like whether they actually voted). Their theory is that by asking respondents first whether they agreed with statements about meat reduction, discomfort with the way animals are used in the food industry, and animal sentience it cued their ideal self so that “the desire to not appear hypocritical induced them to activate a mixture of their ideal and common self” when answering questions about bans on animal farming, factory farming, and slaughterhouses.
The actual design of their study is a little too complicated to explain here (involving four treatments that altered the order and wording of ideal and common self questions, some food-related and some non-food related, as well as inserting buffer questions), but basically some respondents saw the ban questions before the ideal self questions, and others saw them in the same order as in the original 2017 SI study. Furthermore, to build on their tests about whether respondents understood the implications of bans, “roughly half of the subjects are given the [common self] statements exactly as they appeared on the Animal Sentience survey, while the other half contain an addition [. . .] For example, some see the statement “I support a ban on slaughterhouses” while others see the statement “I support a ban on slaughterhouses and will stop eating meat”. ”
While the primary aim of their study was to test something they call “identity inertia” and they fail to find convincing evidence of it, their finding on the slaughterhouse ban issue was “once individuals are informed about the implications of actions like banning slaughterhouses, they are less eager to do so.”
Data were collected via an online survey through Qualtrics from August to October 2019 of a representative sample of nearly 2600 drawn from the U.S. population. A subset of the results (N=1528) show
26.48% “agreed” with the statement “I support a ban on slaughterhouses” (36.5% disagreed, 37.02% no opinion) (N=759)
18.21% “Agreed” with the statement “I support a ban on slaughterhouses and will stop eating meat.” (58.13% disagreed, 23.67% no opinion (N=769)
(Though Norwood say they couldn’t confirm this was correct since they never went into the data to get raw numbers like that, and I couldn’t see an easy way to break these results down according to whether respondents saw the ideal-self or common-self questions first- though that probably doesn’t matter since Norwood didn’t find a lot of evidence that it matters)
This is great data to have! Thanks for collecting and sharing it. I think the Sioux Falls (Metaculus underestimate of the 48% ban support) and Swiss (Metaculus overestimate of the 37% ban support) factory farming ban proposals are particularly interesting opportunities to connect this survey data to policy results. I’ll share a few scattered, preliminary thoughts to spark discussion, and I hope to see more work on this topic in the future.
These 2022 results seem to be in line with the very similar surveys conducted by Rethink Priorities in 2019, which I found very useful, but I don’t know if those results have been shared publicly. Will you be sharing that data too? I know it’s been eagerly anticipated, and Sentience Institute has held off on similar work while waiting for it. I’m not sure if that 2019 data is now seen as just a pilot for this 2022 data collection?
In addition to 2017 and 2020, Sentience Institute asked these questions in the 2019 and 2021 Animals, Food, and Technology (AFT) surveys with similar results.
In 2017, we preregistered credible intervals and informally solicited estimates from many others. The data was surprisingly ban-supporting relative to priors, which may be a more important takeaway than any post-hoc explanation. I didn’t preregister any CIs for the 2019 or 2022 RP results. I think these drops in ban support are around what I’d expect, but it’s very hard to say in hindsight, especially with other variation (e.g., different outcome scales, presumably different samples).
The Sentience Institute AFT survey also has questions with pro/con information, e.g., “Some people think that we should ban all animal farming and transition to plant-based and cultured foods, to reduce harm to humans and animals. Others think that we should keep using animals for food, to provide the conventional meat consumers are used to eating. Where would you place yourself on this scale?” (wording based on the GSS). That wording seems to elicit much stronger ban support than this new wording (though take with a large grain of salt due to other variation in the surveys), which seems to make sense as it is much more ban-supporting than the ban-opposing “it is wrong to kill animals” and “right to eat meat if they choose” wordings. Concretely, on a 1–6 support scale, we found a mean of 4.12 (95% CI: 4.04–4.21) for “ban all animal farming” with our nationally representative sample in 2021. I think it’s fair to say that’s much higher despite also having pro/con information, and I think it’s important qualification for interpreting the 2022 RP results that people may miss out on in this post.
Social scientists have long asked “Is there really any such thing as public opinion?” (Lewis 1939), and I think the majority answer has been some version of “public opinion does not exist” (e.g., Blumer 1948, Bourdieu 1972): There are many interesting wordings to consider: simple vs complex, ban-supporting vs ban-opposing, socially desirable and acquiescing vs socially undesirable and anti-acquiescing, politically left vs right favored, financially incentivized, politically engaged, etc. All question wordings matter, and none are objectively correct or objectively biased. I think we may disagree on this point because you say some question wordings “are biased towards answering “Yes”...”, though you may mean some subjective standard of bias, such as distance from likely counterfactual ballot measure results. Some wordings more naturally jibe with what people have in mind when they see survey results—I prioritize simple wordings in part for this reason—but ideally we share the exact survey wording alongside percentages or scores whenever possible to ensure that clarity.
If you have time, what was the sample (M Turk, Prolific, Civis Analytics, Ipsos Omnibus, Knowledge Panel, etc.), what were the demographics, and was it weighted for representativeness, and if so, how?
What exactly do you mean by “strong” in “strong basis for more radical action”? One operationalization I like is: All things considered, I think these survey and ballot results should update the marginal farmed animal advocate towards more radical approaches relative to their prior. I’d love to know if you agree.
Well-done again on this very interesting work! [minor edits made to this comment for clarity and fixing typos]
Thanks for reading and engaging with our work!
In 2019 we conducted some exploratory small-N, low-confidence studies on this topic that informed these high-quality, larger N studies. We feel comfortable presenting these recent results as we want to promote the norm of advocates choosing strategies and messages based on the best quality evidence, so would rather the community update based on the results of high-quality studies rather than low-confidence small-N studies where the wrong inferences may be drawn.
[Update 2022-Nov-18: I added a methods section to make it clearer for readers. Thanks Jacy for flagging this issue]. Respondents were recruited via Prolific and surveyed using Qualtrics and were weighted for representativeness using 5-year 2019 American community survey data and general social survey data. More details and data are available in the OSF project, but let us know if it’s not clear or something is missing since the 2nd survey was part of a larger survey that is described here which we have not finished publication of yet. (minor update- the pre-registration for the experimental study was still, unintentionally, embargoed at the time of publication so that information wasn’t easily accessible for readers, but it has been lifted and should be available here by Nov 12)
I don’t think one should take a survey showing 16% support for banning slaughterhouses when animal welfare frames are used as reason for the marginal farmed animal advocate to update towards more radical approaches relative to their prior, unless that farmed advocate assumed there would be basically 0% support. In practice, I imagine many advocates have been relying on surveys showing much higher support as their benchmark so this survey would update them toward more pessimism.
More generally, given the clear variation in support results from well-done surveys from different organizations on this specific item it doesn’t seem advisable for advocates to update their beliefs strongly on how support for other specific or the whole class of radical policies will be. As we wrote, it would be better to test the specific policies in question with a variety of messages to get closer to understanding how the public might actually act if given the chance to vote. And of course compare these to more moderate asks that the movement thinks are also worthwhile in expectation. I don’t think we should take attitudes towards a specific policy as very indicative of attitudes to all of animal agriculture or other anti-factory farming policies. I also don’t think we should take attitudes towards all of animal agriculture as indicative of support for specific policy proposals advocated for under specific messages. It was the claim that support for banning slaughterhouses should encourage advocates to push stronger messages and policies that we wanted to raise more questions about.
Thanks for the reminder about the 2021 AFT survey!- which if I’m correct also showed ~43% support for banning slaughterhouses, when including No Opinion. I’ll add it to the report.
We should also note that Norwood (one of the authors who replicated SI’s original 2017 study) this year ran a new slaughterhouse ban survey experiment ([Britton & Norwood 2022](https://doi.org/10.1017/aae.2022.17)) and found lower support. (I only just received the data from them so I couldn’t include it in the post).
Here is my summary from just skimming the article and quickly aggregating the data.
They test a hypothesis that the question ordering in the 2017 SI study cued respondents’ ideal self (like whether voting is a moral virtue) rather than their common self (like whether they actually voted). Their theory is that by asking respondents first whether they agreed with statements about meat reduction, discomfort with the way animals are used in the food industry, and animal sentience it cued their ideal self so that “the desire to not appear hypocritical induced them to activate a mixture of their ideal and common self” when answering questions about bans on animal farming, factory farming, and slaughterhouses.
The actual design of their study is a little too complicated to explain here (involving four treatments that altered the order and wording of ideal and common self questions, some food-related and some non-food related, as well as inserting buffer questions), but basically some respondents saw the ban questions before the ideal self questions, and others saw them in the same order as in the original 2017 SI study. Furthermore, to build on their tests about whether respondents understood the implications of bans, “roughly half of the subjects are given the [common self] statements exactly as they appeared on the Animal Sentience survey, while the other half contain an addition [. . .] For example, some see the statement “I support a ban on slaughterhouses” while others see the statement “I support a ban on slaughterhouses and will stop eating meat”. ”
While the primary aim of their study was to test something they call “identity inertia” and they fail to find convincing evidence of it, their finding on the slaughterhouse ban issue was “once individuals are informed about the implications of actions like banning slaughterhouses, they are less eager to do so.”
Data were collected via an online survey through Qualtrics from August to October 2019 of a representative sample of nearly 2600 drawn from the U.S. population. A subset of the results (N=1528) show
26.48% “agreed” with the statement “I support a ban on slaughterhouses” (36.5% disagreed, 37.02% no opinion) (N=759)
18.21% “Agreed” with the statement “I support a ban on slaughterhouses and will stop eating meat.” (58.13% disagreed, 23.67% no opinion (N=769)
(Though Norwood say they couldn’t confirm this was correct since they never went into the data to get raw numbers like that, and I couldn’t see an easy way to break these results down according to whether respondents saw the ideal-self or common-self questions first- though that probably doesn’t matter since Norwood didn’t find a lot of evidence that it matters)