Great work! Thanks for doing this! This is encouraging and an update for me. I was originally optimistic about ballot initiatives, but I didn’t see any formal cost-effectiveness analyses, and I had heard a pessimistic take from an established member of the community.
I have two minor comments/suggestions:
1. You assume a speed-up time of 4 years for ballot initiatives, but this seems pretty low to me if we want to include potential future impacts (maybe you only wanted impacts so far?), because, for the counterfactual, I’d use “us”[1] not pushing for these ballot initiatives or lobbying for other reform legislation at all ever,[2] and I wouldn’t have expected similar reforms to pass within 4 years and be implemented 4 years later than otherwise without “us”. Similarly, as you wrote, Šimčikas 2019 “assumes a period of impact that is lognormally distributed between 4.2 and 40 years with 90% confidence, resulting in a mean period of impact of 15 years”, and I think he compared to counterfactuals where “we” don’t support these corporate campaigns at all and projected future impacts. I’d probably use a distribution similar to Šimčikas 2019′s, based on similar evidence cited by him and others for corporate campaign speed-up times (although I’m not really well-informed), which would make the historical US ballot initiatives pretty competitive ex ante with historical corporate chicken welfare campaigns even when their impacts are aggregated over time, and not just per year of impact. Otherwise, corporate campaigns look ~10x better on their direct effects when the impacts are aggregated over time,[3] and I might not conclude ballot initiative are competitive with them.
2. On the comparisons between corporate campaigns and ballot initiatives, if we’re maximizing the expected value of a sum of utilities, rather than expected values of ratios, it seems more informative to report ratios like
(expected impact of ballot initiatives/expected costs of ballot initiatives) / (expected impact of corporate campaigns/expected costs of corporate campaigns)[4]
This is basically to avoid two envelopes problems. These ratios of expected values should be lower than the means of ratios you reported, but not much lower, so it won’t change your conclusions. For example, you wrote:
In terms of years of suffering per year of impact, the three ballot initiatives that affected egg-laying hens were about 25% as cost-effective (11% to 47%) in terms of years of suffering per year of impact as were historical cage-free corporate campaigns. I estimate that historical ballot initiatives averted 0.025 (0.014 to 0.038) years of suffering per dollar per year of impact, compared to 0.12 (0.05 to 0.23) years of suffering per dollar per year of impact for corporate campaigns.
Taking ratios of means gives 0.025 / 0.12 = 0.2083333..., which is only slightly lower than the 25% you reported.
The credence intervals for the ratios and medians of the ratios can still be informative to report, though, but expected values of ratios of random variables can be misleading because of two envelopes problem-like concerns.
There’s a question of whom we’re including in “us”, e.g. only those close to the effective animal advocacy movement (by member self-identification, organization self-identification or funding from EAA), or also other animal advocates involved? For example, maybe other animal advocates would run ballot initiatives 4 years later without the support of EAA, but if they wouldn’t have done so or succeeded without EAA when the ballot initiatives were actually run, how likely do we think they’d be to succeed without EAA in their attempt 4 years later? And, at any rate, when considering the cost-effectiveness of future ballot initiatives, their costs seem relevant. (As far as I can tell, you did include other animal advocates’ costs supporting the initiatives, too.)
Or if “we” would support these reforms later, and you’re comparing to that, then you should use the difference in costs between doing it earlier and later, too, and do the same for corporate campaigns.
I believe 4 years is very conservative. I’m working on a paper due November that should basically answer the question in part 1, but suffice it to say I think the ballot measures should look many times more cost-effective than corporate campaigns.
Given similar costs per hen-year per year of impact according to Laura’s report, are you expecting ballot initiatives to have longer counterfactuals than corporate campaigns? Or, do you think ballot initatives are more cost-effective per hen-year per year of impact? (Or both?)
The former, though I don’t have estimates of the counterfactual timeline of corporate campaigns. (I’d like to find a way to do that and have toyed with it a bit but currently don’t have one.)
One consideration that Peter Wildeford made me think of is that, with the initiatives that do fall under Congress’ Interstate Commerce Clause authority, we might expect the longevity to be reduced. For example, if every five years a Congressperson puts into the Farm Bill a proposal to ban states from having Prop 12-style regulations, there’s some chance this passes eventually.
Does your research include any initiatives that do fall under Congressional authority?
Although a hostile Congress could also take steps to neuter the effects of corporate campaigns. Less likely than preempting stuff like Prop 12, but we’ve seen pretty hostile legislation at the state level like so-called “ag gag” laws.
Fascinating, I hadn’t thought about that with respect to Congress. One thing I wonder about with ag-gag laws is whether they run afoul of the First Amendment. Do you know if there’s a strong legal case to be made that they’re unconstitutional?
My gut instinct here would be that it’s probably somewhat harder to pass Congressional legislation that both is constitutional and effectively limits corporate campaigns (because it’s private entities choosing what kinds of products to sell). Am I wrong here? (I am really interested in this topic, so I would love to be corrected)
From that, my guess is that some form of ag gag can be done if carefully drawn; I don’t see a conclusion that no possible law would stand as consistent with the court’s prior decision in Food Lion.
I agree that the First Amendment makes much anti-corp campaign legislation much harder to pull off. My starting point was that the FA offers less protection to purely commercial speech, especially on stuff like product labeling, and so restrictions on what the company can say in advertising its products might pass muster. That could at least weaken corporate incentives.
My other theory is whether e.g. the pork industry could convince Congress that having various different animal-welfare standards based on various corporate policies disrupted the national pork market (akin to as how much a state policy allegedly would) and justified restrictions on those corporate policies. Sounds like a stretch, but if Congress can ban you from growing weed for personal use under the Commerce Clause (and it can under Gonzales vs Raich), the effects on interstate commerce seem much greater here...
Still, between the legal concerns and political realities, I’d estimate the hostile Congress risk for corp campaigns at roughly an order of magnitude less than the risk to state legislative initiatives (low confidence).
Yeah, I think that would reduce the longevity in expectation, maybe by something like 2x. My research includes things that could hypothetically fall under congressional authority and occasionally do. (Anything could fall under congressional authority, though some might require a constitutional amendment.) So I don’t think this is dramatically out of sample, but I do think it’s worth keeping in mind.
I’ll take the second one first: thanks for bringing to my attention the two-envelopes problem. I’ll look more into this, and I’ll revise accordingly!
As for the years of counterfactual impact, I wanted this report to err on the side of being too conservative, because the impact still appears to be pretty large even under this conservative assumption.
A couple of reasons as to why I used four years include:
1. It’s still relatively consistent with other organizations’ assumptions of years of counterfactual impact. Šimčikas 2019 also gives an overview of other cost-effectiveness analyses’ counterfactual impact assumptions, which I quote below:
“Bollard (2016) assumes that cage-free commitments accelerate changes by five years. He also adds: “In my view, the assumption that these campaigns only accelerated pledges by five years is very conservative. It seems equally likely that these companies would never have dropped battery cages, or would have merely transitioned to “enriched” cages. For instance, as recently as March 2015, a coalition backed by McDonald’s, General Mills, and other major food companies issued a report which largely endorsed “enriched” cages as an alternative to cage-free systems.
“ACE uses a subjective 90% confidence interval of 1.6 to 14 years (mean 5.6 years) for all corporate pledges. They explain that “This is the number of years for which we expect these commitments to have an effect for. It is primarily based on counterfactual reasoning—how long before another factor, such as a legislative change or a shift in consumer demand, leads to a similar result.”
“Capriati (2018) estimate does not have a direct equivalent to years of impact expected. Instead, it estimates the number of years THL moves the policy forward by. It assigns the value to this variable based on how important THL’s role was in bringing policies about. By analyzing six randomly selected campaigns, it concludes that on average, THL’s cage-free and broiler campaigns moved policies forward by one year. Note that this assumes that other organisations would have still done corporate campaigns.”
2. Given these estimates, four years seemed like an appropriate lower bound that also aligns with US political cycle. 3. I think, as you and zdgroff have pointed out, there are probably good reasons as to why the counterfactual impact period would be longer than that of corporate campaigns. I really look forward to reading this research! But I also wanted to maintain a degree of conservatism.
So perhaps one’s takeaway is: “this is a good lower bound on the cost-effectiveness of ballot initiatives and, if designed well, they can still look pretty competitive with corporate campaigns nonetheless.”
These were just some very conservative guesses rather than estimates. Also, I think that the effect depends on circumstances:
In the case of eggs and Prop 12, by the time it was passed, most companies in the U.S. had already committed to only use cage-free eggs, often by 2025 or 2026. So I guess you could say that Prop 12 made California do it sooner (2022) and hence sped it up (although it’s unclear if that is even a good thing in itself).[1] But a more important effect of Prop 12 is that it increased the probability that California and the whole U.S. will go cage-free in 2020s, and this is how I might model the impact of Prop 12. That is, I’d probably ask various people about what would they expect future cage-free rates to be with and without Prop 12.
For some other animals, I imagine that there were no corporate commitments or anything? The situation in such cases seems very different.
According to King (2019b), some producers react to cage-free commitments by building new cage-free facilities, but not destroying old conventional caged houses which don’t yet need to be replaced. This could increase the overall amount of hens (and suffering) in the short term. O’Keefe (2020) claims that between December 2016 and December 2019, “U.S. egg producers added 33.2 million head of cage-free hens, the number of cage-housed hens only declined by 4.3 million head.” Even if the number of caged hens will decline eventually, this trend during the transition period is worrying.
Great work! Thanks for doing this! This is encouraging and an update for me. I was originally optimistic about ballot initiatives, but I didn’t see any formal cost-effectiveness analyses, and I had heard a pessimistic take from an established member of the community.
I have two minor comments/suggestions:
1. You assume a speed-up time of 4 years for ballot initiatives, but this seems pretty low to me if we want to include potential future impacts (maybe you only wanted impacts so far?), because, for the counterfactual, I’d use “us”[1] not pushing for these ballot initiatives or lobbying for other reform legislation at all ever,[2] and I wouldn’t have expected similar reforms to pass within 4 years and be implemented 4 years later than otherwise without “us”. Similarly, as you wrote, Šimčikas 2019 “assumes a period of impact that is lognormally distributed between 4.2 and 40 years with 90% confidence, resulting in a mean period of impact of 15 years”, and I think he compared to counterfactuals where “we” don’t support these corporate campaigns at all and projected future impacts. I’d probably use a distribution similar to Šimčikas 2019′s, based on similar evidence cited by him and others for corporate campaign speed-up times (although I’m not really well-informed), which would make the historical US ballot initiatives pretty competitive ex ante with historical corporate chicken welfare campaigns even when their impacts are aggregated over time, and not just per year of impact. Otherwise, corporate campaigns look ~10x better on their direct effects when the impacts are aggregated over time,[3] and I might not conclude ballot initiative are competitive with them.
2. On the comparisons between corporate campaigns and ballot initiatives, if we’re maximizing the expected value of a sum of utilities, rather than expected values of ratios, it seems more informative to report ratios like
(expected impact of ballot initiatives/expected costs of ballot initiatives) / (expected impact of corporate campaigns/expected costs of corporate campaigns)[4]
This is basically to avoid two envelopes problems. These ratios of expected values should be lower than the means of ratios you reported, but not much lower, so it won’t change your conclusions. For example, you wrote:
Taking ratios of means gives 0.025 / 0.12 = 0.2083333..., which is only slightly lower than the 25% you reported.
The credence intervals for the ratios and medians of the ratios can still be informative to report, though, but expected values of ratios of random variables can be misleading because of two envelopes problem-like concerns.
There’s a question of whom we’re including in “us”, e.g. only those close to the effective animal advocacy movement (by member self-identification, organization self-identification or funding from EAA), or also other animal advocates involved? For example, maybe other animal advocates would run ballot initiatives 4 years later without the support of EAA, but if they wouldn’t have done so or succeeded without EAA when the ballot initiatives were actually run, how likely do we think they’d be to succeed without EAA in their attempt 4 years later? And, at any rate, when considering the cost-effectiveness of future ballot initiatives, their costs seem relevant. (As far as I can tell, you did include other animal advocates’ costs supporting the initiatives, too.)
Or if “we” would support these reforms later, and you’re comparing to that, then you should use the difference in costs between doing it earlier and later, too, and do the same for corporate campaigns.
In your figure “Hen Suffering Avoided per Dollar for Ballot Initiatives vs. Corporate Campaigns” from your model, you get a mean ratio of cost-effectivenesses of 0.1, so corporate campaigns would be ~10x better. Similarly, if I compare your 5.0 years of animal life improved per dollar spent more directly to Šimčikas 2019′s expected 41 (9 to 120) chicken-years affected per dollar, corporate campaigns look ~8x better.
Or expected impacts per year of impact
I believe 4 years is very conservative. I’m working on a paper due November that should basically answer the question in part 1, but suffice it to say I think the ballot measures should look many times more cost-effective than corporate campaigns.
Awesome, I’m looking forward to it!
Given similar costs per hen-year per year of impact according to Laura’s report, are you expecting ballot initiatives to have longer counterfactuals than corporate campaigns? Or, do you think ballot initatives are more cost-effective per hen-year per year of impact? (Or both?)
The former, though I don’t have estimates of the counterfactual timeline of corporate campaigns. (I’d like to find a way to do that and have toyed with it a bit but currently don’t have one.)
Maybe you can get estimates for corporate campaign counterfactuals from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4219976 or based on a similar methodology?
Oh, that’s a good idea. I had thought of something quite different and broader, but this also seems like a promising approach.
One consideration that Peter Wildeford made me think of is that, with the initiatives that do fall under Congress’ Interstate Commerce Clause authority, we might expect the longevity to be reduced. For example, if every five years a Congressperson puts into the Farm Bill a proposal to ban states from having Prop 12-style regulations, there’s some chance this passes eventually.
Does your research include any initiatives that do fall under Congressional authority?
Although a hostile Congress could also take steps to neuter the effects of corporate campaigns. Less likely than preempting stuff like Prop 12, but we’ve seen pretty hostile legislation at the state level like so-called “ag gag” laws.
Fascinating, I hadn’t thought about that with respect to Congress. One thing I wonder about with ag-gag laws is whether they run afoul of the First Amendment. Do you know if there’s a strong legal case to be made that they’re unconstitutional?
My gut instinct here would be that it’s probably somewhat harder to pass Congressional legislation that both is constitutional and effectively limits corporate campaigns (because it’s private entities choosing what kinds of products to sell). Am I wrong here? (I am really interested in this topic, so I would love to be corrected)
Quickly skimmed the recent Fourth Circuit ruling striking down an ag gag statute by 2-1 vote. See https://aldf.org/article/fourth-circuit-enjoins-north-carolina-ag-gag-law/
From that, my guess is that some form of ag gag can be done if carefully drawn; I don’t see a conclusion that no possible law would stand as consistent with the court’s prior decision in Food Lion.
I agree that the First Amendment makes much anti-corp campaign legislation much harder to pull off. My starting point was that the FA offers less protection to purely commercial speech, especially on stuff like product labeling, and so restrictions on what the company can say in advertising its products might pass muster. That could at least weaken corporate incentives.
My other theory is whether e.g. the pork industry could convince Congress that having various different animal-welfare standards based on various corporate policies disrupted the national pork market (akin to as how much a state policy allegedly would) and justified restrictions on those corporate policies. Sounds like a stretch, but if Congress can ban you from growing weed for personal use under the Commerce Clause (and it can under Gonzales vs Raich), the effects on interstate commerce seem much greater here...
Still, between the legal concerns and political realities, I’d estimate the hostile Congress risk for corp campaigns at roughly an order of magnitude less than the risk to state legislative initiatives (low confidence).
Yeah, I think that would reduce the longevity in expectation, maybe by something like 2x. My research includes things that could hypothetically fall under congressional authority and occasionally do. (Anything could fall under congressional authority, though some might require a constitutional amendment.) So I don’t think this is dramatically out of sample, but I do think it’s worth keeping in mind.
Hi Michael, thanks for the comments!
I’ll take the second one first: thanks for bringing to my attention the two-envelopes problem. I’ll look more into this, and I’ll revise accordingly!
As for the years of counterfactual impact, I wanted this report to err on the side of being too conservative, because the impact still appears to be pretty large even under this conservative assumption.
A couple of reasons as to why I used four years include:
1. It’s still relatively consistent with other organizations’ assumptions of years of counterfactual impact. Šimčikas 2019 also gives an overview of other cost-effectiveness analyses’ counterfactual impact assumptions, which I quote below:
“Bollard (2016) assumes that cage-free commitments accelerate changes by five years. He also adds: “In my view, the assumption that these campaigns only accelerated pledges by five years is very conservative. It seems equally likely that these companies would never have dropped battery cages, or would have merely transitioned to “enriched” cages. For instance, as recently as March 2015, a coalition backed by McDonald’s, General Mills, and other major food companies issued a report which largely endorsed “enriched” cages as an alternative to cage-free systems.
“ACE uses a subjective 90% confidence interval of 1.6 to 14 years (mean 5.6 years) for all corporate pledges. They explain that “This is the number of years for which we expect these commitments to have an effect for. It is primarily based on counterfactual reasoning—how long before another factor, such as a legislative change or a shift in consumer demand, leads to a similar result.”
“Capriati (2018) estimate does not have a direct equivalent to years of impact expected. Instead, it estimates the number of years THL moves the policy forward by. It assigns the value to this variable based on how important THL’s role was in bringing policies about. By analyzing six randomly selected campaigns, it concludes that on average, THL’s cage-free and broiler campaigns moved policies forward by one year. Note that this assumes that other organisations would have still done corporate campaigns.”
2. Given these estimates, four years seemed like an appropriate lower bound that also aligns with US political cycle.
3. I think, as you and zdgroff have pointed out, there are probably good reasons as to why the counterfactual impact period would be longer than that of corporate campaigns. I really look forward to reading this research! But I also wanted to maintain a degree of conservatism.
So perhaps one’s takeaway is: “this is a good lower bound on the cost-effectiveness of ballot initiatives and, if designed well, they can still look pretty competitive with corporate campaigns nonetheless.”
Again, thanks for the comment!
These were just some very conservative guesses rather than estimates. Also, I think that the effect depends on circumstances:
In the case of eggs and Prop 12, by the time it was passed, most companies in the U.S. had already committed to only use cage-free eggs, often by 2025 or 2026. So I guess you could say that Prop 12 made California do it sooner (2022) and hence sped it up (although it’s unclear if that is even a good thing in itself).[1] But a more important effect of Prop 12 is that it increased the probability that California and the whole U.S. will go cage-free in 2020s, and this is how I might model the impact of Prop 12. That is, I’d probably ask various people about what would they expect future cage-free rates to be with and without Prop 12.
For some other animals, I imagine that there were no corporate commitments or anything? The situation in such cases seems very different.
According to King (2019b), some producers react to cage-free commitments by building new cage-free facilities, but not destroying old conventional caged houses which don’t yet need to be replaced. This could increase the overall amount of hens (and suffering) in the short term. O’Keefe (2020) claims that between December 2016 and December 2019, “U.S. egg producers added 33.2 million head of cage-free hens, the number of cage-housed hens only declined by 4.3 million head.” Even if the number of caged hens will decline eventually, this trend during the transition period is worrying.
I’ll also note that I think the counterfactual impact period was one of the model decisions I struggled the most with, which is why you can change it in the model here and see how the results change! https://my.causal.app/models/165404?token=6e8998626d0643db9c86482475aecc2c
Is that lognormal distribution responsible for
If yes, what’s the intuition behind this distribution? If not, why is cost-effectiveness non-linear in speed-up time?