I’ll take the second one first: thanks for bringing to my attention the two-envelopes problem. I’ll look more into this, and I’ll revise accordingly!
As for the years of counterfactual impact, I wanted this report to err on the side of being too conservative, because the impact still appears to be pretty large even under this conservative assumption.
A couple of reasons as to why I used four years include:
1. It’s still relatively consistent with other organizations’ assumptions of years of counterfactual impact. Šimčikas 2019 also gives an overview of other cost-effectiveness analyses’ counterfactual impact assumptions, which I quote below:
“Bollard (2016) assumes that cage-free commitments accelerate changes by five years. He also adds: “In my view, the assumption that these campaigns only accelerated pledges by five years is very conservative. It seems equally likely that these companies would never have dropped battery cages, or would have merely transitioned to “enriched” cages. For instance, as recently as March 2015, a coalition backed by McDonald’s, General Mills, and other major food companies issued a report which largely endorsed “enriched” cages as an alternative to cage-free systems.
“ACE uses a subjective 90% confidence interval of 1.6 to 14 years (mean 5.6 years) for all corporate pledges. They explain that “This is the number of years for which we expect these commitments to have an effect for. It is primarily based on counterfactual reasoning—how long before another factor, such as a legislative change or a shift in consumer demand, leads to a similar result.”
“Capriati (2018) estimate does not have a direct equivalent to years of impact expected. Instead, it estimates the number of years THL moves the policy forward by. It assigns the value to this variable based on how important THL’s role was in bringing policies about. By analyzing six randomly selected campaigns, it concludes that on average, THL’s cage-free and broiler campaigns moved policies forward by one year. Note that this assumes that other organisations would have still done corporate campaigns.”
2. Given these estimates, four years seemed like an appropriate lower bound that also aligns with US political cycle. 3. I think, as you and zdgroff have pointed out, there are probably good reasons as to why the counterfactual impact period would be longer than that of corporate campaigns. I really look forward to reading this research! But I also wanted to maintain a degree of conservatism.
So perhaps one’s takeaway is: “this is a good lower bound on the cost-effectiveness of ballot initiatives and, if designed well, they can still look pretty competitive with corporate campaigns nonetheless.”
These were just some very conservative guesses rather than estimates. Also, I think that the effect depends on circumstances:
In the case of eggs and Prop 12, by the time it was passed, most companies in the U.S. had already committed to only use cage-free eggs, often by 2025 or 2026. So I guess you could say that Prop 12 made California do it sooner (2022) and hence sped it up (although it’s unclear if that is even a good thing in itself).[1] But a more important effect of Prop 12 is that it increased the probability that California and the whole U.S. will go cage-free in 2020s, and this is how I might model the impact of Prop 12. That is, I’d probably ask various people about what would they expect future cage-free rates to be with and without Prop 12.
For some other animals, I imagine that there were no corporate commitments or anything? The situation in such cases seems very different.
According to King (2019b), some producers react to cage-free commitments by building new cage-free facilities, but not destroying old conventional caged houses which don’t yet need to be replaced. This could increase the overall amount of hens (and suffering) in the short term. O’Keefe (2020) claims that between December 2016 and December 2019, “U.S. egg producers added 33.2 million head of cage-free hens, the number of cage-housed hens only declined by 4.3 million head.” Even if the number of caged hens will decline eventually, this trend during the transition period is worrying.
Hi Michael, thanks for the comments!
I’ll take the second one first: thanks for bringing to my attention the two-envelopes problem. I’ll look more into this, and I’ll revise accordingly!
As for the years of counterfactual impact, I wanted this report to err on the side of being too conservative, because the impact still appears to be pretty large even under this conservative assumption.
A couple of reasons as to why I used four years include:
1. It’s still relatively consistent with other organizations’ assumptions of years of counterfactual impact. Šimčikas 2019 also gives an overview of other cost-effectiveness analyses’ counterfactual impact assumptions, which I quote below:
“Bollard (2016) assumes that cage-free commitments accelerate changes by five years. He also adds: “In my view, the assumption that these campaigns only accelerated pledges by five years is very conservative. It seems equally likely that these companies would never have dropped battery cages, or would have merely transitioned to “enriched” cages. For instance, as recently as March 2015, a coalition backed by McDonald’s, General Mills, and other major food companies issued a report which largely endorsed “enriched” cages as an alternative to cage-free systems.
“ACE uses a subjective 90% confidence interval of 1.6 to 14 years (mean 5.6 years) for all corporate pledges. They explain that “This is the number of years for which we expect these commitments to have an effect for. It is primarily based on counterfactual reasoning—how long before another factor, such as a legislative change or a shift in consumer demand, leads to a similar result.”
“Capriati (2018) estimate does not have a direct equivalent to years of impact expected. Instead, it estimates the number of years THL moves the policy forward by. It assigns the value to this variable based on how important THL’s role was in bringing policies about. By analyzing six randomly selected campaigns, it concludes that on average, THL’s cage-free and broiler campaigns moved policies forward by one year. Note that this assumes that other organisations would have still done corporate campaigns.”
2. Given these estimates, four years seemed like an appropriate lower bound that also aligns with US political cycle.
3. I think, as you and zdgroff have pointed out, there are probably good reasons as to why the counterfactual impact period would be longer than that of corporate campaigns. I really look forward to reading this research! But I also wanted to maintain a degree of conservatism.
So perhaps one’s takeaway is: “this is a good lower bound on the cost-effectiveness of ballot initiatives and, if designed well, they can still look pretty competitive with corporate campaigns nonetheless.”
Again, thanks for the comment!
These were just some very conservative guesses rather than estimates. Also, I think that the effect depends on circumstances:
In the case of eggs and Prop 12, by the time it was passed, most companies in the U.S. had already committed to only use cage-free eggs, often by 2025 or 2026. So I guess you could say that Prop 12 made California do it sooner (2022) and hence sped it up (although it’s unclear if that is even a good thing in itself).[1] But a more important effect of Prop 12 is that it increased the probability that California and the whole U.S. will go cage-free in 2020s, and this is how I might model the impact of Prop 12. That is, I’d probably ask various people about what would they expect future cage-free rates to be with and without Prop 12.
For some other animals, I imagine that there were no corporate commitments or anything? The situation in such cases seems very different.
According to King (2019b), some producers react to cage-free commitments by building new cage-free facilities, but not destroying old conventional caged houses which don’t yet need to be replaced. This could increase the overall amount of hens (and suffering) in the short term. O’Keefe (2020) claims that between December 2016 and December 2019, “U.S. egg producers added 33.2 million head of cage-free hens, the number of cage-housed hens only declined by 4.3 million head.” Even if the number of caged hens will decline eventually, this trend during the transition period is worrying.
I’ll also note that I think the counterfactual impact period was one of the model decisions I struggled the most with, which is why you can change it in the model here and see how the results change! https://my.causal.app/models/165404?token=6e8998626d0643db9c86482475aecc2c