Thank you for your thoughtful analysis. This is helpful for us to understand areas that must be improved or made clearer. This is especially important as we will be soon publishing the full Welfare Footprint of the Egg (various systems), where we analyse over 120 experiences (diseases, injuries, deprivations, imbalances – nearly everything we could identify), in different housing systems, from birth to slaughter (more info here), for layers and breeders.
On your specific points on the direction of the results: there are various ways in which the existing analysis was conservative (i.e. favored caged systems). For example, in estimating the prevalence of keel fractures and other ailments in cage-free systems, we considered prevalences as they were reported, which typically was in the first few cycles of experience with cages. Evidence indicates that these prevalence rates go down as management experience increases (examples in the Prevalence Chapter), yet we preferred not to make that assumption and use the data as it was. Also, we did not take into account positive welfare (greater in cage-free settings) and more diffuse experiences, like fear, helplessness and boredom (greater in cages). Nor did we consider the flow-through effects of frustrations from behavioural deprivations beyond the period corresponding to the time budget of engagement, or practices like induced-molting (more likely in cages), or the longer cycles of caged-hens (with end disproportionally worse at the end)
Importantly, there is substantial evidence indicating that Pain is more intense (and healing delayed), even for the same injury/disease, in cages than in cage-free systems. We did not consider these modulatory effects, but they are likely present. That said, we’ll look deeper into the references you mentioned.
Unfortunately, many existing comparisons of cage and cage-free cite the CSES studies, to which the analysis from Fulton are part of. These studies were funded by the American Egg Board and facilitated by another industry-funded organization focused on building consumer confidence and maintaining the industry’s viability. While management in the caged systems was good and based on decades of experience, the cage-free systems were implemented for the first time, and did not adopt nearly any of the good management practices required. For example, during the laying phase, birds in the aviary were confined for many weeks before accessing the floor litter (something that makes injurious pecking much more likely). Also, insufficient space allowance and perch space in the aviary led to crowding, collisions, and failed landings, likely contributing to the high rate of keel injury and mortality. The authors themselves declared publicly they were still learning about what to do in the aviary systems during the research, which led to many failures. These design and management failures likely substantially inflated the negative outcomes in CF systems, including mortality (mortality data is also inconsistently reported in their publications, with some mortality—e.g. during placement—apparently excluded from caged systems).
Some info that may be useful:
Keel injuries: so far, the most consensual explanation is the earlier onset of laying, before the keel bone is ossified. This affects both cage and cage-free systems.
Nest deprivation: the justification for the ranges of durations and probabilities of distress intensities is provided in this chapter. These inferences are grounded in multiple lines of evidence (reviewed in the chapter). As such, they remain open to revision if new data or alternative interpretations of the evidence emerge. If you believe that the evidence supports a different intensity, please share your reasoning. This kind of scientific exchange is precisely what helps us approach values that are likely closer to reality.
Mortality: the lack of differences in mortality is seen in many datasets, with and without beak trimming (the control mentioned in the 2021 meta-analysis is for Week’s data, not for the meta-analysis data). Industry data also shows that the best cage free systems have lower mortality than the best caged systems.
More generally, both in the prior analysis, and in the forthcoming book, a major issue is data scarcity. Therefore, we inevitably rely on estimating uncertainty ranges for parameters like duration and prevalence. Inter-rater agreements should be made available together with future estimates, but what we have seen so far shows reasonable levels of agreement among WFI and independent academic raters/estimators.
As we build more comprehensive analyses, we’d be keen to have our estimates scrutinized as you did, so thank you!
Aaron, this is a great idea. I strongly agree that bringing research closer to commercial farms is essential if we want findings that actually reflect what happens in practice, with much of what is produced in research settings (even those that try to mimick commercial practice) suffering from what we call the ‘healthy farm effect’. Commercial data capture the full messiness of real production, which is why it is so valuable.
External validity is not the only thing missing in welfare science. Most of what we know about animal welfare at commercial scale comes from single visits to farms, essentially a photograph of what happens. We need the video (longitudinal research). We need to know when different welfare problems start, how long they last, and how many animals are affected. In short, we need an epidemiology of animal welfare, and commercial farms are the only place where that can happen.
That said, for this to work, I believe a few things are needed:
Independence between funding and publication. In my experience working with industry-funded groups, there are often contractual or simply informal pressures (e.g. anticipation of future fundign) that discourage publishing unfavorable results . Agreements need to explicitly guarantee the right to publish regardless of outcome.
Independent analysis. Data collection can happen on farms, but analysis should be conducted or audited independently to reduce potential biases.
A clever design of incentives. Farmers should benefit from participating, but incentives shouldn’t promote selective reporting.
Standardization. The power of research on commercial farms will come from collecting consistents data and indicators across many farms, enabling large scale analyses with the proper statistical power and validity. This needs pre-defined protocols, clear definitions of variables and methods, preregistration where possible, and transparent data access. Without this, farm data risk being too messy or selectively reported.
With these safeguards, I believe research on farms will be of immense value. Farms are already generating huge amounts of data as you mentioned, one challenge now is creating the proper systems to use it .