In case it’s helpful, I figured I’d share a few reasons why these quantifiable metrics are so tricky to find.
It’s not obvious how to calculate a given metric (such as DALYs). DALYs are calculated as the sum of YLLs YLDs. Implicitly, this assumes YLLs averted would be a good thing. However, if you believe broiler chickens living in battery cages are living a net-negative life (not an unpopular opinion in EA), then this is pretty obviously not the case. Therefore, YLLs would actually have a positive effect on welfare because YLLs imply the suffering has ended? Of course this rests on the idea the chicken is not going to be replaced by another chicken, which it almost certainly will be. This would imply YLLs don’t matter at all for at least some farm animals. However, maybe the killing of the animal matters too? Again, not a crazy opinion to have (at least in the animal movement more broadly). Then YLLs would be affected by this, but probably not in a way aligning with what you intuitively think matters.
There is not common agreement about which metrics matter. In the GHD space, some people care more about QALYs, others DALYs, and others lives saved. However, in reality, everyone cares about all three and all the metrics are pretty closely related to each other. This is not the case in the animal space. Many in the animal space care about the abolition of the factory farming system while others put more value on reducing suffering in the short term. These interventions CAN be somewhat different (and there’s plenty of debate about how symbiotic these frameworks are or can be). Others will care about whether animals have rights. This is an entirely different framing which may not overlap with other frameworks at all for some interventions. In GHD, it’s pretty generally agreed upon that women and people in poverty should have rights and also that having rights is a component of their quality of life. With animals, many people disagree on some of these basic concepts.
Interaction effects between different interventions play such a large role in the effectiveness of an intervention. When one organization gets a company to sign a cage-free commitment, most reasonable metrics would give them credit for the welfare benefits of the commitment. However, sometimes a more radical animal rights group has been protesting their activity for years and this made it much easier for the more moderate organization to succeed. However, how would we attribute this contribution to the more radical group? It’s not obvious at all. My guess is you won’t reach reasonable agreement on this no matter how you quantify it. This type of thinking about change in the animal space is the reason why plurality in the movement is so valued my many (including myself) and, again, virtually any metric of cost-effectiveness of a given charity would miss this critical component of impact/social change.
The funding in the animal space is so much lower than in the GHD space. The GHD space has many billions of dollars to allocate to support a few billion people. The farmed animal space has about $200 million to allocate to support trillions of animals. This difference implies there is much less benefit to developing precise cost-effectiveness analyses for every intervention in every context. Rather, we put more resources into implementation and try to get a reasonable idea of the effectiveness of different interventions.
There are far fewer (human) stakeholders in some areas of the animal space and even fewer who are friendly. Working in MEL in the animal space, I’ve been able to adapt a lot of the standard practices in the GHD space. However, the biggest difference I see (other than the funding environment) is how GHD charities generally have many thousands of people in their target group and will target a few thousand each year while many animal charities have a few hundred people in their target group and they will likely target a few dozen per year (some target far fewer). This is particularly true when it comes to corporate pressure campaigns. Additionally, when you target corporations with a pressure campaign, your target group generally hates you. These differences mean methods pretty common in the GHD space, such as widescale surveying, oftentimes do not work at all in the animal space. For pressure campaigns, you’re lucky if you can conduct some in-depth interviews with members of your target group, but this is often not possible because of the hostile nature of your relationship with them.
All of these point together generally support a much more qualitative type of evaluation in the animal space. While qualitative information may be much less precise, it has the possibility of being much more accurate. Meanwhile quantitative metrics are likely to be more precise, but are also very prone to being inaccurate given the budget and sample size constraints.
Thank you for your comment, it makes sense why quantitatively measuring a charity as a whole is extremely difficult and not as meaningful as I would think.
In case it’s helpful, I figured I’d share a few reasons why these quantifiable metrics are so tricky to find.
It’s not obvious how to calculate a given metric (such as DALYs). DALYs are calculated as the sum of YLLs YLDs. Implicitly, this assumes YLLs averted would be a good thing. However, if you believe broiler chickens living in battery cages are living a net-negative life (not an unpopular opinion in EA), then this is pretty obviously not the case. Therefore, YLLs would actually have a positive effect on welfare because YLLs imply the suffering has ended? Of course this rests on the idea the chicken is not going to be replaced by another chicken, which it almost certainly will be. This would imply YLLs don’t matter at all for at least some farm animals. However, maybe the killing of the animal matters too? Again, not a crazy opinion to have (at least in the animal movement more broadly). Then YLLs would be affected by this, but probably not in a way aligning with what you intuitively think matters.
There is not common agreement about which metrics matter. In the GHD space, some people care more about QALYs, others DALYs, and others lives saved. However, in reality, everyone cares about all three and all the metrics are pretty closely related to each other. This is not the case in the animal space. Many in the animal space care about the abolition of the factory farming system while others put more value on reducing suffering in the short term. These interventions CAN be somewhat different (and there’s plenty of debate about how symbiotic these frameworks are or can be). Others will care about whether animals have rights. This is an entirely different framing which may not overlap with other frameworks at all for some interventions. In GHD, it’s pretty generally agreed upon that women and people in poverty should have rights and also that having rights is a component of their quality of life. With animals, many people disagree on some of these basic concepts.
Interaction effects between different interventions play such a large role in the effectiveness of an intervention. When one organization gets a company to sign a cage-free commitment, most reasonable metrics would give them credit for the welfare benefits of the commitment. However, sometimes a more radical animal rights group has been protesting their activity for years and this made it much easier for the more moderate organization to succeed. However, how would we attribute this contribution to the more radical group? It’s not obvious at all. My guess is you won’t reach reasonable agreement on this no matter how you quantify it. This type of thinking about change in the animal space is the reason why plurality in the movement is so valued my many (including myself) and, again, virtually any metric of cost-effectiveness of a given charity would miss this critical component of impact/social change.
The funding in the animal space is so much lower than in the GHD space. The GHD space has many billions of dollars to allocate to support a few billion people. The farmed animal space has about $200 million to allocate to support trillions of animals. This difference implies there is much less benefit to developing precise cost-effectiveness analyses for every intervention in every context. Rather, we put more resources into implementation and try to get a reasonable idea of the effectiveness of different interventions.
There are far fewer (human) stakeholders in some areas of the animal space and even fewer who are friendly. Working in MEL in the animal space, I’ve been able to adapt a lot of the standard practices in the GHD space. However, the biggest difference I see (other than the funding environment) is how GHD charities generally have many thousands of people in their target group and will target a few thousand each year while many animal charities have a few hundred people in their target group and they will likely target a few dozen per year (some target far fewer). This is particularly true when it comes to corporate pressure campaigns. Additionally, when you target corporations with a pressure campaign, your target group generally hates you. These differences mean methods pretty common in the GHD space, such as widescale surveying, oftentimes do not work at all in the animal space. For pressure campaigns, you’re lucky if you can conduct some in-depth interviews with members of your target group, but this is often not possible because of the hostile nature of your relationship with them.
All of these point together generally support a much more qualitative type of evaluation in the animal space. While qualitative information may be much less precise, it has the possibility of being much more accurate. Meanwhile quantitative metrics are likely to be more precise, but are also very prone to being inaccurate given the budget and sample size constraints.
Thank you for your comment, it makes sense why quantitatively measuring a charity as a whole is extremely difficult and not as meaningful as I would think.