Senior Researcher in the Farmed Animal Welfare Team at Rethink Priorities.
Sagar K Shah
Karma: 343
Thank @Vasco Grilo you for your thoughtful comments. Appreciate it!
I don’t think you’ve missed anything. I think you’ve identified a very valid critique of the assumptions I used to express cost-effectiveness as a cost per DALY averted range. Expressing the welfare ranges in units of seconds or years is a great way of bringing this out – so thank you for doing that.
Some comments:
If I were to rebuild the cost-effectiveness model, with the benefit of hindsight (and more time), I’d have probably used a probabilistic rather than deterministic variable for the assumption converting the % improvement in the human welfare range (for one year) that is equivalent to averting a DALY.
I’m pretty sure assumption feeds through linearly into the $/DALY results. So if you believed an assumption of 5% of human welfare range was more appropriate than 50%, you could divide the 5th and 95th percentiles of the cost per DALY averted range.
The formal sensitivity tests I did suggest the conclusions of how this intervention looks compared to the most promising GHD and animal welfare interventions wouldn’t change with ‘relatively small’ adjustments to the assumptions needed to convert results into DALY space (e.g. doubling the fish welfare range relative to humans and assuming averting a DALY is equivalent to a human intervention that raises human welfare by 10% of the human welfare range for 1 year).
I think once you start making bigger adjustments to these assumptions, you can run into the risk of being criticised for placing too much moral value on short-duration but high intensity suffering. I don’t think we have good empirical evidence to support any particular assumption here.
The moral value section of the results more formally illustrates how the fish stunning intervention compares to various $/DALY benchmarks depending on the moral value you might assign to improving a year of fish life via the intervention relative to averting a DALY.
I don’t think the narrative expressed in the executive summary would change even if I were to change the assumption on the moral value of averting intense suffering relative to extending healthy lifespan.
While I think there is a lot of value in trying to place results into a ‘common currency’, I think this is also a good reason why cost per DALY averted numbers should always be treated with some caution (there is will always be moral value judgements there, some of which may be objectionable). I think it’s valuable important to look at a number of different metrics (number of animals affected, amount of time affected) to assess how promising an animal welfare intervention looks.
Thanks for sharing this piece.
From your review, did you get a sense of how many ABFT are farmed at the moment (in terms of number of animals), or how big the industry could get if successful?
Looking very quickly at fishcount’s 2017 data, it would suggest 7.4k metric tonnes, but they don’t provide a mean slaughter weight. Based on the weight of a mature adult provided above, it might suggest only around 30k were slaughtered in 2017. That’s seems pretty small for a farmed finfish species.
All else equal, I’d be surprised that growth since then might make it look like a top priority farmed fish species from a purely scale perspective. That said its possible the FAO data might not accurately reflect extent of farming, that hatchery mortality rates could be several times higher than for other farmed fish species, and growth could accelerate if there are breakthroughs in farming methods.Assuming scale in terms of number farmed is relatively small, do you think there would be a case for prioritising allocating resources to improving the welfare this species?
I can see the case that the high-value (both per kg and per animal) of ABFT means that producers might have the financial means to experiment with and implement welfare improvements in a way that might not be true for other fish. I can also see public campaigns about ABFT farming gathering more support than campaigns about other farmed fish species.
Very interesting. I’m of Indian origin and was born in the UK. But my parents were born and grew up in East Africa (Uganda and Kenya). As vegetarians, we ate a lot of beans/pulses. It was the norm in our house to soak beans overnight, and also to soak rice prior to cooking.
Based on my own experience, I’d always assumed that it was common to soak beans in both India and Africa. So this post is an update for me. That said, I still believe with around 60% confidence that soaking is the norm in India (most Indian recipes I’ve come across suggest soaking).
I think the edits you made to the summary work very well in making it clear what the quantitative analysis does and doesn’t cover! Thank you for taking on board my comments so promptly.
Fully agree with your points on the difficulty of quantifying the indirect benefits, and also how/where those benefits should be attributed.
I think the challenge is that excluding indirect benefits from a quantitative analysis effectively assigns them a zero value. That is ok when indirect benefits are most likely only a small fraction of the direct benefits. But it becomes problematic when the indirect benefits could plausibly be several times (or orders of magnitude) larger than the direct ones and relevant to decision-making.
If judgements about the size of the indirect benefits might be important, think it is valuable to make the exclusions clear—as you’ve now done!Thanks again for the time you put into the piece, and the clear write-up / reasoning transparency!
Thanks so much for this post. Very impressive how quickly you put this together.
I think your analysis is a very helpful take on how cost-effective it might be for advocates to purchase shrimp stunners for industry, and how this might compare to estimates of the cost-effectiveness of historical corporate hen welfare campaigns. Very useful to try and make different interventions comparable for interested stakeholders.
That said, I think your quantitive analysis probably misses most of the expected value of the SWP’s shrimp stunning intervention. My guess is that the vast majority of the expected value of this opportunity lies in how much it might bring forward widespread adoption of shrimp stunning by industry, and not the easier-to-quantify short-term direct impacts of the stunners SWP purchases for industry.
I think you recognise this issue in the last paragraph. And I also fully appreciate that even a relatively shallow attempt at quantifying wide-spread industry adoption scenarios was probably beyond the amount of time you could dedicate to this post.
That said, I’m not sure this limitation comes through fully in your summary, even after you the updates you added to the top. My biggest concern is that your report might put off potential donors who might assume your assessment more comprehensively covers benefits than it does, and might not have time to fully engage with your methodology or reach the final paragraph.
If you agree with my assessment, share my concerns, and were open to making changes while fundraising is ongoing, my suggestions would be to be more explicit about the scope limitations of your analysis and how most of the expected value might not be quantified (either by amending the summary or adding an additional update at the top). When describing the factors that might drive donor decisions, I’d personally also add optimism/pessimism about the potential for this project to bring forward widespread adoption by industry, which I see as independent of the factors you’ve already listed.
If you don’t agree with my assessment about where expected value might come from (e.g. because you think donors purchasing stunners for industry might set a dangerous precedent that might in fact delay adoption), or just happen to be deeply uncertain, I think it would be great for you to articulate this more explicitly.
Hope you don’t mind me making these suggestions. Really great work again—thank you for putting the time and effort into it.
I’ve been vegetarian since birth, and a vegan since 2007, and am based in the UK.
I take the Vegan Society Veg1 supplement daily (my kids take half a tablet), and also take an omega 3 (EPA+DPA) supplement. I use the lucky iron fish when cooking to improve iron content of food.
I was on the Board of the Vegan Society when the Veg1 was reformulated. I can vouch for the evidence base being taken very seriously during reformulation, led by Stephen Walsh, phD. There was careful consideration of balancing risk of deficiency against risks from supplementing nutrients many vegans would otherwise get, as well as practicality and affordability. I personally wouldn’t trust any multivitamin aimed at veg*ns containing antioxidants (vitamin E, A) given the possible risk of increasing mortality.
Many vegans don’t get enough calcium to avoid risk of fracture, and the Veg1 doesn’t include it, mainly because it would make the tablets too large to be practical. I consume enough fortified plant-milks, calcium set tofu, and bread to not worry too much about calcium.
The Veg1 supplement generally isn’t suitable for people in the US as it contains iodine (important for veg*ns in the UK), and there is risk of harm of excess iodine intake given salt is iodized in the US.
I think the evidence reviews of veganhealth.org are generally of high quality. Even though I think recommended intakes for some nutrients are higher than might be justified than the literature, I’m enormously grateful for Jack Norris (who runs veganhealth.org) for his work developing B12 recommendations with Stephen Walsh.
Thanks for the helpful comment @Ula Zarosa . I’ve now added a shorter non-technical summary.
I don’t think I would agree with that as a general explanation of the results.… in order for us to get the results we did, you’d need some people who were selecting meat when there two veg options to select meat-free when there are three veg options. The folks who always selected a meat-free dish (around 20% of respondents) don’t drive variation in meat-free meal selection across different menu types, and so can’t explain our results. Same applies to those who always selected meat-based dishes. Indeed the regression results on menu characteristics were identical when we excluded those respondents (see Table 10 of full-writeup).
But I think your quote above might be able to explain why the pp change in meat dishes chosen is lower than the change of menu options. Would probably make a few changes though (completeness at the expense of brevity).
“When offered more non-meat menu options (20pp increase), survey respondents selected fewer meat-based dishes (12pp decrease). Respondents, on average, selected a relatively low share of meat-based meals across the experiment. This is one reason why the fall in the share of meat-dishes is smaller than the change in the share of menu options.”
Thanks for the question @Jeff Kaufman .
The short answer is that our headline odds ratio (OR) on the number of meat-free meals was pretty close to one associated with your random choice example. But the absolute increase in meat-free meal selections arising from an extra veg option (around 12pp) was lower than in the random choice example (20pp). That inconsistency reflects the high number of meat-free meal selections made by participants in the study even when there were only two meat-free options.
DetailIn the random selection example you provided (meat-free selections increasing from 40% to 60%), the OR would be 2.25:
That’s just below our estimate for the OR (2.33) on the additional meat-free meal coefficient, but well within the 90% CI (1.81 to 2.99).
When interpreting results presented in odds ratio space, its important to bear in mind that odds ratios don’t translate linearly into pp changes—the starting odds/probability matters. An OR of 2.25 is associated with a 20pp increase for a baseline probability of 40%, but only a 10pp increase for a baseline probability of 80%.
The last column of Table 10 in the full write-up on the RP website provides the results of a linear regression model, which is easier to interpret in probability space. That model suggests 56% of meal selections were meat-free in a theoretical menu containing two (non-analogue) meat-free options, and three (red) meat-based dishes. Adding an additional (non-analogue) meat-free option would increase meat-free meal selection by 12pp (90% CI: 0.08 to 0.16). That’s quite a bit lower than the absolute increase in your random selection example (20pp).
So even though the results in odds ratio results were similar to one that might be expected under random selection, the absolute increase in number of meat-free meal selections was quite a bit lower. That inconsistency arises because participants in the study were opting for a high number of meat-free meals even when there were only two meat-free options.
Thank you for your comments, Matt!
I would agree on that this intervention would look better (in $/DALY space) if I were to have adopted the same assumptions as @Laura Duffy and come up with some plausible assumptions how much time in various pain intensities that would be averted through the intervention. I also think its very unlikely the intervention would look competitive the top AW and GHD interventions. Under the assumptions where this intervention were to look very competitive, I’d suspect shrimp stunning interventions would look even better.
Thanks also for your very valid comments on using DALYs as a unit to compare interventions (and your general engagement on the research that @Rethink Priorities does!).