I am a Senior Research Manager in the Animal Welfare department at Rethink Priorities. The views I express here do not represent Rethink Priorities unless stated otherwise.
Before working in effective altruism, I completed a Ph.D. in psychology studying the evolution of cooperation in humans, with a concentration in quantitative psychology. After that, I was a postdoctoral fellow studying public health. My main interests now are animal welfare and social science methodology/statistics.
Hey Vasco,
In our code, we estimate that shrimp live on ongrowing farms (this analysis doesn’t look at earlier stages of production) for about 115 days*24 hours = 2,760 hours. However, due to preslaughter mortality and variation in when farmers choose to harvest, the 90% interval is [~14 days, ~175 days].
The total raw number of hours in pain exceeds the lifespan of the average shrimp because we examined each welfare threat in isolation, whereas in practice many are occurring concurrently. This is an issue Welfare Footprint has considered but for simplicity we do not address it here. We do note, “We assume purely additive relationships between welfare threats. This means that some estimates of time spent in pain can exceed the lifespan of a shrimp.” But I am glad for the opportunity to mention this again, as it is also an example of why we caution against treating our headline results as representative of the life of an actual shrimp.
For more context, see also Box 2 and Figure 1 of Pre-slaughter mortality of farmed shrimp.
Reducing the influence of impression management on the measurement of prosocial and antisocial traits was the topic of my doctoral research. When I started, I thought that better behavioral paradigms and greater use of open-ended text analysis could meaningfully move the needle. By the time I moved onto other things I was much more pessimistic that there is low-hanging fruit that can both (a) meaningfully move the needle (here’s one example of a failed attempt of mine to improve the measurement of prosocial traits; McAuliffe et al., 2020), and (b) be implemented at scale in a practical context. The general issue is that harder-to-game measures are much noisier than easier-to-game measures (e.g., see Schimmack, 2021 on implicit measures), so the gameable measures tend to be more useful for making individual predictions in spite of their systematic biases. The level of invasiveness required to increase the signal on a non-gameable measure (e.g., scraping all of a person’s online text without their permission) would probably be at odds with other goals of the movement. The same probably goes for measures that do not rely on actual evidence of concerning behavior (e.g., polygenic scores).
More fundamentally, I disagree that this is a neglected topic– measuring malevolence and reducing responses biases are both mainstream topics within personality psychology, personnel psychology, developmental psychology, behavioral genetics, etc. For example, considerable effort has gone into testing whether multidimensional forced-choice personality questionnaires do a good job reducing faking (e.g., Wetzel et al., 2020). An academic psychologist who is EA-sympathetic and getting funding from standard academic sources might have more impact from pursuing this topic rather than whatever else they would have studied instead, but I see limited value in people changing careers or funding grants that would have otherwise gone to other EA causes. I also do not see a strong case for carrying on the discussion outside of the normal academic outlets where there is a lot more measurement expertise.
I personally would not make mortality the focus of the marginal research project, but I do think you would get it ‘for free’ in the sort of project I would prioritize. In my view, the main considerations are:
1. A lot of uncertainty is an artifact of inconsistent reporting practices. An article arguing for a standardized methodology in an aquaculture magazine signed by a bunch of prestigious researchers (or a presentation at an aquaculture industry event) might do more to reduce uncertainty than more data per se.
2. A lot of the basic trends are robust to the uncertainty. Cumulative mortality is probably around ~50% even in ideal circumstances, more intensive farms have less mortality, larval mortality is steeper than juvenile mortality, and wild shrimp have higher mortality rates than farmed shrimp.
3. Hannah’s upcoming report, a Monte Carlo model of which welfare issues cause the most harm in aggregate while shrimp are still alive, contains enormous uncertainty due to limitations in the surveys of farms that have been conducted. As a result, the rank-order of the badness of many issues is not robust, an issue that new, higher-quality data could address. Improved surveys would presumably also measure survival, so we would gain clarity on premature mortality even though it was not the main focus.
4. It would probably be at least as valuable to get larval mortality estimates for the farmed fish species to which we compared farmed shrimp in Figure 4.
Our next report explores preslaughter mortality in much greater depth, including cobbling together estimates from a wider variety of sources, some of which are broken down by extensive vs. intensive. We expect to publish it very soon, stay tuned!
Interesting idea! I will have to look into whether it has been tried on farmed animals or laboratory animals. I would have a concern similar to the concern I have with the classical conditioning experiments: aversion to the more intense pain might reflect reduced volition rather than welfare maximization. But it does seem plausible that volition is not as much of an issue when the pain is only administered with a low probability.
I am not familiar with the authors you cite so I will refrain from commenting on their specific proposals until I have read them. I speculate that my comment below is not particularly sensitive to their views; I am a realist about morality and phenomenal consciousness but nevertheless believe that what you are suggesting is a constructive way forward.
So long as it is transparent, I definitely think it would be reasonable to assign relative numerical weights to Welfare Footprint’s categories according to how much you yourself value preventing them. The weights you use might be entirely based on moral commitments, or might partly be based on empirical beliefs about their relative cardinal intensities (if you believe they exist), or even animals’ preferences (if you believe the cardinal intensities do not exist or believe that preferences are what really matter). Unless one assigns lexical priority to the most severe categories, we have to make a prioritization decision somehow, and assigning weights at least makes the process legible.
Joel Michell argues that the theory of conjoint measurement provides indirect tests for whether psychological constructs are quantitative. I do not yet understand the approach in much detail or the arguments for alternative approaches.
I like your summary. I feel (slightly) less hopeless because I think…
Comparisons that involve multiple dimensions of pain are, in principle, possible. I think I would only regard them as impossible if I came upon evidence that pain severity is, in reality, an ordinal construct.
In one sense, I might be more pessimistic about this topic than many because I think it is plausible that many psychological constructs are ordinal.
Behavioral evidence could in theory license cardinal comparisons among different pains. Practical issues of feasibility (and permission from institutions) stand in the way, and I would grant that these will probably never be overcome.
Possibly, cardinal differences in severity are explicitly represented in the brain. If so, then in principle we could measure these representations, though I do not think that we ever will.
We may be able to prioritize between relieving severe pain and long-lasting pain without making direct cardinal comparisons, so long as we have a sense of just how many orders of magnitude pain severity can span. Many aspects of pain experience appear conserved across a large number of species. If we find that pain in humans or laboratory animals have a wide range of pain severity, then there is an above-chance possibility that farmed animals do too. There is also an above-chance possibility that the most severe pains on factory farms are close to the end of the negative side of the range, given that it is difficult to see the adaptive value of being able to represent threats more extreme than, say, being boiled alive.
I would agree that the point above is partly grounded in intuition that has only a vague relationship to a well-established theory of the evolution of pain. Hopefully, advances in this area will reduce our reliance on intuitions that are not grounded by a plausible scientific theory.
Strongly agreed. For those who want exposition on this point, see Ashford’s article on demandingness in contractualism vs. utilitarianism https://doi.org/10.1086/342853
FiveThirtyEight podcast scrutinizes this poll (segment starts 30 minutes in)
https://fivethirtyeight.com/features/politics-podcast-the-politics-of-ai/
Disappointingly shallow analysis, though.
This survey item may represent a circumstance under which YouGov estimates would be biased upwards. My understanding is that YouGov uses quota samples of respondents who have opted-in to panel membership through non-random means, such as responses to advertisements and referrals. They do not have access to respondents without internet access, and those who do but are not internet-savvy are also less likely to opt in. If internet savviness is correlated with item response, then we should expect a bias in the point estimate. I would speculate that internet savviness is positively correlated with worrying about AI risk because they understand the issue better (though I could imagine arguments in the opposite direction—e.g., people who are afraid of computers don’t use them).
To give a concrete example, Sturgis and Kuha (2022) report that YouGov’s estimate of problem gambling in the U.K. was far higher than estimates from firms that used probability sampling that can reach people who don’t use the internet, especially when the interviews were conducted in person. The presumed reasons are that online gambling is more addictive and that people at higher risk of problem gambling prefer online gambling to in-person gambling.
Thanks for this passionate post. Because of it I also donated to the White Helmets. For other readers I would suggest also considering Doctors Without Borders, but in general I have a lot of uncertainty about how best to help.
That was beautifully put, Saulius.
Thanks for your work on this, Tessa! I have some similar follow-up questions:
”Thanks for your support and comment. Unfortunately, it appears as though the environmental permitting regarding this specific farm is being allowed to proceed.”
To clarify, do you mean that Nueva Pescanova has in fact received its environmental permit?
”How likely do you think it is that the farm will succeed in creating a commercially viable product, apart from public pressure? Sounds like there are significant biological and ecological barriers.”
I am also interested in ALI’s take on this. Nueva Pescanova claims it will be able to raise 3k tonnes of farmed octopus starting in 2023. Has ALI been able to verify that this scale is actually feasible right now?
Finally, is an outright, blanket ban on octopus legally possible in Spain or the EU (or even narrowly within the Canary Islands)? Or is a “ban” shorthand for “convince legislators that, in practice, octopus farming won’t meet existing minimal environmental and animal welfare standards”? And what existing farmed animal welfare standards could be invoked, given that octopuses are invertebrates, not vertebrates?
Agreed re: other measures of well-being.
I think the standard approach here would be a two-way fixed effects models with whatever time-varying covariates you can get access to. It makes strong assumptions though:
https://doi.org/10.1017/pan.2020.33
The cutting edge here is probably the general cross-lagged panel model, which in the tutorial below could not distinguish the long-run effect of national income on national well-being from zero.
I do not know of such evidence. I would note the Gallup world data suggesting that the very down and out are the exception to the “most people are happy most of the time” generalization.
“We can’t decisively rule out scale shifts. Obviously, it’s in the nature of something subjective that we can’t objectively check it. As with assessing validity, we have to reason about what’s likely—the fact something is possible doesn’t mean it’s likely.”
There is an objective way to test for scale shifts across countries and over time: Measurement invariance testing (or testing for “differential item functioning,” to use the item response theory vocabulary). Though I agree “we have to reason about what’s likely,” as the test makes strong statistical and substantive assumptions.
There is evidence that subjective well-being measures violate invariance across countries. I do not know of studies examining invariance of well-being across generations off the top of my head, but there is evidence that scale-shifts do occur across generations for other traits, such as narcissism.
“The other part of the explanation appeals to hedonic adaptation and the fact that we do get used to lots of things”
Without taking a stance on the broader thesis of this report, I think the evidence of hedonic adaptation is easy to overstate. Latent state-trait models show that changes in circumstances have detectable changes on well-being at least 10 years later, especially for affective measures. Winning the lottery also has long-term effects of well-being (though more so on life satisfaction), contra the Brickman study that played a role in popularizing hedonic adaptation.
More supportive of your thesis is evidence that the relationship between wealth and happiness is mostly driven by stable factors .
”Also in this context, the research by my own organisation, the Happier Lives Institute, finds that cash transfers to the very poor — those on the global poverty line — actually do have a small but significant effect on subjective wellbeing, one that continues over several years (McGuire, Kaiser, Bach-Mortensen, 2022).”
I would suggest also citing the evidence that this result may be an artifact of publication bias.
“More generally, I think what underlies these ideas of using lower salaries as a costly signal of value-alignment is the tacit assumption that value-alignment is a relatively cohesive, unidimensional trait. But I think that assumption isn’t quite right—as stated, our factor analyses rather suggested there are two core psychological traits defining positive inclinations to effective altruism (expansive altruism and effectiveness-focus), which aren’t that strongly related. (And I wouldn’t be surprised if we found further sub-facets if we did more extensive research on this.)”
I agree with the last sentence of this—there are probably at least as many sub-facets as there are distinct tenets of effective altruism, and only most or all of them coming together in the same person is sufficient for making someone aligned. Two facets is too few, and, echoing David, I do not think that the effectiveness-focus and expansive altruism measures are valid measures of actual psychological constructs (though these constructs may nevertheless exist). My view is that these measures should only be used for prediction, or reconstructed from scratch.
I am less sure the final part of the following:
“I think it’s better for EA recruiters to try to gauge, e.g. inclinations towards cause-neutrality, willingness to overcome motivated reasoning, and other important effective altruist traits, directly, rather than to try to infer them via their willingness to accept a lower salary—since those inferences will typically not have a high degree of accuracy.”
This depends, I think, on how difficult it is to ape effective altruism. As effective altruism becomes more popular and more materials are available to figure out the sorts of things walking-talking EAs say and think, I would speculate that aping effective altruism becomes easier. In this case, if you care about selecting for alignment, a willingness to take on a lower salary could be an important independent source of complimentary evidence.
“In an intermediate step, we removed items that were not predictive of relevant EA outcome measures (these outcome measures include overall agreement with the key principles of EA, interest in learning more about EA, willingness to change one’s career in order to have more impact).”
“Notably, we found that some items with an obligation framing (e.g., “it would be wrong …”) yielded high variance and were particularly predictive. Remember that our goal was not to create items that together define effective altruism conceptually or philosophically, but rather to identify items that are predictive of our outcome measures. Thus, our inclusion of these obligation items does not mean that we think that effective altruism should be defined in terms of an obligation to help effectively.”
I think prioritizing items with predictive power is questionable from a psychometric perspective. Predictive power should not be the goal of measurement (McIntosh, 2007), and latent indicators should not be preferentially chosen because they optimize prediction (Smits et al., 2018). A valid indicator of a construct does not necessarily have good predictive power and invalid indicators often do. Even though you are not conceptually defining Effective Altruism, the indicators should conceptually match your theoretical beliefs about Expansive Altruism and Effectiveness-Focus. If feelings of obligation are constitutive of those constructs, great, but if not then you might want to toss them even if they have predictive value. (Though having a single item with an obligation focus would not be a bad idea, as the variance attributable to the obligation framing would be partialed out anyway when you load the item on the factor.)
This procedure risks attributing more predictive power to Expansive Altruism and Effectiveness-Focus than would be found with optimal indicators for these constructs. Accordingly, measurement validity can decrease, and the latent variables may be endogenous with respect to the outcome variables that you optimized the factor to predict. One potential source of evidence for this would be implausibly large associations between the factors and the outcomes. Other sources of evidence would include the Effectiveness and Expansiveness factors correlating with the disturbances of the outcome variables, and the Effectiveness and Expansiveness factors’ disturbances correlating with the common variance of the outcome measures. Using Study 2 data, I found evidence consistent with each of these possibilities by regressing a latent variable composed of the charitable giving tasks on the Expansive and Effectiveness factors (model and modification indices), as well as regressing a latent variable composed of the EA-interest items on the Expansive and Effectiveness factors (model and modification indices). (I also wrote up a more expository walk-through of these analyses here.)
A stronger way of testing whether predictive power has been overstated would be to examine the predictive validity of measures of similar constructs. The principle of care scale, for instance, could stand in for the Expansive Altruism scale.
If I were Reviewer 2, I would probably also nitpick on these much more minor issues:
“α = 0.80, CFI = .97.”
I would recommend reporting McDonald’s omega instead of Cronbach’s alpha unless you provide evidence that the factor loadings are all equal or nearly so. Even more emphatically I would recommend reporting the chi-square test instead of or in addition to CFI. No approximate fit index can (or ever could) measure how much discrepancy there is between the fitted model and true model (Greiff & Heene, 2017); local fit indices and careful judgment are the only real guides, though fallible in their own ways. Note also that the models I linked to above find much worse fit by all metrics, probably because I embedded the factors in a larger model. Testing the fit of a factor in isolation of the other variables is a weak test because in practice those factors will be entered into models with other variables (Hayduk & Glaser, 2000; e.g., the Expansive Altruism and Effectiveness-Focus factors will often be entered into the same model in future studies).
“We believe that expansive altruism is a particular type of altruism. In contrast to other forms of altruism, expansive altruism is driven by a desire to not just help proximate and emotionally salient individuals, but also distant individuals that may be less emotionally salient. A similar type of distinction is made by psychologist Paul Bloom in his book Against Empathy (2017), in which he contrasts empathy-driven altruism with, what he refers to as, “rational compassion”. Our studies support the notion that such a form of rational compassion exists. However, our studies don’t suggest that this sort of compassion entails a lack of empathy. To the contrary, we find that expansive altruism strongly correlates with empathy (i.e., the trait “Empathic Concern”; see below).”
Another good source that I sense is sometimes overlooked is Hoffman. There is also closely related older work on the universal care ethic, such as in The Altruistic Personality. One thing I am currently unclear about with the Expansive Altruism scale is whether it measures the belief that one ought to care about distant others (which might or might not cause a desire to help) or an actual desire to help (which may or may not be caused by a belief that one ought to help).
“And notably effectiveness-focus and instrumental harm are also psychologically different: they correlated only weakly (r = .34; see below).”
“Only weakly” is probably too strong. This correlation is large for individual differences research (Gignac et al., 2016).
In case it’s useful, Adam Shriver and I ran a workshop about this issue with some pain scientists and animal welfare scientists, and reported some of our findings here: https://rethinkpriorities.org/publications/dimensions-of-pain-workshop-summary-and-updated-conclusions. Welfare Footprint also wrote about it recently: https://welfarefootprint.org/2024/02/20/shortagony-or-longache/. Both reports cover some of the relevant survey data.
Also, I have found it useful to directly incorporate uncertainty about the appropriate severity weights directly into welfare footprint-style models, as we recently did for shrimp aquaculture welfare threats: https://rethinkpriorities.org/publications/quantifying-and-prioritizing-shrimp-welfare-threats