We also find it implausible that bees have larger welfare ranges than salmon. But (a) we’re also worried about pro-vertebrate bias; (b) bees are reallyimpressive; (c) there’s a great deal of overlap in the plausible welfare ranges for these two types of animals, so we aren’t claiming that their welfare ranges are significantly different; and (d) we don’t know how to adjust the scores in a non-arbitrary way. So, we’ve let the result stand. (We’d make similar points in response to: “I can’t believe that octopuses beat carp!”)
(I could believe octopuses beat carps, because octopuses seem unusually cognitively sophisticated among animals.)
I’d guess the main explanation for this (at least sentience-adjusted, if that’s what’s meant here), which may have biased your results against salmons and carps, is that you used the prior probability for crab sentience (43% mean, 31% median from table 3 in the doc) as the prior probability for salmon and carp sentience, and your posterior probabilities of sentience are generally very similar to the priors (compare tables 3 and 4 in the doc). Honeybees, fruit flies, crabs, crayfish, salmons and carps all ended up with similar sentience probabilities, but I’d assign higher probabilities to salmons and carps than to the others. You estimated octopuses to be about 2x as likely to be sentient as salmons and carps, according to both your priors and posteriors, with means and medians roughly between 73% and 78% for octopuses. On the other hand, your sentience-conditioned welfare ranges didn’t differ too much between the fish, octopuses and bees. It’s worth pointing out that Luke Muehlhauser had signficantly higher probabilities for rainbow trouts (70%, in the salmonid family like salmons) than Gazami crabs (20%) and fruit flies (25%), and you could use his prior for rainbow trouts for salmons and carps instead (or something in between). That being said, his probabilities were generated in different ways from yours, so that might introduce other biases. You could instead use your prior for octopuses (or something in between). Or, most consistent with your methodology, would be to have the authors of the original estimates for RP just estimate these probabilities directly, with or without the data you gathered for salmons and carps. Any of these would be relatively small fixes.
As an aside, should we interpret this sentience probability work as not primarily refining your old estimates (since the posteriors and priors are very similar), but as adding other species and further modelling your uncertainty?
There may be some other smaller potential sources of bias that contributed here, but I don’t expect them to have been that important:
I’m guessing salmon and carp (and apparently zebrafish, which seem to often have been used when direct evidence wasn’t available, maybe more for carp) are less well-studied than bees, so your conservative assumptions of assigning 0 to “unknown” for both probabilities of sentience and welfare ranges conditional on sentience may count more against them. For example, there were some studies found for “cognitive sophistication” for honeybees but not for salmons or carps, and more found for “mood state behaviors” for honeybees than salmons and carps in your new Sentience Table. For your Welfare Range Table, bees had fewer “unknowns” for cognitive proxies than salmons and carps, but more for hedonic proxies and representational proxies.
One possible quick-ish fix would be to use a prior for the presence/absence of proxies across animals based on the ones for which there are studies (possibly just those you collected evidence for), although this may worsen other biases, like publication bias, and it requires you to decide how to weigh different animal species (but uniformly across those you collected evidence for is one way, although somewhat arbitrary).
Another quick-ish fix could be to make more assumptions between species you gathered evidence for, e.g. if a fruit fly has some capacity, I’d expect fish to, as well, and if some mammal is missing some capacity, I’d expect salmon and carp to not have it either. This may be too strong, but you did use the crab sentience prior for the fish.
Longer fixes could use more sophisticated missing data methods.
Also, among the proxies you’ve used, I’d be inclined to give almost all of my weight to a handful of hedonic proxies, namely panic-like behavior, hyperalgesia, PTSD-like behavior, prioritizes pain response in relevant context and motivational trade-off (a cognitive proxy) as indicating the extremes of welfare conditional on sentience, and roughly in that order by weight. The first three all came up “unknown” due to no studies for bees, but there were a few studies suggesting their presence (and none negative) for the fish. Giving almost all of your weight to these proxies would favor the fish over bees. That being said, I wouldn’t be that surprised to find out that bees display those behaviors, too, because I also think bees are very impressive and behaviorally complex.
I might use joy-like behavior and play behavior for the other end of the welfare range, but I expect them to be overshadowed by the intense suffering indicators above, and I don’t expect them to differ too much across the species. There was evidence of play behavior in all three, but only evidence for joy-like behavior in carps.
The next proxies that could make much difference that I think could matter on some models (although I don’t assign them much weight) would be neuron counts and the number of just-noticeable differences, and neuron counts would also favor the fish.
Thanks for all this, Michael. Lots to say here, but I think the key point is that we don’t place much weight on these particular numbers and, as you well know and have capably demonstrated, we could get different numbers (and ordinal rankings) with various small changes to the methodology. The main point to keep in mind (which I say not for your sake, but for others, as I know you realize this) is that we’d probably get even smaller differences between welfare ranges with many of those changes. One of the main reasons we get large differences between humans and many invertebrates is because of the sheer number of proxies and the focus on cognitive proxies. There’s an argument to be given for that move, but it doesn’t matter here. The point is just that if we were to focus on the hedonic proxies you mention, there would be smaller differences—and it would be more plausible that those would be narrowed further by further research.
If I had more time, I would love to build even more models to aggregate various sets of proxies. But only so many hours in the day!
(I could believe octopuses beat carps, because octopuses seem unusually cognitively sophisticated among animals.)
I’d guess the main explanation for this (at least sentience-adjusted, if that’s what’s meant here), which may have biased your results against salmons and carps, is that you used the prior probability for crab sentience (43% mean, 31% median from table 3 in the doc) as the prior probability for salmon and carp sentience, and your posterior probabilities of sentience are generally very similar to the priors (compare tables 3 and 4 in the doc). Honeybees, fruit flies, crabs, crayfish, salmons and carps all ended up with similar sentience probabilities, but I’d assign higher probabilities to salmons and carps than to the others. You estimated octopuses to be about 2x as likely to be sentient as salmons and carps, according to both your priors and posteriors, with means and medians roughly between 73% and 78% for octopuses. On the other hand, your sentience-conditioned welfare ranges didn’t differ too much between the fish, octopuses and bees. It’s worth pointing out that Luke Muehlhauser had signficantly higher probabilities for rainbow trouts (70%, in the salmonid family like salmons) than Gazami crabs (20%) and fruit flies (25%), and you could use his prior for rainbow trouts for salmons and carps instead (or something in between). That being said, his probabilities were generated in different ways from yours, so that might introduce other biases. You could instead use your prior for octopuses (or something in between). Or, most consistent with your methodology, would be to have the authors of the original estimates for RP just estimate these probabilities directly, with or without the data you gathered for salmons and carps. Any of these would be relatively small fixes.
As an aside, should we interpret this sentience probability work as not primarily refining your old estimates (since the posteriors and priors are very similar), but as adding other species and further modelling your uncertainty?
There may be some other smaller potential sources of bias that contributed here, but I don’t expect them to have been that important:
I’m guessing salmon and carp (and apparently zebrafish, which seem to often have been used when direct evidence wasn’t available, maybe more for carp) are less well-studied than bees, so your conservative assumptions of assigning 0 to “unknown” for both probabilities of sentience and welfare ranges conditional on sentience may count more against them. For example, there were some studies found for “cognitive sophistication” for honeybees but not for salmons or carps, and more found for “mood state behaviors” for honeybees than salmons and carps in your new Sentience Table. For your Welfare Range Table, bees had fewer “unknowns” for cognitive proxies than salmons and carps, but more for hedonic proxies and representational proxies.
One possible quick-ish fix would be to use a prior for the presence/absence of proxies across animals based on the ones for which there are studies (possibly just those you collected evidence for), although this may worsen other biases, like publication bias, and it requires you to decide how to weigh different animal species (but uniformly across those you collected evidence for is one way, although somewhat arbitrary).
Another quick-ish fix could be to make more assumptions between species you gathered evidence for, e.g. if a fruit fly has some capacity, I’d expect fish to, as well, and if some mammal is missing some capacity, I’d expect salmon and carp to not have it either. This may be too strong, but you did use the crab sentience prior for the fish.
Longer fixes could use more sophisticated missing data methods.
You may have underestimated salmon and carp neuron counts around 100x.
Also, among the proxies you’ve used, I’d be inclined to give almost all of my weight to a handful of hedonic proxies, namely panic-like behavior, hyperalgesia, PTSD-like behavior, prioritizes pain response in relevant context and motivational trade-off (a cognitive proxy) as indicating the extremes of welfare conditional on sentience, and roughly in that order by weight. The first three all came up “unknown” due to no studies for bees, but there were a few studies suggesting their presence (and none negative) for the fish. Giving almost all of your weight to these proxies would favor the fish over bees. That being said, I wouldn’t be that surprised to find out that bees display those behaviors, too, because I also think bees are very impressive and behaviorally complex.
I might use joy-like behavior and play behavior for the other end of the welfare range, but I expect them to be overshadowed by the intense suffering indicators above, and I don’t expect them to differ too much across the species. There was evidence of play behavior in all three, but only evidence for joy-like behavior in carps.
The next proxies that could make much difference that I think could matter on some models (although I don’t assign them much weight) would be neuron counts and the number of just-noticeable differences, and neuron counts would also favor the fish.
Thanks for all this, Michael. Lots to say here, but I think the key point is that we don’t place much weight on these particular numbers and, as you well know and have capably demonstrated, we could get different numbers (and ordinal rankings) with various small changes to the methodology. The main point to keep in mind (which I say not for your sake, but for others, as I know you realize this) is that we’d probably get even smaller differences between welfare ranges with many of those changes. One of the main reasons we get large differences between humans and many invertebrates is because of the sheer number of proxies and the focus on cognitive proxies. There’s an argument to be given for that move, but it doesn’t matter here. The point is just that if we were to focus on the hedonic proxies you mention, there would be smaller differences—and it would be more plausible that those would be narrowed further by further research.
If I had more time, I would love to build even more models to aggregate various sets of proxies. But only so many hours in the day!