Hi Vasco, Thanks for the good question! I think it’s important to note that there are (at least) 3 types of model choices and uncertainty at work: a) we have a good deal of uncertainty about each theory of welfare represented in the model, b) we don’t have a ton of confidence that the function we included to represent each theory of welfare is accurate (especially the undiluted experiences function, which partially drives the high mean results), a) we could have uncertainty that our approach to estimating welfare ranges in general is correct, but we’ve not included this overall model uncertainty. For instance, our model has no “prior” welfare ranges for each species, so the distribution output by the calculation entirely determines our judgement of the welfare range of the species involved. We also might be uncertain that simply taking a weighted mixture of each theory of welfare is a good way to arrive at an overall judgement of welfare ranges. Etc.
Our preliminary method used in this project incorporates model uncertainty in the form of (a) by mixing together the separate distributions generated by each theory of welfare, but we don’t incorporate model uncertainty in the ways specified by (b) or (c). I think these additional layers of uncertainty are epistemically important, and incorporating them would likely serve to “dampen” the effect that the mean result of the model affects our all-things-considered judgement about the welfare capacity of any species. Using the median is a quick (though not super rigorous or principled) of encoding that conservatism/additional uncertainty into how you apply the moral weight project’s results in real life. But there are other ways to aggregate the estimates, which could (and likely would) be better than using the median.
I think these additional layers of uncertainty are epistemically important, and incorporating them would likely serve to “dampen” the effect that the mean result of the model affects our all-things-considered judgement about the welfare capacity of any species.
I tend to agree.
But there are other ways to aggregate the estimates, which could (and likely would) be better than using the median.
I wondered whether it would be better for you to aggregate the results from the different models with the geometric mean of odds. For example, if models 1 and 2 implied a probability of 50 % and 90 % of the welfare range being smaller than 0.2, corresponding to odds of 1 (= 0.5/(1 − 0.5)) and 9 (= 0.9/(1 − 0.9)), the aggregated model would imply odds of 3 (= (1*9)^0.5) of the welfare range being smaller than 0.2, corresponding to a probability of 75 % (= 1/(1 + 1⁄3)). There is some evidence for using the geometric mean of odds, so I believe an approach like this combined with using the means of the aggregated distributions would be better than your approach of using the medians of the final distributions at the end.
Hi Vasco,
Thanks for the good question! I think it’s important to note that there are (at least) 3 types of model choices and uncertainty at work:
a) we have a good deal of uncertainty about each theory of welfare represented in the model,
b) we don’t have a ton of confidence that the function we included to represent each theory of welfare is accurate (especially the undiluted experiences function, which partially drives the high mean results),
a) we could have uncertainty that our approach to estimating welfare ranges in general is correct, but we’ve not included this overall model uncertainty. For instance, our model has no “prior” welfare ranges for each species, so the distribution output by the calculation entirely determines our judgement of the welfare range of the species involved. We also might be uncertain that simply taking a weighted mixture of each theory of welfare is a good way to arrive at an overall judgement of welfare ranges. Etc.
Our preliminary method used in this project incorporates model uncertainty in the form of (a) by mixing together the separate distributions generated by each theory of welfare, but we don’t incorporate model uncertainty in the ways specified by (b) or (c). I think these additional layers of uncertainty are epistemically important, and incorporating them would likely serve to “dampen” the effect that the mean result of the model affects our all-things-considered judgement about the welfare capacity of any species. Using the median is a quick (though not super rigorous or principled) of encoding that conservatism/additional uncertainty into how you apply the moral weight project’s results in real life. But there are other ways to aggregate the estimates, which could (and likely would) be better than using the median.
Thanks for the good reply too, Laura.
I tend to agree.
I wondered whether it would be better for you to aggregate the results from the different models with the geometric mean of odds. For example, if models 1 and 2 implied a probability of 50 % and 90 % of the welfare range being smaller than 0.2, corresponding to odds of 1 (= 0.5/(1 − 0.5)) and 9 (= 0.9/(1 − 0.9)), the aggregated model would imply odds of 3 (= (1*9)^0.5) of the welfare range being smaller than 0.2, corresponding to a probability of 75 % (= 1/(1 + 1⁄3)). There is some evidence for using the geometric mean of odds, so I believe an approach like this combined with using the means of the aggregated distributions would be better than your approach of using the medians of the final distributions at the end.