I’m not sure I understand how it works, but isn’t one of the most important parameters how someone would want to trade 1 tonne of CO2 for 1 h of suffering on a factory farm? I.e. I could imagine that ratio could vary by orders of magnitude, and could make either the suffering or the carbon effects dominate.
It seems like your current approach is to normalize both scales and then add them. This will be implicitly making some tradeoff between the two units, but that tradeoff is hidden from the user, which seems like a problem if it’s going to be one of the main things driving the results.
Moreover, (apologies if I’ve misunderstood) but as far as I can see, the way the tradeoff is made is effectively that whichever animal is worst is set to 100 on each dimension. This doesn’t seem likely to give the right results to me.
For instance:
Perhaps I think beef = 10 CO2, and chicken = 1 CO2
Beef = 1 unit suffering, chicken = 100 units of suffering
In your process, I would normalize both scales so the worst is ’100 points’, so I’d need to increase beef to 100 and chicken to 10 on the CO2 scale.
If I weight each at 50%, I end up with overall harm scores of:
Beef = 100 + 1 = 101
Chicken = 10 + 100 = 110
However, suppose my view is that 1 tonne of CO2 doesn’t result in much animal suffering, so I think 1 unit of suffering = 100 CO2.
(If instead I had a human-centric view that didn’t put much weight on reducing animal suffering, the picture would be reversed.)
I could try to fix the results for myself by changing the relative weighting, but given that I’m not given any units, it’s hard for me to know I’m doing this correctly.
Thank you for the thoughtful feedback, Benjamin! I will try to explain the model a bit more thouroughly than the methods section of the post.
Let’s forget normalising and weights for a moment. If we measure suffering in hours/kcal and emissions in CO2eq/kcal then the subscales have different units and can’t be added (unless we have a conversion formula from one unit to the other somehow). A common solution in this case is to multiply the subscale values. If we do this a 1% change in suffering changes the combined score by the same amount that a 1% change in emissions would.
We still might want to prioritise some subscales more than others. If we would have added subscale scores we could have multiplied the subscale scores by some constant weights beforehand. If instead we multiply subscale scores we would exponentiate the subscale scores by weights beforehand. This simple idea is called a weighted product model (WPM) in the multiple-criteria decision analysis discipline which studies how to make decisions when we have multiple conflicting criteria.
This tool uses a weighted product model. The unnormalised suffering and emissions scores are: 1. exponentiated by their corresponding weight, 2. multiplied together to get a combined score, 3. the combined score is normalised to the 0-100 range for cleaner display.
WPM is a dimensionless method used for ranking options when making decisions. That is, to answer questions like “is it more important to avoid chicken or beef” not “what is the cardinal utility of avoiding chicken”. This model is only useful for prioritising if I have decided to reduce meat consumption but am only able to leave one species off my plate. I understand now that I should have made it more clear.
Somehow measuring the utility of leaving a species off my plate would be much more interesting but seemed difficult considering the time and skills I had. I did consider using something like DALYs. There is research on converting emissions to DALYs which would allow us to use a parameter for converting non-human animal DALYs to human DALYs but I decided for the simpler ranking-only model.
That makes sense. The point I’m trying to make, though, is that the choice of how to do the conversion from CO2/kcal to hours/kcal is probably the most important bit that drives the results. I’d prefer to make that clearer to users, and get them to make their own assessment.
Instead, the WPM ends up coming up with an implicit conversion rate, which could be way different from what the person would say if asked. Given this, it seems like the results can’t be trusted.
(I expect a WPM would be fine in domains where there are multiple difficult-to-compare criteria and we’re not sure which criteria are most important – as in many daily decisions – but in this case, it could easily be that either CO2 or suffering should totally dominate your ranking, and it just depends on your worldview.)
You are right. I spent time thinking about your comments and I agree that making the tradeoff clearer is one of the most important improvements I can make. Thank you for bringing it out.
Cool idea!
I’m not sure I understand how it works, but isn’t one of the most important parameters how someone would want to trade 1 tonne of CO2 for 1 h of suffering on a factory farm? I.e. I could imagine that ratio could vary by orders of magnitude, and could make either the suffering or the carbon effects dominate.
It seems like your current approach is to normalize both scales and then add them. This will be implicitly making some tradeoff between the two units, but that tradeoff is hidden from the user, which seems like a problem if it’s going to be one of the main things driving the results.
Moreover, (apologies if I’ve misunderstood) but as far as I can see, the way the tradeoff is made is effectively that whichever animal is worst is set to 100 on each dimension. This doesn’t seem likely to give the right results to me.
For instance: Perhaps I think beef = 10 CO2, and chicken = 1 CO2 Beef = 1 unit suffering, chicken = 100 units of suffering
In your process, I would normalize both scales so the worst is ’100 points’, so I’d need to increase beef to 100 and chicken to 10 on the CO2 scale.
If I weight each at 50%, I end up with overall harm scores of: Beef = 100 + 1 = 101 Chicken = 10 + 100 = 110
However, suppose my view is that 1 tonne of CO2 doesn’t result in much animal suffering, so I think 1 unit of suffering = 100 CO2.
Then, my overall harm scores would be:
Beef = 10⁄100 + 1 = 1.1 Chicken = 1⁄100 + 100 = 100.1
So the picture is totally different.
(If instead I had a human-centric view that didn’t put much weight on reducing animal suffering, the picture would be reversed.)
I could try to fix the results for myself by changing the relative weighting, but given that I’m not given any units, it’s hard for me to know I’m doing this correctly.
Thank you for the thoughtful feedback, Benjamin! I will try to explain the model a bit more thouroughly than the methods section of the post.
Let’s forget normalising and weights for a moment. If we measure suffering in hours/kcal and emissions in CO2eq/kcal then the subscales have different units and can’t be added (unless we have a conversion formula from one unit to the other somehow). A common solution in this case is to multiply the subscale values. If we do this a 1% change in suffering changes the combined score by the same amount that a 1% change in emissions would.
We still might want to prioritise some subscales more than others. If we would have added subscale scores we could have multiplied the subscale scores by some constant weights beforehand. If instead we multiply subscale scores we would exponentiate the subscale scores by weights beforehand. This simple idea is called a weighted product model (WPM) in the multiple-criteria decision analysis discipline which studies how to make decisions when we have multiple conflicting criteria.
This tool uses a weighted product model. The unnormalised suffering and emissions scores are:
1. exponentiated by their corresponding weight,
2. multiplied together to get a combined score,
3. the combined score is normalised to the 0-100 range for cleaner display.
WPM is a dimensionless method used for ranking options when making decisions. That is, to answer questions like “is it more important to avoid chicken or beef” not “what is the cardinal utility of avoiding chicken”. This model is only useful for prioritising if I have decided to reduce meat consumption but am only able to leave one species off my plate. I understand now that I should have made it more clear.
Somehow measuring the utility of leaving a species off my plate would be much more interesting but seemed difficult considering the time and skills I had. I did consider using something like DALYs. There is research on converting emissions to DALYs which would allow us to use a parameter for converting non-human animal DALYs to human DALYs but I decided for the simpler ranking-only model.
That makes sense. The point I’m trying to make, though, is that the choice of how to do the conversion from CO2/kcal to hours/kcal is probably the most important bit that drives the results. I’d prefer to make that clearer to users, and get them to make their own assessment.
Instead, the WPM ends up coming up with an implicit conversion rate, which could be way different from what the person would say if asked. Given this, it seems like the results can’t be trusted.
(I expect a WPM would be fine in domains where there are multiple difficult-to-compare criteria and we’re not sure which criteria are most important – as in many daily decisions – but in this case, it could easily be that either CO2 or suffering should totally dominate your ranking, and it just depends on your worldview.)
You are right. I spent time thinking about your comments and I agree that making the tradeoff clearer is one of the most important improvements I can make. Thank you for bringing it out.