First, The google doc states that the life-years affected per dollar is 12 to 120, but Sallius report says it’s range is 12 to 160. Why the difference? Is this just a typo in the google doc?
I believe that is a typo in the doc. The model linked from the doc uses a log normal distribution between 13 and 160 in the relevant row (Hen years / $). (I can’t speak to why we chose 13 rather than 12, but this difference is negligible.)
Second, the default values in the tool are given as 160 to 3600. Why is this range higher (on a percentage basis) than the life years affected? Is this due to uncertainty somehow?
You’re right that this is mislabeled. The range is interpreted as units ‘per $1000’ rather than per dollar as the text suggests. Both the model calculations and the default values assume the per $1000 interpretation. The parameter labeling will be corrected, but the displayed results for the defaults still reflect our estimates.
Finally and most importantly, the report here seems to state that each hen is in the laying phase for approximately 1 year (40-60 weeks), and that switching from cage to cage-free averts roughly 2000 hours of hurtful pain and 250 hours of disabling pain (and that excruciating pain is largely negligible). If I take the maximum DALY conversion of 10 for disabling and 0.25 for hurtful (and convert hours to years), I get an adjusted result of (25010 + 0.252000)/(365*24) = 0.34 DALYs per chicken affected per year. If I multiply this by sallius estimate, I get a lower value than the straight “life years affected”, but the default values are actually around 13 time higher. Have I made a mistake here? I couldn’t find the exact calculations
The main concerns here probably result from the mislabeling, but if you’re interested in the specifics, Laura’s model (click over to the spreadsheet) predicts 0.23 DALYs per $ per year (with 2 chickens per $ affected). This seems in line with your calculations given your more pessimistic assumptions. These numbers are derived from the weights via the calculations labeled “Annual CC/CF DALYS/bird/yr” under ‘Annual DALY burden’.
Thanks, hope the typos will be fixed. I think I’ve almost worked through everything to replicate the results, but the default values still seem off.
If I take sallius’s median result of 54 chicken years life affected per dollar, and then multiply by Laura’s conversion number of 0.23 DALYs per $ per year, I get a result of 12.4 chicken years life affected per dollar. If I convert to DALY’s per thousand dollars, this would result in a number of 12,420.
This is outside the 90% confidence interval for the defaults given on the site, which gives it as “between 160 and 3.6K suffering-years per dollar”. If I convert this to the default constant value, it gives the suggested value of 1,900, which is roughly ten time lower than the value if I take Sallius’s median and laura’s conversion factor.
If I put in the 12420 number into the field, the site gives out 4630 DALY’s per thousand dollars, putting it about 10 times higher than originally stated in the post, which seems more in line with other RP claims (after all, right now the chicken campaign is presented as only 10 times more cost effective, whereas others are claiming it’s 1000x more effective using RP numbers).
If I take sallius’s median result of 54 chicken years life affected per dollar, and then multiply by Laura’s conversion number of 0.23 DALYs per $ per year, I get a result of 12.4 chicken years life affected per dollar. If I convert to DALY’s per thousand dollars, this would result in a number of 12,420.
Laura’s numbers already take into account the number of chickens affected. The 0.23 figure is a total effect to all chickens covered per dollar per year. To get the effect per $1000, we need to multiply by the number of years the effect will last and by 1000. Laura assumes a log normal distribution for the length of the effect that averages to about 14 years. So roughly, 0.23 * 14 * 1000 = 3220 hen DALYs per 1000 dollars.
Note: this is hen DALYs, not human DALYs. To convert to human DALYs we would need to adjust by the suffering capacity and sentience. In Laura’s model (we use slightly different values in the CCM), this would mean cutting the hen DALYs by about 70% and 10%, resulting in about 900 human-equivalent DALYs per 1000 dollars total over the lifespan of the effect. Laura was working in a Monte Carlo framework, whereas the 900 DALY number is derived just from multiplying means, so she arrived at a slightly different value in her report. The CCM also uses slightly different parameter settings for moral weights, but the result it produces still is in the same ballpark.
Laura’s numbers already take into account the number of chickens affected. The 0.23 figure is a total effect to all chickens covered per dollar per year. To get the effect per $1000, we need to multiply by the number of years the effect will last and by 1000. Laura assumes a log normal distribution for the length of the effect that averages to about 14 years. So roughly, 0.23 * 14 * 1000 = 3220 hen DALYs per 1000 dollars.
I’m sorry, but this just isn’t true. You can look at the field for “annual CC DALYs per bird per year” here (with the 0.2 value), it does not include Saulius’s estimates. (I managed to replicate the value and checked it against the fields here, they match).
Saulius’s estimates already factor inthe 14 year effect of the intervention. You’ll note that the “chickens affected per dollar” is multiplied by the mean years of impact when giving out the “12 to 160” result.
Saulius is saying that each dollar affects 54 chicken years of life, equivalent to moving 54 chickens from caged to cage free environments for a year. The DALY conversion is saying that, in that year, each chicken will be 0.23 DALY’s better off. So in total, 54*0.23 = 12.43 DALYs are averted per dollar, or 12430 DALYS per thousand, as I said in the last comment. However, I did notice in here that the result was deweighted by 20%-60% because they expected future campaigns to be less effective, which would bring it down to around 7458.
I didn’t factor in the moral conversions because those are seperate fields in the site. If I use P(sentience) of 0.8 and moral weight of 0.44 as the site defaults to, the final DALy per thousand should be 7458*0.8*0.44= 2386 DALYs/thousand dollars, about three times more than the default value on the site.
Saulius is saying that each dollar affects 54 chicken years of life, equivalent to moving 54 chickens from caged to cage free environments for a year. The DALY conversion is saying that, in that year, each chicken will be 0.23 DALY’s better off. So in total, 54*0.23 = 12.43
I don’t believe Saulius’s numbers are directly used at any point in the model or intended to be used. The model replicates some of the work to get to those numbers. That said, I do think that you can use your approach to validate the model. I think the key discrepancy here is that the 0.23 DALY figure isn’t a figure per bird/year, but per year. The model also assumes that ~2.18 birds are affected per dollar. The parameter you would want to multiply by Saulius’s estimate is the difference between Annual CC Dalys/bird/year and Annual CF Dalys/bird/year, which is ~0.1. If you multiply that through, you get about ~1000 DALYs/thousand dollars. This is still not exactly the number Laura arrives at via her Monte Carlo methods and not exactly the estimate in the CCM, but due to the small differences in parameters, model structure, and computational approaches, this difference is in line with what I would expect.
Okay, I was looking at the field DALYs per bird per year” in this report, which is 0.2 matching with I have replicated. The 0.23 figure is actually something else, which explains a lot of the confusion in this conversation. I’ll include my calculation at the end.
Before I continue, I want to thank you for being patient and working with me on this. I think people are making decisions based on these figures so it’s important to be able to replicate them.
This report states that Saulius’s numbers are being used:
I estimate the “chicken-DALYs” averted per $1000 spent on corporate campaigns, conditioned on hens being sentient. To do so, I use Šimčikas’ rport, data from the Welfare Footprint Project on the duration of welfare harms in conventional and cage-free environments, and intensity weights by type of pain.
I think I’ve worked it out: if we take the 2.18 birds affected per year and multiply by the 15 year impact, we get 32.7 chicken years affected/dollar , which is the same as the 54 chicken years given by saulius discounted by 40% (54*0.6 = 32.4). This is the number that goes into the 0.23 figure, and this does already take into account the 15 years of impact.
So I don’t get why there’s still a discrepancy: although we take different routes to get there, we have the same numbers and should be getting the same results.
Before I continue, I want to thank you for being patient and working with me on this. I think people are making decisions based on these figures so it’s important to be able to replicate them.
I appreciate that you’re taking a close look at this and not just taking our word for it. It isn’t inconceivable that we made an error somewhere in the model, and if no one pays close attention it would never get fixed. Nevertheless, it seems to me like we’re making progress toward getting the same results.
Total DALYs averted:
4.47274/(36524) = 0.14 disabling DALYS averted
0.152259/(36524) = 0.0386 hurtful DALYS averted
0.015* 4645/(365*24) =0.00795 hurtful Dalys averted
Total is about 0.19 DALY’s averted per hen per year.
I take it that the leftmost numbers are the weights for the different pains? If so, the numbers are slightly different from the numbers in the model. I see an average weight of about 6 for disabling pain, 0.16 for hurtful pain, and 0.015 for annoying pain. This works out to ~0.23 in total. Where are your numbers coming from?
No worries, and I have finally managed to replicate Laura’s results, and find the true source of disagreement. The key factor missing was the period of egg laying: I put in ~1 year year for both Caged and uncaged, as is assumed on the site that provided the hours of pain figures. This 1 year of laying period assumption seems to match with other sources. Whereas in the causal model, the caged length of laying is given as 1.62 years, and the cage free length of laying is given as 1.19 years. The causal model appears to have tried to calculate this, but it makes more sense to me to use the site that measured the pains estimate: I feel they made they measurements, they are unlikely to be 150% off, and we should be comparing like with like here.
When I took this into account, I was able to replicate Lauras results, which I have summarised in this google doc, which also contains my own estimate and another analysis for broilers, as well as the sources for all the figures.
My DALY weights were using the geometric means (I wasn’t sure how to deal with lognormal), but switching to regular averages like you suggest makes things match better.
Under lauras laying period, my final estimate is 3742 Chicken-Dalys/thousand dollars, matching well with the causal number of 3.5k (given i’m not using distributions). Discounting this by the 0.332 figure from moral weights (this includes sentience estimates, right?) gives a final DALY’s per thousand of 1242 (or 1162 if we use the 3.5k figure directly)
Under my laying period figures, the final estimate is 6352 Chicken-Dalys/thousand, which discounted by the RP moral weights comes to 2108 DALYs/thousand dollars. A similar analysis for broilers gives 1500 chicken-dalys per thousand dollars and 506 DALY’s per thousand dollars.
The default values from the cross cause website should match with either Laura’s or mines estimates.
I believe that is a typo in the doc. The model linked from the doc uses a log normal distribution between 13 and 160 in the relevant row (Hen years / $). (I can’t speak to why we chose 13 rather than 12, but this difference is negligible.)
You’re right that this is mislabeled. The range is interpreted as units ‘per $1000’ rather than per dollar as the text suggests. Both the model calculations and the default values assume the per $1000 interpretation. The parameter labeling will be corrected, but the displayed results for the defaults still reflect our estimates.
The main concerns here probably result from the mislabeling, but if you’re interested in the specifics, Laura’s model (click over to the spreadsheet) predicts 0.23 DALYs per $ per year (with 2 chickens per $ affected). This seems in line with your calculations given your more pessimistic assumptions. These numbers are derived from the weights via the calculations labeled “Annual CC/CF DALYS/bird/yr” under ‘Annual DALY burden’.
Thanks, hope the typos will be fixed. I think I’ve almost worked through everything to replicate the results, but the default values still seem off.
If I take sallius’s median result of 54 chicken years life affected per dollar, and then multiply by Laura’s conversion number of 0.23 DALYs per $ per year, I get a result of 12.4 chicken years life affected per dollar. If I convert to DALY’s per thousand dollars, this would result in a number of 12,420.
This is outside the 90% confidence interval for the defaults given on the site, which gives it as “between 160 and 3.6K suffering-years per dollar”. If I convert this to the default constant value, it gives the suggested value of 1,900, which is roughly ten time lower than the value if I take Sallius’s median and laura’s conversion factor.
If I put in the 12420 number into the field, the site gives out 4630 DALY’s per thousand dollars, putting it about 10 times higher than originally stated in the post, which seems more in line with other RP claims (after all, right now the chicken campaign is presented as only 10 times more cost effective, whereas others are claiming it’s 1000x more effective using RP numbers).
Laura’s numbers already take into account the number of chickens affected. The 0.23 figure is a total effect to all chickens covered per dollar per year. To get the effect per $1000, we need to multiply by the number of years the effect will last and by 1000. Laura assumes a log normal distribution for the length of the effect that averages to about 14 years. So roughly, 0.23 * 14 * 1000 = 3220 hen DALYs per 1000 dollars.
Note: this is hen DALYs, not human DALYs. To convert to human DALYs we would need to adjust by the suffering capacity and sentience. In Laura’s model (we use slightly different values in the CCM), this would mean cutting the hen DALYs by about 70% and 10%, resulting in about 900 human-equivalent DALYs per 1000 dollars total over the lifespan of the effect. Laura was working in a Monte Carlo framework, whereas the 900 DALY number is derived just from multiplying means, so she arrived at a slightly different value in her report. The CCM also uses slightly different parameter settings for moral weights, but the result it produces still is in the same ballpark.
I’m sorry, but this just isn’t true. You can look at the field for “annual CC DALYs per bird per year” here (with the 0.2 value), it does not include Saulius’s estimates. (I managed to replicate the value and checked it against the fields here, they match).
Saulius’s estimates already factor in the 14 year effect of the intervention. You’ll note that the “chickens affected per dollar” is multiplied by the mean years of impact when giving out the “12 to 160” result.
Saulius is saying that each dollar affects 54 chicken years of life, equivalent to moving 54 chickens from caged to cage free environments for a year. The DALY conversion is saying that, in that year, each chicken will be 0.23 DALY’s better off. So in total, 54*0.23 = 12.43 DALYs are averted per dollar, or 12430 DALYS per thousand, as I said in the last comment. However, I did notice in here that the result was deweighted by 20%-60% because they expected future campaigns to be less effective, which would bring it down to around 7458.
I didn’t factor in the moral conversions because those are seperate fields in the site. If I use P(sentience) of 0.8 and moral weight of 0.44 as the site defaults to, the final DALy per thousand should be 7458*0.8*0.44= 2386 DALYs/thousand dollars, about three times more than the default value on the site.
I don’t believe Saulius’s numbers are directly used at any point in the model or intended to be used. The model replicates some of the work to get to those numbers. That said, I do think that you can use your approach to validate the model. I think the key discrepancy here is that the 0.23 DALY figure isn’t a figure per bird/year, but per year. The model also assumes that ~2.18 birds are affected per dollar. The parameter you would want to multiply by Saulius’s estimate is the difference between Annual CC Dalys/bird/year and Annual CF Dalys/bird/year, which is ~0.1. If you multiply that through, you get about ~1000 DALYs/thousand dollars. This is still not exactly the number Laura arrives at via her Monte Carlo methods and not exactly the estimate in the CCM, but due to the small differences in parameters, model structure, and computational approaches, this difference is in line with what I would expect.
Okay, I was looking at the field DALYs per bird per year” in this report, which is 0.2 matching with I have replicated. The 0.23 figure is actually something else, which explains a lot of the confusion in this conversation. I’ll include my calculation at the end.
Before I continue, I want to thank you for being patient and working with me on this. I think people are making decisions based on these figures so it’s important to be able to replicate them.
This report states that Saulius’s numbers are being used:
I think I’ve worked it out: if we take the 2.18 birds affected per year and multiply by the 15 year impact, we get 32.7 chicken years affected/dollar , which is the same as the 54 chicken years given by saulius discounted by 40% (54*0.6 = 32.4). This is the number that goes into the 0.23 figure, and this does already take into account the 15 years of impact.
So I don’t get why there’s still a discrepancy: although we take different routes to get there, we have the same numbers and should be getting the same results.
My calculation, taken from here.
laying time is 40 to 60 weeks, so we’ll assume it goes for exactly 1 year.
Disabiling: 430-156 = 274 hours disabling averted
Hurtful = 4000-1741 = 2259 hours hurtful averted.
Annoying 6721-2076 =4645 hours annoying averted.
Total DALYs averted:
4.47*274/(365*24) = 0.14 disabling DALYS averted
0.15*2259/(365*24) = 0.0386 hurtful DALYS averted
0.015* 4645/(365*24) =0.00795hurtful Dalys averted
Total is about 0.19 DALY’s averted per hen per year.
I appreciate that you’re taking a close look at this and not just taking our word for it. It isn’t inconceivable that we made an error somewhere in the model, and if no one pays close attention it would never get fixed. Nevertheless, it seems to me like we’re making progress toward getting the same results.
I take it that the leftmost numbers are the weights for the different pains? If so, the numbers are slightly different from the numbers in the model. I see an average weight of about 6 for disabling pain, 0.16 for hurtful pain, and 0.015 for annoying pain. This works out to ~0.23 in total. Where are your numbers coming from?
No worries, and I have finally managed to replicate Laura’s results, and find the true source of disagreement. The key factor missing was the period of egg laying: I put in ~1 year year for both Caged and uncaged, as is assumed on the site that provided the hours of pain figures. This 1 year of laying period assumption seems to match with other sources. Whereas in the causal model, the caged length of laying is given as 1.62 years, and the cage free length of laying is given as 1.19 years. The causal model appears to have tried to calculate this, but it makes more sense to me to use the site that measured the pains estimate: I feel they made they measurements, they are unlikely to be 150% off, and we should be comparing like with like here.
When I took this into account, I was able to replicate Lauras results, which I have summarised in this google doc, which also contains my own estimate and another analysis for broilers, as well as the sources for all the figures.
My DALY weights were using the geometric means (I wasn’t sure how to deal with lognormal), but switching to regular averages like you suggest makes things match better.
Under lauras laying period, my final estimate is 3742 Chicken-Dalys/thousand dollars, matching well with the causal number of 3.5k (given i’m not using distributions). Discounting this by the 0.332 figure from moral weights (this includes sentience estimates, right?) gives a final DALY’s per thousand of 1242 (or 1162 if we use the 3.5k figure directly)
Under my laying period figures, the final estimate is 6352 Chicken-Dalys/thousand, which discounted by the RP moral weights comes to 2108 DALYs/thousand dollars. A similar analysis for broilers gives 1500 chicken-dalys per thousand dollars and 506 DALY’s per thousand dollars.
The default values from the cross cause website should match with either Laura’s or mines estimates.