FWIW I’m also suspicious of the 0.001 DALYs per person number.
AFAICT, the way you get it is by combining two methods: method 1 is to ask people a chain of questions like “as a fraction of death, how bad is life imprisonment”, “as a fraction of life imprisonment, how bad is not being able to eat tasty stuff”, “as a fraction of not being able to eat tasty stuff, how bad is not being able to have sugary drinks”, multiply their answers to get how bad losing sugary drinks is as a fraction of dying, and then multiply by the fraction 64⁄74 (for remaining life years? this was opaque to me), to get a DALY loss of 0.016 +/- 0.009 [1]. You then do method 2: ask people how much of their annual income they’d give up to get a 1-year exemption from a ban on drinking sugary drinks, take the binary logarithm of 1 + that fraction, and multiply by 2 to get DALY loss. This gives you a loss of 0.0012 +/- 0.0009 [1]. You then average each respondent’s result from each method to get a per-respondent DALY loss estimate, before aggregating that accross respondents. Because the standard deviation of responses from method 1 is 10 times higher than that of method 2 [2], you weight method 2 10x higher in the per-respondent average, meaning that the overall loss is basically just that of method 2.
But I don’t think you’re right to conclude that method 2 is more accurate than method 1: it’s just that method 2 gives ~10x smaller results for whatever reason, so it makes sense that its error is also ~10x smaller. If you look at the spread in responses as a fraction of the mean response, methods 1 and 2 are pretty close (if anything, it looks like method 1 is a bit more precise). If you instead weighted the methods equally, you would get 3x the per-person DALY loss [3], and if I’m right in the parent comment, that would net out to a 15% reduction in the value of the program.
(also more fundamentally, the fact that the methods give 10x different values suggests that they plausibly are just measuring different things, and we should be unsure which (if either) is actually measuring the disvalue of the loss of freedom to drink sugary drinks)
[1] My error here is the standard error of the mean of each result: basically, how much we’d expect our calculated mean to vary if we resampled. It’s equal to the empirical standard deviation divided by the square root of the number of samples (which is 4).
[2] You also list one benefit of method 1 and one benefit of method 2, which I’m assuming cancel out in your considerations.
[3] Sanity check: the mean of the first method is 10x bigger than the second method, previously we were ~ignoring the first method, now we’re taking the geometric mean, and the geometric mean of 1 and 10 is 3 (because 3^2 is about 10), so this looks right.
FWIW I’m also suspicious of the 0.001 DALYs per person number.
AFAICT, the way you get it is by combining two methods: method 1 is to ask people a chain of questions like “as a fraction of death, how bad is life imprisonment”, “as a fraction of life imprisonment, how bad is not being able to eat tasty stuff”, “as a fraction of not being able to eat tasty stuff, how bad is not being able to have sugary drinks”, multiply their answers to get how bad losing sugary drinks is as a fraction of dying, and then multiply by the fraction 64⁄74 (for remaining life years? this was opaque to me), to get a DALY loss of 0.016 +/- 0.009 [1]. You then do method 2: ask people how much of their annual income they’d give up to get a 1-year exemption from a ban on drinking sugary drinks, take the binary logarithm of 1 + that fraction, and multiply by 2 to get DALY loss. This gives you a loss of 0.0012 +/- 0.0009 [1]. You then average each respondent’s result from each method to get a per-respondent DALY loss estimate, before aggregating that accross respondents. Because the standard deviation of responses from method 1 is 10 times higher than that of method 2 [2], you weight method 2 10x higher in the per-respondent average, meaning that the overall loss is basically just that of method 2.
But I don’t think you’re right to conclude that method 2 is more accurate than method 1: it’s just that method 2 gives ~10x smaller results for whatever reason, so it makes sense that its error is also ~10x smaller. If you look at the spread in responses as a fraction of the mean response, methods 1 and 2 are pretty close (if anything, it looks like method 1 is a bit more precise). If you instead weighted the methods equally, you would get 3x the per-person DALY loss [3], and if I’m right in the parent comment, that would net out to a 15% reduction in the value of the program.
(also more fundamentally, the fact that the methods give 10x different values suggests that they plausibly are just measuring different things, and we should be unsure which (if either) is actually measuring the disvalue of the loss of freedom to drink sugary drinks)
[1] My error here is the standard error of the mean of each result: basically, how much we’d expect our calculated mean to vary if we resampled. It’s equal to the empirical standard deviation divided by the square root of the number of samples (which is 4).
[2] You also list one benefit of method 1 and one benefit of method 2, which I’m assuming cancel out in your considerations.
[3] Sanity check: the mean of the first method is 10x bigger than the second method, previously we were ~ignoring the first method, now we’re taking the geometric mean, and the geometric mean of 1 and 10 is 3 (because 3^2 is about 10), so this looks right.