As you point out, the section on population is kind of messy. I spent a bit trying to think about how to express uncertainty in a population pyramid, but didn’t come up with anything particularly elegant (in particular, you can have uncertainty about what fraction of the population is of a particular age, but then you have to ensure that it sums up to 100%, which is annoying). Some scratchpad thoughts here.
Edit: Ok, this bothered me. You could specify the uncertainty over number of people reached in each age directly, and do something like this (using fake numbers):
num_people_reached_by_distribution_of_age(a) = {
1 to 10 // fake dist
}
mortality_reduction_by_age(a) = {
reduction = a < 1 ? 0.05 : 0.05* 1/a // figure out life expectancy at age in target population.
reduction
}
value_of_life_at_age(a) = {
to(1+a, 2 + a^2)
}
integrate(fun, min, max) = {
// assume that min and max are integers.
epsilon = 1
l = max - min
intervals = map(List.upTo(0, (l/epsilon)), ({|n| min + n*epsilon}))
values = map(intervals, ({|x| fun(x)}))
result = reduce(values, 0, ({|acc, x| acc + x})) * epsilon
result
}
// Sanity check
// Check that this distribution corresponds to the uncertainty of the overall distribution
num_people_reached = integrate(num_people_reached_by_distribution_of_age, 0, 100) // make sure this makes sense
// Value of the distribution
value_from_age(a) = {
n = num_people_reached_by_distribution_of_age(a)
mortality_reduction_in_group = mortality_reduction_by_age(a)
value_of_life = value_of_life_at_age(a)
result = n * mortality_reduction_in_group * value_of_life
result
}
value_of_distribution = integrate(value_from_age, 0, 100)
The problem with this is that specifying the number of people of each age reached by a distribution directly looks a bit annoying.
GiveWell starts with the mortality per 1000 child-years in 2004, pre-distribution of LLINs in the countries where the AMF currently distributes nets.
Huh, seems pretty long ago.
We can now calculate the mortality today relative to the pre-nets 2004 baseline. GiveWell estimates that 25% of this mortality difference can be attributed to LLIN distributions.
- Uncertainty for people under 5 is shockingly wide - My uncertainty about the lack of counterfactualness adjustment would be wider and my estimate lower (i.e., less impact). If AMF didn’t exist, some other org might step in (but then that org would not have capacity for their marginal project? unclear).
A few points as I read this:
As you point out, the section on population is kind of messy. I spent a bit trying to think about how to express uncertainty in a population pyramid, but didn’t come up with anything particularly elegant (in particular, you can have uncertainty about what fraction of the population is of a particular age, but then you have to ensure that it sums up to 100%, which is annoying). Some scratchpad thoughts here.
Edit: Ok, this bothered me. You could specify the uncertainty over number of people reached in each age directly, and do something like this (using fake numbers):
The problem with this is that specifying the number of people of each age reached by a distribution directly looks a bit annoying.
Huh, seems pretty long ago.
Huh, interesting.
More notes:
adjustment_for_net_coverage_years_lost_due_to_residual_nets_from_previous_distributions could be a distribution
Means of cost-effectiveness can be misleading, see here. I would instead calculate the expected number of deaths averted, and divide it by the funds.
So instead of mean(cost_per_life_saved),
More notes:
- Uncertainty for people under 5 is shockingly wide
- My uncertainty about the lack of counterfactualness adjustment would be wider and my estimate lower (i.e., less impact). If AMF didn’t exist, some other org might step in (but then that org would not have capacity for their marginal project? unclear).