Shallow Report on Diabetes Mellitus Type 2
Note: This report has been superseded by our subsequent Intermediate Report on Diabetes Mellitus Type 2 - please browse that report for our most updated findings.
Summary
Taking into account the expected benefits of eliminating diabetes mellitus type 2 (i.e. improved health and greater economic output), as well as the tractability of policy advocacy for a sugar-sweetened beverages tax, I find that the marginal expected value of policy advocacy for a sugar-sweetened beverages tax to eliminate diabetes mellitus type 2 to be 634,277 DALYs per USD 100,000, which is around 1000x as cost-effective as giving to a GiveWell top charity (CEA).
Key Points
Importance: This is a strongly important cause, with 3.74 * 1010 DALYs at stake from now to the indefinite future. Around 69% of the burden is health related, while 31% is economic in nature.
Neglectedness: Whatever governments/charities/businesses are doing to solve this problem (e.g. existing extent of sugar taxes/efforts to expand access to metformin/food reformulation) may be making a difference, but clearly insufficient given rising DALY burdens, in the face of largely unfavourable structural trends (e.g. improving education may cause improved eating/exercise habits, but against this you have: an ageing population increasing disease burden per capita given that diabetes tends to affect older individuals more; economic growth involving greater access to western pattern diet foods; and larger future populations mechanically increasing disease burdens for the same DALY burden per capita).
Tractability: A moderately tractable solution is available, in the form of advocating for sugary drinks taxes. This is highly effective if and when implemented, and encouragingly many governments are already doing this – though of course, there is still uncertainty as to whether future efforts will be similarly successful.
Caveats
This report was produced with only one week of research, and critically, only desktop research was used, without experts consulted due to the lack of time. More research – at the intermediate stage and subsequently deep stage – will be needed before we can have high confidence in these findings.
The headline cost-effectiveness will almost certainly fall if this cause area is subjected to deeper research: (a) this is empirically the case, from past experience; and (b) theoretically, we suffer from optimizer’s curse (where causes appear better than the mean partly because they are genuinely more cost-effective but also partly because of random error favouring them, and when deeper research fixes the latter, the estimated cost-effectiveness falls). As it happens, CEARCH intends to perform deeper research in this area, given that the headline cost-effectiveness meets our threshold of 10x that of a GiveWell top charity.
Further Discussion
The whole analysis is predicated on projections of the growth in the disease burden of hypertension up to 2100, which is necessarily highly uncertain. Based on the available evidence, we should have high confidence in continuous growth over the next two decades at least, medium confidence in growth up to 2070, and low confidence that this will last till 2100 (as opposed to levelling off or declining).
We should be somewhat uncertain about the economic burden, which as a fraction of the health burden is fairly high relative to the norm in other cause areas previously examined (e.g. hypertension, fungal infections).
There are multiple pathways to impact for the sugar tax – a higher price will of course reduce consumption through substitution and income efforts, but there is also a signalling effect (i.e. reminding people that sugar is bad and they should consider reducing their consumption of it independent on any economic incentives), and a reformulation effort (i.e. producers try to lessen the impact of the tax on profits by using less sugar in their drinks, which not only reduces sugar intake per sugary drink consumed, but also the probability that sugar drinks are consumed relative to baseline, insofar as SSBs that are less sweet are less attractive to consumers)
Even in jurisdictions where there are sugar taxes, additional gains are possible through further raising taxes.
A non-financial cost to the intervention is reduced freedom of choice, but such costs typically tend to be minor compared to the health benefits, based on past analysis of similar health policy interventions (e.g. tobacco, alcohol, road traffic safety).
Expected Benefit: Improved Health from Eliminating Diabetes Mellitus Type 2
The primary expected benefit from eliminating diabetes mellitus type 2 is improved health, in terms of fewer deaths as well as less disability and suffering. Overall, around 2.57 * 1010 DALYs are at stake here, with this benefit modelled in the following way.
Moral Weights & Scale: The global disease burden of diabetes mellitus type 2 in DALY terms for our baseline year – 2024 – is around 7.48 * 107 DALYs. This is calculated by using the latest available 2019 figures and projecting forward using a model that will be discussed in greater detail subsequently.
Persistence: The problem of diabetes mellitus is likely to persist, and eliminating it will bring benefits not just for one year but across multiple years. In terms of how this multi-year benefit is calculated:
Firstly, I discount for the probability of the solution – a sugar-sweetened beverages tax, the choice of which will be discussed later – being reversed. To calculate the rate of policy reversal on sugar-sweetened beverages being taxed, I take jurisdiction-years in which repeal occurred and divide by the jurisdiction-years in which the tax existed and repeal could have occurred (whether or not it did); jurisdictions here refer to not just countries but also states, overseas territories, indigenous reservations and cities. Overall, this gives us a reversal rate of 0.5% per annum.
Secondly, I model the proportion of disease burden remaining after being counterfactually solved (i.e. DMT2 declining due to the intervention of other actors or else due to structural changes). To model this growth, I use the simple theoretical model that takes total DALYs lost to DMT2 to be a function of DALYs lost per capita and population size.
Note that whatever efforts that agents (i.e. governments, nonprofits and businesses) are making to solve the problem (e.g. taxing sugar, expanding access to metformin at nonprofit hospitals, reformulating sugar-sweetened beverages), and whatever impact that non-population structural trends are having (e.g. an ageing population increasing disease burden per capita given that diabetes tends to affect older individuals more; or economic growth involving greater access to western pattern diet foods; or improving education causing improved eating/exercise habits) all this will involve either increased (or decreased) prevalence of DMT2 or the disease burden thereof, and hence greater (or fewer) DALYs lost per capita; in short, the variable of DALYs lost per capita accounts for all these agentic and structural factors. The only exception, of course, is population size (where a larger population mechanically increases the disease burden for a given disease burden per capita), with that handled separately in this model.
Overall, to model how the problem is expected to evolve over the years, I (a) project future DALYs lost per capita to DMT2 by estimating the year-on-year change via a linear regression of past DALYs lost per capita on discrete time; (b) use UN estimates of projected future population growth; and (c) multiply each future year’s DALYs lost per capita and population size to obtain the expected total DALY burden for each year. That said, I limit the extrapolation to 2100, as the high-confidence UN estimate ends there, and since projections of future DALYs lost per capita based on past trends will be unreliable in the far future (i.e. the next century); I assume constancy thereafter.
Note that the estimates here are consistent with expert modelling of future DMT2 prevalence, with Khan et al’s statistical modelling using 1990–2017 GBD data suggesting that global diabetes prevalence could increase to 7079 per 100,000 by 2030 and 7862 per 100,000 by 2040. Given increased prevalence (and hence the probability that any single individual will suffer diabetes), and conservatively assuming DALYs lost per diabetes sufferer remains constant (in fact, it is likely to increase, insofar as the trends worsening insulin resistance and pancreatic exhaustion and hence diabetes prevalence also worsens the severity of diabetes per sufferer), this vindicates the idea that DALYs lost per capita will increase in the future. Further note that the increasingly large DALYs lost per capita, as projected up to 2100, are far from unrealistic – the 2100 figure, which is the highest, is only about what the Czech Republic suffered as of 2019, and indeed 33 other countries experienced DALY burdens per capita even higher than that.
The projected growth of DALYS lost due to diabetes mellitus type 2 up to 2100, after taking into account counterfactual solution, is shown in Diagram 1.
Diagram 1: Remaining DALYs lost to diabetes mellitus type 2 after counterfactual solution
Thirdly, I discount for the probability of the world being destroyed anyway (i.e. general existential risk discount) – around 0.07% per annum. This takes into account the probability of extinction, since the benefits of saving people from diabetes mellitus type 2 in one year is nullified if they would die in an extinction event anyway. For how this risk is calculated, refer to CEARCH’s shallow research on nuclear war.
Fourthly, I apply a broad uncertainty discount of 0.1% per annum to take into account the fact that there is a non-zero chance that in the future, the benefits or costs do not persist for factors we do not and cannot identify in the present (e.g. actors directing resources to solve the problem when none are currently doing so).
Overall, by taking the remaining DALYs lost to diabetes mellitus type 2 after counterfactual solution, and discounting each year’s DALY burden using the other per annum discounts (i.e. solution reversal discount, existential risk discount, uncertainty discount), the total amount of DALYs available for policy advocacy for a sugar-sweetened beverages tax to counterfactually avert is shown in Diagram 2.
Diagram 2: DALYs available for a policy advocacy for a sugar-sweetened beverages tax to counterfactually avert
Finally, by summing the discounted per annum relative values for 2024-2100, and then using a perpetual value formula for 2101 to infinity, we see that the benefit of improved health from eliminating diabetes mellitus type 2 will last for the equivalent of 344 baseline years.
Value of Outcome: Overall, the raw value of improved health from eliminating diabetes mellitus type 2 is 2.57 * 1010 DALYs.
Probability of Occurrence: Unlike longtermist problems, there is no uncertainty that this is an actual problem. Diabetes was a problem a decade ago (n.b. in 2010, an estimated 285 million people worldwide had diabetes mellitus, 90% of whom had type 2 diabetes mellitus) and as a problem it has hardly faded away since; indeed it has grown (n.b. the total number of diabetes mellitus is projected to rise to 439 million by 2030; and one thing which doesn’t help is a vicious intrauterine exposure cycle, where exposure to intrauterine hyperglycemia is an important determinant of diabetes mellitus in adult offspring in addition to genetic susceptibility and independent of maternal diabetes type) – hence, the probability that diabetes mellitus type 2 is a problem can be assigned ~1.
Expected Value: Hence, the expected value of improved health from eliminating diabetes mellitus type 2 is 2.57 * 1010 DALYs.
Expected Benefit: Increased Economic Output
Beyond the health benefits, there are also economic benefits to eliminating diabetes mellitus type 2. Around 1.18 * 1010 DALYs are at stake here, as calculated in the following manner.
Moral Weights: I take the value of doubling consumption for one person for one year in DALY terms to 0.21. This is calculated as a function of (a) the value of consumption relative to life from GiveWell’s IDinsight survey of the community perspective, as adjusted for social desirability bias, and (b) CEARCH’s estimate of the value of a full, healthy life in DALY terms. For more details, refer to CEARCH’s evaluative framework.
Scale: I start by calculating the economic burden of diabetes mellitus type 2 relative to annual income per DMT2 sufferer, using three separate estimates to do so.
The first estimate is the American Diabetes Association’s. I take this estimate of total US national costs of diabetes (both direct treatment and indirect productivity costs), dividing by number of diabetes sufferers, and then dividing by annual income. The second estimate is Koster et al’s. I take this estimate of Germany’s counterfactual cost per diabetic sufferer (both direct treatment and indirect productivity costs), converting euros to dollars, and then dividing by annual income. The third estimate is Wang et al’s. I take this estimate of China’s average cost per diabetic sufferer (both direct treatment and indirect productivity costs), before dividing by annual income. Using equal weightage, the average economic burden of diabetes mellitus type 2 relative to annual income per DMT2 sufferer is 0.33. This in turn yields the degree of consumption doubling per DMT2 sufferer if their DMT2 were prevented or cured.
At the same time, the total number of diabetes mellitus type 2 sufferers in the baseline year of 2024 is around 488 million – as predicted using 2019 GBD baseline and predicting forward based on past growth trends.
Multiplying the degree of consumption doubling per diabetes mellitus type 2 sufferer if their DMT2 is eliminated by the total number of DMT2 sufferers as of 2024, the total number of consumption doublings achievable by eliminating DMT2 in the baseline year of 2024 is around 163 million.
Persistence: The same per annum discounts and the same projections of the disease burden (and hence economic burden) over time, as discussed in the previous section, are used here as well, such the benefit of increased economic output will similarly last for the equivalent of 344 baseline years.
Value of Outcome: Overall, the raw value of increased economic output is 1.17 * 1010 DALYs.
Probability of Occurrence: Same probability as before is applied.
Expected Value: All in all, the expected value of increased economic output is 1.17 * 1010 DALYs.
Tractability
To summarize our tractability findings: we can solve 0.00004 of the problem with a USD 208,000 investment into policy advocacy for a sugar-sweetened beverages tax (factoring in the subsequent governmental costs of regulation), which means the proportion of the problem solved per additional USD 100,000 spent is around 0.00002.
In terms of eliminating the disease burden of DMT2, we can hypothetically either prevent the disease from occuring in the first place, or treat it such that the disease does not burden its sufferers. That said, there are strong theoretical reasons to favour treatment: (a) the health burden will be higher under treatment vs prevention, given that treatment may not start simultaneously with the onset of the disease and given that treatment may not fully eliminate the disease burden; and (b) the economic burden will be higher as well, not just because of the disease burden being higher for the aforementioned reasons, but also because treatment with drugs etc will cost more compared to a preventive approach that may not necessarily involve drugs but comparatively costless dietary changes (whether through voluntary lifestyle changes or policy changes).
Meanwhile, in terms of the best preventive treatment at a population level – a sugar tax seems most effective, compared to distributing drugs (which requires high-risk individuals to agree, tolerate and adhere to the drug regime) and voluntary lifestyle interventions (which, while cost-effective at the individual level, have poor overall scale and uptake as well as declining adherence). Note that while a sugar tax theoretically is harder to successfully lobby for relative to advocacy for the distribution of metfornin and lifestyle interventions, it also does not face the difficulty and cost of screening and identifying specific high-risk individuals. Note also that the issue of uptake is what prevents us from using raw cost-effectiveness figures from systematic reviews to simply decide what intervention to pursue – an intervention might be cost-effective conditional on uptake, but uptake itself is not guaranteed (e.g. not just for lifestyle interventions, but also screening, drug therapy, counselling and treatment for smoking, etc).
In terms of our theory of change:
Step 1: Lobby a government to raise taxes on sugar-sweetened beverages
Step 2: Sugar-sweetened beverages tax reduces consumption of sugar-sweetened beverages
Step 3: Reduced consumption of sugar-sweetened beverages reduces the prevalence and disease burden of diabetes mellitus type 2 in a single country and hence proportionally the global disease burden
Step 1: To estimate the probability of successfully lobbying a government to raise taxes on sugar-sweetened beverages, I take both an outside and inside view.
For the outside view, I consult three reference classes.
The first reference class is simply the success rate of previous attempts to raise taxes on sugar-sweetened beverages. We can obviously consult the past track record of attempts to raise such taxes, and estimate the base rate of advocacy success by taking instances of success and dividing through by attempts at advocacy.
Meanwhile, the second reference class is the success rate of sodium tax advocacy. This is relevant to the extent that this is another health policy intervention that involves lobbying governments to introduce or raise taxes on food. This is calculated here by taking the number of governments that have implemented a tax on sodium, and dividing by the number of governments trying to reduce salt consumption (which indicates intra-government efforts to control salt, as a proxy for external efforts).
Finally, the third reference class is the success rate of tobacco tax advocacy. This is relevant to the extent that this is another health policy intervention that involves lobbying governments to introduce or raise taxes on a consumption good.
Aggregating these three reference classes, I use the following weighting scheme. While the case study analysis of the success rate of attempts to raise taxes on sugar-sweetened beverages is obviously directly relevant, there are serious worries about selection bias – the jurisdictions that even see attempts to implement such taxes will doubtlessly have a higher baseline level of support for such taxes (such that health advocates see that they have a reasonable chance of success and will even make the attempt in the first place). Hence, a penalty is applied to this reference class. Meanwhile, the sodium reference class is also flawed – not just because it is less directly relevant, but also because intra-government lobbying is not especially representative of external lobbying efforts; it skips the first step in any policy advocacy theory of change (i.e. securing internal champions for your position). Hence, a double penalty is applied. Lastly, the tobacco taxation reference class is also flawed – it is even less relevant, not being about a tax on food. Hence, a double penalty is applied. Overall, therefore, I weigh the SSB case study more relative to the other two, and produce an estimate of lobbying success of 77%.
For the inside view, I reason as follows. Many countries are imposing sugar taxes, which suggests that a sugar tax is politically feasible and that successful advocacy is not extremely improbable (i.e. <=10%); that said, policy advocacy is fundamentally difficult, and my sense is that there is a less than even (i.e. <50%) chance of success; finally, the fact that advocates can point to a long list of countries that have imposed sugar taxes already is highly beneficial to the advocacy process (n.b. policymakers care about international best practice), and I assign about a 33% chance that advocacy efforts are successful.
When aggregating the outside and inside views, we have to note that the outside view – as driven by the sugar tax reference class – is flawed due to the selection bias issue, while the inside view is subject to the usual worries about inferential uncertainty; hence, equal weightage is used, yielding about a 55% chance of advocacy success.
Step 2: To estimate the degree to which the sugar-sweetened beverages tax reduces consumption of sugar-sweetened beverages, I rely on an empirical estimate. Teng et al’s meta-analysis suggests that a 10% increase in sugar-sweetened beverage tax was associated with a decline in purchases and dietary intake of 10%. Note that this meta-analysis is of real world results, where there is strong relevance/external validity, but concerns over causal attribution/internal validity; hence, it would be beneficial to weigh these results against other available meta-analyses of (a) setting-specific experimental results, where there is strong causal attribution/internal validity, but concerns over relevance/external validity, and of (b) price change studies that simulate the effect of SSB taxes, where there there is strong relevance/external validity, but concerns over uncertainties over pass-through rates – the idea here being that averaging helps smooth out random errors and arrive at an estimate that is closer to the true elasticity. Equal weightage is used given that the studies all have their strengths and weaknesses. Tax is assumed to be 20%, which is what the WHO recommends, and which policy advocates will probably have an easier time pushing, and this yields an overall 20% reduction in consumption.
Step 3: To estimate the degree to which reduced consumption of sugar-sweetened beverages reduces the prevalence and disease burden of diabetes mellitus type 2 in a single country and hence proportionally the global disease burden, I similarly use an empirical estimate. This takes the population attributable fraction for DMT2 from consumption of sugar-sweetened beverages, and divides by the number of countries. The following assumptions are made – (a) US and UK data (which we rely upon) are representative; (b) consumption of SSBs vary linearly with prevalence; and (c) prevalence varies linearly with disease burden.
Overall, the proportion of disease reduction from sugar-sweetened beverage taxes policy advocacy to eliminate diabetes mellitus type 2 – as a function of (a) the probability of successfully lobbying a government to raise taxes on sugar-sweetened beverages; (b) the degree to which the sugar-sweetened beverages tax reduces consumption of sugar-sweetened beverages; and (c) the degree to which reduced consumption of sugar-sweetened beverages reduces the prevalence and disease burden of diabetes mellitus type 2 in a single country and hence proportionally the global disease burden – is ultimately 0.00004.
Meanwhile, on the costing side, we have to be concerned with both the cost of advocacy (for a nonprofit working on the matter) and the cost of regulation (for the government).
To estimate the cost of advocacy, I consider two reference classes – an existing non-EA charity and a hypothetical EA charity. For the former, I look at Action on Sugar, which appears to be a reasonably successful NGO working on sugar reduction in the UK, and take its annual income in FY 2020-21 in GBP and multiply it by the average 2020 GBP-USD exchange rate. Moreover, I assess that around 1 year of operations is a reasonable timeframe for the charity to conduct preparatory activities (e.g. prepare supporting research reports on the health and economics benefits of the tax, conducting public polling to show public support, construct a coalition of NGOs and advocates, convince past and present politicians to be legislative champions) and to actually lobby the sitting government – and hence succeed (in which case it can pivot to a different country) or judge that policymakers are just not receptive and that its efforts have failed (in which case it can pivot or else shutdown). For the hypothetical EA charity, I assume the EA charity will be in the style of Charity Entrepreneurship incubatees – 2 co-founders, and funding of around USD 50,000 per person per annum. Here, we again assume 1 year of operations, for reasons aforementioned. And aggregating both reference classes, we use equal weightage – while the actual non-EA organization’s financial track record gives a much better indication of baseline expenditure requirements in the cause area, an EA or EA-identified organization will almost certainly be more cost-effective. This yields around USD 123,000 in advocacy costs, within a single country.
As for the cost of regulation – taking Long et al’s estimate of the cost of implementing a sugar tax in the United States as about USD 430 million for 10 years, we can then (a) divide by US population to obtain cost per capita; (b) multiply by the size of the average country (in which the intervention will occur) to yield the baseline regulatory cost in the intervention country; (c) discount for the probability that advocacy succeeds (and that the costs are incurred at all); and (e) discount for the lower counterfactual cost of the average government’s spending relative to EA funding going to top GiveWell charities or similar, as a function of diminishing marginal utility of income, higher average global GDP per capita relative to the poor country average, and the top GiveWell health charity’s cost-effectiveness relative to just giving cash to poor people, correcting for GiveWell’s undervaluation of life vs income. This yields a lifetime cost of regulation of around USD 85,000.
Put together, the total cost of the intervention will be around USD 208,000.
Consequently, the proportion of the problem solved per additional USD 100,000 spent is around 0.00002.
Marginal Expected Value of Policy Advocacy for Sugar-Sweetened Beverages Tax to Eliminate Diabetes Mellitus Type 2
All in all, the marginal expected value of policy advocacy for a sugar-sweetened beverages tax to eliminate diabetes mellitus type 2 is 634,277 DALYs per USD 100,000 spent, making this around 1000x as cost-effective as a GiveWell top charity.
Interesting post. I haven’t conducted the depth research to verify most of the figures, but I do think the idea that you have a 55% chance of success with a $208k 1 year advocacy campaign pretty implausible and suspect there’s something dubious going on with the method of estimating P(success) here.
I think an appropriate fact to incorporate which I did not see would be “actual costs of lobbying in the US” and “frequency of novel regulations passing” on which I presume there is quite a bit of data available.
The probability of advocacy success is a fairly critical variable, and I agree that the estimate provided could well be too optimistic. It really depends on (a) what reference class you take, and (b) how you weigh it against subjective inside view estimates. For example, my estimate of (b) as informed by working in the public sector/politics is fairly low, but if you look do a case study of when sugar taxes were actually advocated (and implemented or not), it’s really impressive (~90%), and the real challenge becomes adjusting for selection bias—both with respect to it being tried (in countries where political conditions were more favourable in the first place), and successful attempts being noted in the news (while failed ones die inside the government, unreported).
On the one hand, sugary drinks taxes really aren’t that uncommon, so it’s not that surprising that it wouldn’t be too difficult to advocate for (relative to something like sodium tax advocacy, which is probably a quarter as tractable). I would also caution against using US lobbying costs, since that isn’t necessarily representative (i.e. the modal campaign wouldn’t be hiring K-street lobbyists in the US, so much as an NGO talking to low and middle-income countries governments, which tend to defer to NGOs than western governments do).
In general, I hope to get a better sense of this by talking to experts (even while noting that the public health experts may well also be overoptimistic due to halo effects/wishful thinking!)
My general sense is that a lot of policy advocacy projects look really well in terms of CEAs as the scope tends to be high but few properly discount for likelihood of success or indeed, as you suggest, actual lobbying costs over time and relevancy, frequency, take up of regulations.
It seems like there is a general trend for public health interventions to look insanely cost-effective wrt. to daly’s per dollar. I’d be curious to see a more detailed meta-review of this type of intervention, as they all are likely to share the same pitfalls if there are any.
Government policies I think are harmful tend have some easily measurable upside, at the cost of a much larger but difficult to measure downside. (ie. by making it more costly for firms to fire, employers respond by being much more cautious and discriminatory in their hiring practices)
These public health interventions seem to follow the pattern of an easily quantifiable upside and a difficult to measure downside, which leads me to worry if we are getting mugged by what is measurable.
Cheers, Mathias.
In the context of public health taxation policies, the main hard-to-quantify downside would be freedom of choice. As you know, I’m sympathetic to the concern, and we worked together before on quantifying freedom lost from tobacco taxation while in the CE incubation programme (and I think it’s linked to somewhere in CEARCH’s evaluative framework, as an example of how one might do it).
Having explicitly run the numbers in the context of tobacco/alcohol taxation + tightening road traffic standards in Singapore , my sense is that (a) for the range of realistic moral weights, very high taxes are still justified (i.e. the health benefits > the freedom of choice considerations), but outweigh bans are not (i.e. once you move from very high taxes to outright bans, the marginal health benefits decline below the value of freedom of choice).
Will definitely look to do this (i.e. incorporate freedom of choice considerations) at intermediate stages, and if we reach the deep stage I can definitely see the value of funding actual moral weights research on the matter (i.e. surveys to see how much people value being able to drink sweet drinks etc).
Hi Joel, I skimmed your report really quickly (sorry) but suspect that you did not account for soda taxes being eventually passed anyway. So the modeled impact of any intervention shouldn’t be going to 2100 or beyond but out only a few years (I’d think <10 years) when soda taxes would eventually be passed without any active intervention. You are trying to measure the impact of a counterfactual donated dollar in the presence of all the forces already at play that are pushing for soda taxes (how some countries already have them). This makes for a more plausible model, and I believe is how LEEP or OpenPhil model policy intervention cost-effectiveness (I could be wrong though).
Hi Wayne,
You’re right! I’m currently working on the intermediate report for diabetes, and one factor we’re looking at that the shallow report did not cover is the speeding up effect, which we model by looking at the base rate from past data (i.e. country-years in which passage occurred, divided by total country-years). This definitely cuts into the headline cost-effectiveness estimate.
On a related note, one issue, I think, is whether we think of tax policy success as counterfactually mutually exclusive, or as additive. (A) For the former, as you say, the idea is that the tax would have occurred anyway. (B) For the latter, the idea is that the tax an EA or EA-funded advocacy organization pushes shifts upwards the tax over time curve (i.e. what the tax rate is over time; presumably this slopes upwards, as countries get stricter). In short, we’re having a counterfactual effect because the next round of tax increases don’t replace so much as add on to what we’ve achieved, and our actions ensure that the tax rate at any one point in time is systematically higher than it otherwise would have been.
I think reality is a mix between both both viewpoints (A) & (B) - success means draining the political capital to do more in the short to medium term, but you’re probably also ensuring that the tax rate is systematically higher going forward. In practice, I tend to model using (A), just to be conservative
Thanks for your response, Joel!
Stepping back, CEARCH’s goal is to identify cause areas that have been missed by EA. But to be successful, you need to compare apples with apples. If you’re benchmarking everything to GiveWell Top Charities, readers expect your methodology to be broadly consistent with GiveWell’s and their conservative approach (and for other cause areas, consistent with best-practice EA approaches). The cause areas that are standing out for CEARCH should be because they are actually more cost-effective, not because you’re using a more lax measuring method.
Coming back to the soda tax intervention, CEARCH’s finding that it’s 1000x GiveWell Top Charities raised a red flag for me so it seemed that you must somehow be measuring things differently. LEEP seems comparable since they also work to pass laws that limit a bad thing (lead paint), but they’re at most ~10x GiveWell Top Charities. So where’s the additional 100x coming from? I was skeptical that soda taxes would have greater scale, tractability, or neglectedness since LEEP already scores insanely high on each of these dimensions.
So I hope CEARCH can ensure cost-effectiveness comparability and if you’re picking up giant differences w/ existing EA interventions, you should be able to explain the main drivers of these differences (and it shouldn’t be because you’re using a different yardstick). Thanks!
Just to clarify, one should definitely expect cost-effectiveness estimates to drop as you put more time into them, and I don’t expect this cause area to be literally 1000x GiveWell. Headline cost-effectiveness always drops, from past experience, and it’s just optimizer’s curse where over (or under) performance comes partly from the cause area being genuinely better (or worse) but also partly from random error that you fix at deeper research stages. To be honest, I’ve come around to the view that publishing shallow reports—which are really just meant for internal prioritization—probably isn’t useful, insofar as it can be misleading.
As an example of how we more aggressive discount at deeper research stages, consider our intermediate hypertension report—there was a fairly large drop from around 300x to 80x GiveWell, driven by (among other things): (a) taking into accounting speeding up effects, (b) downgrading confidence in advocacy success rates, (c) updating for more conservative costing, and (d) doing GiveWell style epistemological discounts (e.g. taking into account a conservative null hypothesis prior, or discounting for publication bias/endogeneity/selection bias etc.)
As for what our priors should be with respect to whether a cause can really be 100x GiveWell—I would say there’s a reasonable case for this, if: (a) One targets NCDs and other diseases that grow with economic growth (instead of being solved by countries getting richer, and improving sanitation/nutrition/healthcare systems etc). (b) There are good policy interventions available, because it really does matter that: (i) a government has enormous scale/impact; (ii) their spending is (counterfactually) relative to EA money that would have gone to AMF and the like; and (iii) policy tends to be sticky, and so the impact lasts in a way that distributing malaria nets or treating depression may not.
Given that this only involved one week of research, this is a strong starting point for more in-depth work—which, as you have indicated, will likely result in a less cost-effective estimate.
One quick comment:
DALYs are bad. They measure the burden of disease. They represent years lost to mortality, and years lost to poor health. We want to avert DALYs. In a couple of places this post talks about averting DALYs, but it mostly seems to talk as if DALYs are a good thing being lost to diabetes.
Similarly your evaluative framework states that ‘A full, healthy life is equivalent to 61.09 DALYs’. This seems to be conflating DALYs with QALYs; QALYs are good—we want more QALYs.
It’s loose talk, I agree—what’s we’re talking about really are health-adjusted life years (HALE), the loss of which are accounted for via DALYs. Probably should fix that going forward, but in general I find that people intuitively get the idea of disability-adjusted life years as a positive thing (for whatever reason)