Hey; I made some comments on this on the doc, but I thought it was worth bringing them to the main thread and expanding.
First of all, I’m really happy to sea other EAs looking at ballot measures. They’re a potentially very high EV method of passing policy/raising funding. They’re particularly high value per dollar when spending on advertising is limited/nothing since the increased probability of passage from getting a relatively popular measure on the ballot is far more than the increased probability from spending the same amount advertising for it.
Also, am I correct in interpreting that you assume 100% chance of passage in your model conditional on good polling? Polling can help, but ballot measure polling does have a lot of error (in both directions). So even a popular measure in polling is hardly guarantee of passage (http://themonkeycage.org/2011/10/when-can-you-trust-polling-about-ballot-measures/).
Finally, in your EV estimates, you seem to be focus on the individual treatment cost of the intervention, which overwhelms the cost of the ballot measure. I don’t think this is getting at the right question when it comes to running a ballot measure. I believe the gains from the ballot measure should be the estimated sum of the utility gains from people being able to purchase the drugs multiplied by the probability of passage; the costs should be how much it would cost to run the campaign. On the doc, you made the point that Givewell doesn’t include leverage on other funding in their estimates, but when it comes to ballot measures, leverage is exactly what you’re trying to produce, so I think an estimate is important.
I believe the gains from the ballot measure should be the estimated sum of the utility gains from people being able to purchase the drugs multiplied by the probability of passage; the costs should be how much it would cost to run the campaign. On the doc, you made the point that Givewell doesn’t include leverage on other funding in their estimates, but when it comes to ballot measures, leverage is exactly what you’re trying to produce, so I think an estimate is important.
One potential way of thinking about this is that the ballot measure in itself does not accomplish much, it just “unlocks” the ability for people to more cheaply help themselves. This could be modeled as the costs of the ballot measure + the costs of people helping themselves over a stream of X years, put against the benefits of people helping themselves over X years. I would use 5 for X, assuming that a lot can change in 5 years and maybe drug legalization would happen anyway, but I think a higher value for X could also be justified.
This kind of (costs of unlocking + costs of what is unlocked over time) vs. benefits of what is unlocked over time is also how I model the cost-benefit of developing a new medicine (like a vaccine), since the medicine is useless unless it is actually given to people, which costs additional money.
That seems similar to Milan_Griffes’ approach. However, when we’re comparing ballot measures to other opportunities, I think the relevant cost to EA would be the cost to launch the campaign. That’s what EAs would actually be spending money on and what could be spent on other interventions.
We don’t have to assume away the additional costs of getting the medicine, but that can be factored into the benefit (ie. the net benefit is the gains they would get from the medicine—the gains they lose from giving up the funds to purchase the drugs)
Cool; had missed that row. Yeah, if it polls, 70% the chance of passage might be close to 80%. Conditional upon that level of support, your estimate seems reasonable to me (assuming the ballot summary language would not be far more complex than the polled language).
Yeah, I agree that it being an effective treatment is a necessary precursor to it being a good ballot law to pass by ballot initiative and part of the EV calculation for spending money on the ballot measure itself.
Hello Miles and thanks for all this, good to see it’s getting discussed.
As you probably saw, I produced a cost-effectiveness model of drug policy campaigning in the final post of my (rather long) series on the subject. In that, I suggest it’s plausible drug policy reform, again just by allowing the use of psychedelics to treat mental health, could be in the range of $166/HALYs (Happiness adjusted life years), which would make it some 300 times more cost effective than you suggest.
It would be really helpful if you could say where and why you disagree with my model, given that, if you think drug policy reform is a promising intervention, my analysis suggest it’s much more promising that yours does! For simplicity, let’s assume HALYs and DALYs are just different types of apples, then I’d want to know why you think the structure of my model is wrong.
Your model assumes that everyone in the UK who might benefit from treatment would seek treatment.
Your model assumes that everyone who receives treatment would benefit from treatment.
FWIW, in my model I don’t assume either of those things. I assume an average counterfactual effect (counter to no rescheduling) of 0.1 HALYs for the 10m in the UK affected by depression or anxiety, not that they all get treatment or everyone benefits from the treatment (to be fair, I specify this in an edit of 14/08/2017 and you might have read it beforehand).
I don’t mention replicability, but then I am assuming the rescheduling only brings a slight improvement (in the latter, more optimistic estimate I discuss whether this might be higher than 0.1 HALYs). I also mention the confusing possibility that treating some people with psychedelics might free up health care resources for other treatments.
I don’t include costs of treatment, as I’m assuming this is an EA-funded campaign where our job, and what the money goes to, is changing the law and then allowing normal health care distribution to occur in the new scenario (i.e. in the US = insurer pays, in UK = govt pays).
Hence, looking at your model, I’m not sure why you include the costs of treatment, unless you think EA funders are going to be paying for those too. Even if you do think this, we should really want to have two seperate models, one for “cost of changing the law, assuming health practices then change aoccrdingly” and another for “cost effectiveness to EA funders to provide psychedelic therapy if it’s available”. As an aside, your model is really thorough, and I’m grateful to you for having put it together, good stuff!
This may also sound picky, but what we want to know (1) what is the most suitable model is for any give intervention, so if we’re disagreeing with each other, we want to know why we’re disagreeing, not just that we’re disagreeing. Hence I was asking where and why you disagreed with my model.
You might reply your model is separate (campaign lobbying in UK vs ballot iniative and treatment funding in the US(?)) but, we also want to know (2) whether some new intervention is more cost-effective than all other current interventions an EA could fund (on one or more moral theory). If it’s not more cost-effective then, all things considered, it would be bad to fund it. That’s why I also asked if, and why, you think your drug policy reform strategy is more cost-effective than the one I proposed.
As it stands, we are perhaps comparing apples and oranges: you seem to have bundled treatment in with a policy change, and assumed this policy change will almost certainly occur depending on the polling numbers. I’ve just looked at policy change and estimated how much we could spend on it to change public/policy opinion and it still be more effective than AMF, assuming AMF is the current most cost-effective intervention. Hence we may need to get on the same page on this first.
I think we’re talking past each other on exactly which counterfactuals we have in mind.
There seem to be a couple of bits:
Counterfactual A is: how much better magic mushrooms (MM) is than conventional treatment for people who undergo conventional treatment. This should be multiplied by the number of years before the rescheduling would otherwise have occured.
An additional counterfactual B is: assuming counterfactual A happens and is cheaper than current treatment, that should free up resources for treating the mentally ill who didn’t get MM treatment. Should also use the same timescale as A.
I’m now lost on exactly what you’re modelling. My model lumps A and B together and assumed a 0.1 HALY increase average across those with depression or anxiety in the UK.
Moving on
My understanding is that most public health cost-effectiveness modeling includes all costs of treatment, regardless of who’s paying.
I think this is the wrong way to think about it from an EA perspective. Imagine I’m a rich funder. I will pay for the ballot iniative, but I won’t be pay for the health treatments. hence when i do my cost-effectiveness analysis for the ballot, my cost is the ballot expenditure only, the benefit is the counterfactual happiness increase that rules from the new treatments occurring, presuming normal health stuff happens, i.e. doctors upgrade to the new treatments.
As the funder who wants to do the most good, I’m comparing the cost effectivess of this ballot to other things I could fund, like bednets. I’m not funding the treatments themselves, so that’s misleading. If I were a government, maybe I’d think about it the way you propose, but then governments dont fund ballot initiative, so that would also be misleading.
It could be the case that, one psychedlics are used in treatment, I could then, as a rich funder, think about paying for those vs paying for bednets. As I said before, that is also an important question. hence we want to split these apart for greater accuracy.
Hey; I made some comments on this on the doc, but I thought it was worth bringing them to the main thread and expanding.
First of all, I’m really happy to sea other EAs looking at ballot measures. They’re a potentially very high EV method of passing policy/raising funding. They’re particularly high value per dollar when spending on advertising is limited/nothing since the increased probability of passage from getting a relatively popular measure on the ballot is far more than the increased probability from spending the same amount advertising for it.
Also, am I correct in interpreting that you assume 100% chance of passage in your model conditional on good polling? Polling can help, but ballot measure polling does have a lot of error (in both directions). So even a popular measure in polling is hardly guarantee of passage (http://themonkeycage.org/2011/10/when-can-you-trust-polling-about-ballot-measures/).
Finally, in your EV estimates, you seem to be focus on the individual treatment cost of the intervention, which overwhelms the cost of the ballot measure. I don’t think this is getting at the right question when it comes to running a ballot measure. I believe the gains from the ballot measure should be the estimated sum of the utility gains from people being able to purchase the drugs multiplied by the probability of passage; the costs should be how much it would cost to run the campaign. On the doc, you made the point that Givewell doesn’t include leverage on other funding in their estimates, but when it comes to ballot measures, leverage is exactly what you’re trying to produce, so I think an estimate is important.
One potential way of thinking about this is that the ballot measure in itself does not accomplish much, it just “unlocks” the ability for people to more cheaply help themselves. This could be modeled as the costs of the ballot measure + the costs of people helping themselves over a stream of X years, put against the benefits of people helping themselves over X years. I would use 5 for X, assuming that a lot can change in 5 years and maybe drug legalization would happen anyway, but I think a higher value for X could also be justified.
This kind of (costs of unlocking + costs of what is unlocked over time) vs. benefits of what is unlocked over time is also how I model the cost-benefit of developing a new medicine (like a vaccine), since the medicine is useless unless it is actually given to people, which costs additional money.
That seems similar to Milan_Griffes’ approach. However, when we’re comparing ballot measures to other opportunities, I think the relevant cost to EA would be the cost to launch the campaign. That’s what EAs would actually be spending money on and what could be spent on other interventions.
We don’t have to assume away the additional costs of getting the medicine, but that can be factored into the benefit (ie. the net benefit is the gains they would get from the medicine—the gains they lose from giving up the funds to purchase the drugs)
Cool; had missed that row. Yeah, if it polls, 70% the chance of passage might be close to 80%. Conditional upon that level of support, your estimate seems reasonable to me (assuming the ballot summary language would not be far more complex than the polled language).
Yeah, I agree that it being an effective treatment is a necessary precursor to it being a good ballot law to pass by ballot initiative and part of the EV calculation for spending money on the ballot measure itself.
Hello Miles and thanks for all this, good to see it’s getting discussed.
As you probably saw, I produced a cost-effectiveness model of drug policy campaigning in the final post of my (rather long) series on the subject. In that, I suggest it’s plausible drug policy reform, again just by allowing the use of psychedelics to treat mental health, could be in the range of $166/HALYs (Happiness adjusted life years), which would make it some 300 times more cost effective than you suggest.
It would be really helpful if you could say where and why you disagree with my model, given that, if you think drug policy reform is a promising intervention, my analysis suggest it’s much more promising that yours does! For simplicity, let’s assume HALYs and DALYs are just different types of apples, then I’d want to know why you think the structure of my model is wrong.
I haven’t engaged closely with your model, but here are some differences that immediately stand out:
Your analysis models the a change that impacts the entire UK.
Your model assumes that everyone in the UK who might benefit from treatment would seek treatment.
Your model assumes that everyone who receives treatment would benefit from treatment.
Your model doesn’t include a replicability adjustment, to discount effect sizes due to the limited amount of evidence.
As far as I can tell, your model doesn’t include costs of treatment, only costs of rescheduling.
Hello Milan!
FWIW, in my model I don’t assume either of those things. I assume an average counterfactual effect (counter to no rescheduling) of 0.1 HALYs for the 10m in the UK affected by depression or anxiety, not that they all get treatment or everyone benefits from the treatment (to be fair, I specify this in an edit of 14/08/2017 and you might have read it beforehand).
I don’t mention replicability, but then I am assuming the rescheduling only brings a slight improvement (in the latter, more optimistic estimate I discuss whether this might be higher than 0.1 HALYs). I also mention the confusing possibility that treating some people with psychedelics might free up health care resources for other treatments.
I don’t include costs of treatment, as I’m assuming this is an EA-funded campaign where our job, and what the money goes to, is changing the law and then allowing normal health care distribution to occur in the new scenario (i.e. in the US = insurer pays, in UK = govt pays).
Hence, looking at your model, I’m not sure why you include the costs of treatment, unless you think EA funders are going to be paying for those too. Even if you do think this, we should really want to have two seperate models, one for “cost of changing the law, assuming health practices then change aoccrdingly” and another for “cost effectiveness to EA funders to provide psychedelic therapy if it’s available”. As an aside, your model is really thorough, and I’m grateful to you for having put it together, good stuff!
This may also sound picky, but what we want to know (1) what is the most suitable model is for any give intervention, so if we’re disagreeing with each other, we want to know why we’re disagreeing, not just that we’re disagreeing. Hence I was asking where and why you disagreed with my model.
You might reply your model is separate (campaign lobbying in UK vs ballot iniative and treatment funding in the US(?)) but, we also want to know (2) whether some new intervention is more cost-effective than all other current interventions an EA could fund (on one or more moral theory). If it’s not more cost-effective then, all things considered, it would be bad to fund it. That’s why I also asked if, and why, you think your drug policy reform strategy is more cost-effective than the one I proposed.
As it stands, we are perhaps comparing apples and oranges: you seem to have bundled treatment in with a policy change, and assumed this policy change will almost certainly occur depending on the polling numbers. I’ve just looked at policy change and estimated how much we could spend on it to change public/policy opinion and it still be more effective than AMF, assuming AMF is the current most cost-effective intervention. Hence we may need to get on the same page on this first.
I think we’re talking past each other on exactly which counterfactuals we have in mind.
There seem to be a couple of bits:
Counterfactual A is: how much better magic mushrooms (MM) is than conventional treatment for people who undergo conventional treatment. This should be multiplied by the number of years before the rescheduling would otherwise have occured.
An additional counterfactual B is: assuming counterfactual A happens and is cheaper than current treatment, that should free up resources for treating the mentally ill who didn’t get MM treatment. Should also use the same timescale as A.
I’m now lost on exactly what you’re modelling. My model lumps A and B together and assumed a 0.1 HALY increase average across those with depression or anxiety in the UK.
Moving on
I think this is the wrong way to think about it from an EA perspective. Imagine I’m a rich funder. I will pay for the ballot iniative, but I won’t be pay for the health treatments. hence when i do my cost-effectiveness analysis for the ballot, my cost is the ballot expenditure only, the benefit is the counterfactual happiness increase that rules from the new treatments occurring, presuming normal health stuff happens, i.e. doctors upgrade to the new treatments.
As the funder who wants to do the most good, I’m comparing the cost effectivess of this ballot to other things I could fund, like bednets. I’m not funding the treatments themselves, so that’s misleading. If I were a government, maybe I’d think about it the way you propose, but then governments dont fund ballot initiative, so that would also be misleading.
It could be the case that, one psychedlics are used in treatment, I could then, as a rich funder, think about paying for those vs paying for bednets. As I said before, that is also an important question. hence we want to split these apart for greater accuracy.