Thank you! That is a very informative answer. A couple of points.
1) My point was that it is not clear why “Swiss ballot initiatives” is the relevant reference class to use for the base rate, since many of these ballot initiatives would be radically different from yours. In my view, it doesn’t give much information at all—certainly less than, e.g. knowing that most Swiss want to increase aid.
2) I would consider talking to people who know politics well how they would proceed with this, and if they would, e.g. use polls. I know that political parties do make (secret) polls to inform what proposals to make and how to frame them to quite a large extent.
3) I agree that the indirect effects in terms of movement building, etc, are likely to be significant. However, I’m not sure of the claim that “these benefits do not depend heavily on the actual outcome of the referendum”. On the contrary, it seems that if you actually win this, the marketing effects are likely to be larger, and you are more likely to be able to win other similar referenda (cf the bandwagon effect ). So I would put significant effort into increasing the chances of winning—and consider reducing the proportion of the budget going to effective charities, if you find out that that increases your chances of winning significantly.
1) I think the idea was to start with the outside view, and then gradually adjust based on additional evidence. I think this is best practice for such estimates. But of course you’re right that most initiatives tend to be quite different from our proposal.
2) Hm, I’m skeptical of this point, at least in the case of such a small initiative. Most popular initiatives seem to be launched without such polling (but there will be exceptions on the national level). The vote outcome will not only depend on individual opinion, but also a lot on the official recommendations by parliament and government (and the media). Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
3) I agree that a positive outcome would give the movement building / public debate effects a significant boost, but I think even if the initiative is rejected, the result will still be very good. ~75% of all media reporting and public debate will happen before the vote result is out: There will be 2 years of time to report on the initiative with multiple salient milestones (handing in the initiative, official statement by government and parliament, the voting day getting closer, etc.), but only 1-2 weeks after the vote. And if the initiative is rejected, this won’t be deemed a “defeat” (at least not as long as >25% of voters approve, which seems decently likely).
I agree with Stefan here, as someone who has worked on political campaigns and marketing.
I find it very difficult to have much information without polling.
Most popular initiatives seem to be launched without such polling (but there will be exceptions on the national level).
Why should this matter? That ballots succeed without measuring their potential and optimizing against it doesn’t mean anything.
The vote outcome will not only depend on individual opinion, but also a lot on the official recommendations by parliament and government (and the media).
Of course. Having polls can allow you to:
Help influence parliament and the government by polling (and releasing the polls if favorable)
Prepare for the change in public opinion / likelihood of success based on the recommendations that come out
Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
How so? As a charity worker and former political staffer I think I’d have minimal information on a ballot initiative for something like this that I hadn’t done polling on. Why would their expertise be helpful? What information do they have that you don’t? Campaigners would only know about issues on which they’ve researched voting behavior.
at much lower cost.
This seems unlikely. Door-to-door polling could be done by one person with a car/public transit in one day. Calling could make this even easier/quicker. Contact randomly selected, dispersed households, give them the same info they’ll see on the ballot, and ask how they’d vote. Your sample size can be quite small and still be informative.
This is very valuable input, thanks! In particular, I might have overestimated the cost of such polls. We’ll definitely look into this further and will strongly consider doing such polls.
As a result of this thread, we’re also in the process of obtaining data from a votematch tool. It’s not representative but contains demographic data, so we will be able to extrapolate.
Why should this matter? That ballots succeed without measuring their potential and optimizing against it doesn’t mean anything.
It does mean something, but I’m now 1) less confident in the belief that people don’t do such polling, and 2) think that we have more reason to update away from that prior (how well-informed is it, really?).
Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
How so? As a charity worker and former political staffer I think I’d have minimal information on a ballot initiative for something like this that I hadn’t done polling on. Why would their expertise be helpful? What information do they have that you don’t? Campaigners would only know about issues on which they’ve researched voting behavior.
I think polls are mostly helpful for strategic decisions such as how many resources to invest into the vote campaign. For refining our communication strategy, observing the media coverage and updating our strategy based on that will be more useful. Politicians also have intuitions for how people will react to various points, e.g. for the kinds of arguments opponents will bring up, or whether a particular party will support the initiative. They might be able to give us recommendations for how to make sure the parliament recommends accepting the initiative. This is especially true in the frequently-debated question of foreign aid. It’s not clear what a poll result like “45% plan to vote YES” means for the communication strategy.
I don’t think the last point is decisive though, we should simply do both.
Thank you! That is a very informative answer. A couple of points.
1) My point was that it is not clear why “Swiss ballot initiatives” is the relevant reference class to use for the base rate, since many of these ballot initiatives would be radically different from yours. In my view, it doesn’t give much information at all—certainly less than, e.g. knowing that most Swiss want to increase aid.
2) I would consider talking to people who know politics well how they would proceed with this, and if they would, e.g. use polls. I know that political parties do make (secret) polls to inform what proposals to make and how to frame them to quite a large extent.
3) I agree that the indirect effects in terms of movement building, etc, are likely to be significant. However, I’m not sure of the claim that “these benefits do not depend heavily on the actual outcome of the referendum”. On the contrary, it seems that if you actually win this, the marketing effects are likely to be larger, and you are more likely to be able to win other similar referenda (cf the bandwagon effect ). So I would put significant effort into increasing the chances of winning—and consider reducing the proportion of the budget going to effective charities, if you find out that that increases your chances of winning significantly.
1) I think the idea was to start with the outside view, and then gradually adjust based on additional evidence. I think this is best practice for such estimates. But of course you’re right that most initiatives tend to be quite different from our proposal.
2) Hm, I’m skeptical of this point, at least in the case of such a small initiative. Most popular initiatives seem to be launched without such polling (but there will be exceptions on the national level). The vote outcome will not only depend on individual opinion, but also a lot on the official recommendations by parliament and government (and the media). Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
3) I agree that a positive outcome would give the movement building / public debate effects a significant boost, but I think even if the initiative is rejected, the result will still be very good. ~75% of all media reporting and public debate will happen before the vote result is out: There will be 2 years of time to report on the initiative with multiple salient milestones (handing in the initiative, official statement by government and parliament, the voting day getting closer, etc.), but only 1-2 weeks after the vote. And if the initiative is rejected, this won’t be deemed a “defeat” (at least not as long as >25% of voters approve, which seems decently likely).
I agree with Stefan here, as someone who has worked on political campaigns and marketing.
I find it very difficult to have much information without polling.
Why should this matter? That ballots succeed without measuring their potential and optimizing against it doesn’t mean anything.
Of course. Having polls can allow you to:
Help influence parliament and the government by polling (and releasing the polls if favorable)
Prepare for the change in public opinion / likelihood of success based on the recommendations that come out
How so? As a charity worker and former political staffer I think I’d have minimal information on a ballot initiative for something like this that I hadn’t done polling on. Why would their expertise be helpful? What information do they have that you don’t? Campaigners would only know about issues on which they’ve researched voting behavior.
This seems unlikely. Door-to-door polling could be done by one person with a car/public transit in one day. Calling could make this even easier/quicker. Contact randomly selected, dispersed households, give them the same info they’ll see on the ballot, and ask how they’d vote. Your sample size can be quite small and still be informative.
This is very valuable input, thanks! In particular, I might have overestimated the cost of such polls. We’ll definitely look into this further and will strongly consider doing such polls.
As a result of this thread, we’re also in the process of obtaining data from a votematch tool. It’s not representative but contains demographic data, so we will be able to extrapolate.
It does mean something, but I’m now 1) less confident in the belief that people don’t do such polling, and 2) think that we have more reason to update away from that prior (how well-informed is it, really?).
I think polls are mostly helpful for strategic decisions such as how many resources to invest into the vote campaign. For refining our communication strategy, observing the media coverage and updating our strategy based on that will be more useful. Politicians also have intuitions for how people will react to various points, e.g. for the kinds of arguments opponents will bring up, or whether a particular party will support the initiative. They might be able to give us recommendations for how to make sure the parliament recommends accepting the initiative. This is especially true in the frequently-debated question of foreign aid. It’s not clear what a poll result like “45% plan to vote YES” means for the communication strategy.
I don’t think the last point is decisive though, we should simply do both.