I am not sure how relevant the base rate for any popular initiative is.
Have there been any polls on this issue? Have you thought about carrying out polls? Even asking a relatively small number of people could give you a sense of whether this is viable.
Also, do you have a sense how much larger your chance of success would be if you asked for less than 1%? Perhaps you could run several polls asking for different proportions of the city’s budget.
It happens regularly that ballot initiatives are accepted (or rejected) by surprise, despite surveys and media predicting otherwise, so the base rate should be at least somewhat informative. But I agree that the outside view should be adjusted using all the data points we have. Here’s a brief overview of our thinking:
Zurich is one of the most leftist cities in Switzerland, as can be seen from the strong leftist majority in its parliament and government (IIRC the 2nd most leftist out of the ~12 biggest cities according to this measure – and cities also tend to be more leftist than rural areas).
However, Zurich’s budget already seems to be under a lot of pressure, and tax raises are generally unpopular. Even though Zurich’s tax rate is currently rather low compared to Switzerland’s other big cities, I expect this to be a major problem for the initiative.
When collecting signatures for the initiative, our experience was that many people seemed to be at least somewhat skeptical. We don’t have detailed statistics (would be hard to implement with volunteers), but a bit more than half of all people who stopped and talked to us went on to sign the initiative. This is not that good: From experience we can tell that for unsuccessful non-EA initiatives, more people tended to be willing to sign.
Many ballot initiatives that succeeded were rather populist (against immigrants, etc.). This suggests a lower success probability for our initiative. But this seems to apply mostly on the national level, and not so much on the city level.
In general, foreign aid doesn’t seem to be a favorite topic of most citizens, and many seem to believe that it’s ineffective. It’s not clear yet whether our effectiveness requirements will be sufficient to dispel those doubts (moral psychology suggests it probably won’t).
Getting representative samples for polls is difficult, and suboptimal polls provide little additional insight beyond the above data points and considerations, so we decided against this.
Additional evidence will be gathered by gaining official support from charities, politicians and parties (or failing to do so – the latter would be the bigger update). Media reporting will also be informative.
Overall, I think we shouldn’t update that much from the base rate of 11%. But maybe something in the area of 8% would be more accurate than 11%.
We’ve considered going for lower budget percentages, e.g. just 0.7% as in Geneva’s case, or even lower. We think this would have relatively little influence on the success probability: Relative to the current development budget of CHF 3 million (€3 million), the increase will always be significant, and in absolute terms, the amount will always be pretty small, and voters are unlikely to be quantitative thinkers anyway. :-) “1%” sounds catchy and might make it more salient that the amount will be small overall, so we went with that.
Note that much (more than half?) of the initiative’s impact doesn’t depend on whether the vote passes: EA movement building, influence on policy-making, putting evidence-based methods on charities’ radar, improved reputation for the EA movement because it’s not seen as “unpolitical” anymore, etc. are all benefits that will occur in any case.
EDIT: Thanks for asking this question! This seems to be a crucial point that was left unanswered in the above post.
Thank you! That is a very informative answer. A couple of points.
1) My point was that it is not clear why “Swiss ballot initiatives” is the relevant reference class to use for the base rate, since many of these ballot initiatives would be radically different from yours. In my view, it doesn’t give much information at all—certainly less than, e.g. knowing that most Swiss want to increase aid.
2) I would consider talking to people who know politics well how they would proceed with this, and if they would, e.g. use polls. I know that political parties do make (secret) polls to inform what proposals to make and how to frame them to quite a large extent.
3) I agree that the indirect effects in terms of movement building, etc, are likely to be significant. However, I’m not sure of the claim that “these benefits do not depend heavily on the actual outcome of the referendum”. On the contrary, it seems that if you actually win this, the marketing effects are likely to be larger, and you are more likely to be able to win other similar referenda (cf the bandwagon effect ). So I would put significant effort into increasing the chances of winning—and consider reducing the proportion of the budget going to effective charities, if you find out that that increases your chances of winning significantly.
1) I think the idea was to start with the outside view, and then gradually adjust based on additional evidence. I think this is best practice for such estimates. But of course you’re right that most initiatives tend to be quite different from our proposal.
2) Hm, I’m skeptical of this point, at least in the case of such a small initiative. Most popular initiatives seem to be launched without such polling (but there will be exceptions on the national level). The vote outcome will not only depend on individual opinion, but also a lot on the official recommendations by parliament and government (and the media). Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
3) I agree that a positive outcome would give the movement building / public debate effects a significant boost, but I think even if the initiative is rejected, the result will still be very good. ~75% of all media reporting and public debate will happen before the vote result is out: There will be 2 years of time to report on the initiative with multiple salient milestones (handing in the initiative, official statement by government and parliament, the voting day getting closer, etc.), but only 1-2 weeks after the vote. And if the initiative is rejected, this won’t be deemed a “defeat” (at least not as long as >25% of voters approve, which seems decently likely).
I agree with Stefan here, as someone who has worked on political campaigns and marketing.
I find it very difficult to have much information without polling.
Most popular initiatives seem to be launched without such polling (but there will be exceptions on the national level).
Why should this matter? That ballots succeed without measuring their potential and optimizing against it doesn’t mean anything.
The vote outcome will not only depend on individual opinion, but also a lot on the official recommendations by parliament and government (and the media).
Of course. Having polls can allow you to:
Help influence parliament and the government by polling (and releasing the polls if favorable)
Prepare for the change in public opinion / likelihood of success based on the recommendations that come out
Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
How so? As a charity worker and former political staffer I think I’d have minimal information on a ballot initiative for something like this that I hadn’t done polling on. Why would their expertise be helpful? What information do they have that you don’t? Campaigners would only know about issues on which they’ve researched voting behavior.
at much lower cost.
This seems unlikely. Door-to-door polling could be done by one person with a car/public transit in one day. Calling could make this even easier/quicker. Contact randomly selected, dispersed households, give them the same info they’ll see on the ballot, and ask how they’d vote. Your sample size can be quite small and still be informative.
This is very valuable input, thanks! In particular, I might have overestimated the cost of such polls. We’ll definitely look into this further and will strongly consider doing such polls.
As a result of this thread, we’re also in the process of obtaining data from a votematch tool. It’s not representative but contains demographic data, so we will be able to extrapolate.
Why should this matter? That ballots succeed without measuring their potential and optimizing against it doesn’t mean anything.
It does mean something, but I’m now 1) less confident in the belief that people don’t do such polling, and 2) think that we have more reason to update away from that prior (how well-informed is it, really?).
Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
How so? As a charity worker and former political staffer I think I’d have minimal information on a ballot initiative for something like this that I hadn’t done polling on. Why would their expertise be helpful? What information do they have that you don’t? Campaigners would only know about issues on which they’ve researched voting behavior.
I think polls are mostly helpful for strategic decisions such as how many resources to invest into the vote campaign. For refining our communication strategy, observing the media coverage and updating our strategy based on that will be more useful. Politicians also have intuitions for how people will react to various points, e.g. for the kinds of arguments opponents will bring up, or whether a particular party will support the initiative. They might be able to give us recommendations for how to make sure the parliament recommends accepting the initiative. This is especially true in the frequently-debated question of foreign aid. It’s not clear what a poll result like “45% plan to vote YES” means for the communication strategy.
I don’t think the last point is decisive though, we should simply do both.
I am not sure how relevant the base rate for any popular initiative is.
Have there been any polls on this issue? Have you thought about carrying out polls? Even asking a relatively small number of people could give you a sense of whether this is viable.
Also, do you have a sense how much larger your chance of success would be if you asked for less than 1%? Perhaps you could run several polls asking for different proportions of the city’s budget.
It happens regularly that ballot initiatives are accepted (or rejected) by surprise, despite surveys and media predicting otherwise, so the base rate should be at least somewhat informative. But I agree that the outside view should be adjusted using all the data points we have. Here’s a brief overview of our thinking:
According to representative government-funded surveys, 68% of the Swiss population want to increase aid, and 83% want it to remain unchanged or to increase it whilst greatly overestimating the current amount of aid given. Note that this is on the national level.
Zurich is one of the most leftist cities in Switzerland, as can be seen from the strong leftist majority in its parliament and government (IIRC the 2nd most leftist out of the ~12 biggest cities according to this measure – and cities also tend to be more leftist than rural areas).
The canton of Geneva already has a similar law and donates 0.7% of its budget to development charities, so it seems like this is something progressive cities should be able to get behind.
However, Zurich’s budget already seems to be under a lot of pressure, and tax raises are generally unpopular. Even though Zurich’s tax rate is currently rather low compared to Switzerland’s other big cities, I expect this to be a major problem for the initiative.
When collecting signatures for the initiative, our experience was that many people seemed to be at least somewhat skeptical. We don’t have detailed statistics (would be hard to implement with volunteers), but a bit more than half of all people who stopped and talked to us went on to sign the initiative. This is not that good: From experience we can tell that for unsuccessful non-EA initiatives, more people tended to be willing to sign.
Many ballot initiatives that succeeded were rather populist (against immigrants, etc.). This suggests a lower success probability for our initiative. But this seems to apply mostly on the national level, and not so much on the city level.
In general, foreign aid doesn’t seem to be a favorite topic of most citizens, and many seem to believe that it’s ineffective. It’s not clear yet whether our effectiveness requirements will be sufficient to dispel those doubts (moral psychology suggests it probably won’t).
Getting representative samples for polls is difficult, and suboptimal polls provide little additional insight beyond the above data points and considerations, so we decided against this.
Additional evidence will be gathered by gaining official support from charities, politicians and parties (or failing to do so – the latter would be the bigger update). Media reporting will also be informative.
Overall, I think we shouldn’t update that much from the base rate of 11%. But maybe something in the area of 8% would be more accurate than 11%.
We’ve considered going for lower budget percentages, e.g. just 0.7% as in Geneva’s case, or even lower. We think this would have relatively little influence on the success probability: Relative to the current development budget of CHF 3 million (€3 million), the increase will always be significant, and in absolute terms, the amount will always be pretty small, and voters are unlikely to be quantitative thinkers anyway. :-) “1%” sounds catchy and might make it more salient that the amount will be small overall, so we went with that.
Note that much (more than half?) of the initiative’s impact doesn’t depend on whether the vote passes: EA movement building, influence on policy-making, putting evidence-based methods on charities’ radar, improved reputation for the EA movement because it’s not seen as “unpolitical” anymore, etc. are all benefits that will occur in any case.
EDIT: Thanks for asking this question! This seems to be a crucial point that was left unanswered in the above post.
Thank you! That is a very informative answer. A couple of points.
1) My point was that it is not clear why “Swiss ballot initiatives” is the relevant reference class to use for the base rate, since many of these ballot initiatives would be radically different from yours. In my view, it doesn’t give much information at all—certainly less than, e.g. knowing that most Swiss want to increase aid.
2) I would consider talking to people who know politics well how they would proceed with this, and if they would, e.g. use polls. I know that political parties do make (secret) polls to inform what proposals to make and how to frame them to quite a large extent.
3) I agree that the indirect effects in terms of movement building, etc, are likely to be significant. However, I’m not sure of the claim that “these benefits do not depend heavily on the actual outcome of the referendum”. On the contrary, it seems that if you actually win this, the marketing effects are likely to be larger, and you are more likely to be able to win other similar referenda (cf the bandwagon effect ). So I would put significant effort into increasing the chances of winning—and consider reducing the proportion of the budget going to effective charities, if you find out that that increases your chances of winning significantly.
1) I think the idea was to start with the outside view, and then gradually adjust based on additional evidence. I think this is best practice for such estimates. But of course you’re right that most initiatives tend to be quite different from our proposal.
2) Hm, I’m skeptical of this point, at least in the case of such a small initiative. Most popular initiatives seem to be launched without such polling (but there will be exceptions on the national level). The vote outcome will not only depend on individual opinion, but also a lot on the official recommendations by parliament and government (and the media). Direct conversations with politicians and charities seem similarly helpful for informing framings/communications at much lower cost.
3) I agree that a positive outcome would give the movement building / public debate effects a significant boost, but I think even if the initiative is rejected, the result will still be very good. ~75% of all media reporting and public debate will happen before the vote result is out: There will be 2 years of time to report on the initiative with multiple salient milestones (handing in the initiative, official statement by government and parliament, the voting day getting closer, etc.), but only 1-2 weeks after the vote. And if the initiative is rejected, this won’t be deemed a “defeat” (at least not as long as >25% of voters approve, which seems decently likely).
I agree with Stefan here, as someone who has worked on political campaigns and marketing.
I find it very difficult to have much information without polling.
Why should this matter? That ballots succeed without measuring their potential and optimizing against it doesn’t mean anything.
Of course. Having polls can allow you to:
Help influence parliament and the government by polling (and releasing the polls if favorable)
Prepare for the change in public opinion / likelihood of success based on the recommendations that come out
How so? As a charity worker and former political staffer I think I’d have minimal information on a ballot initiative for something like this that I hadn’t done polling on. Why would their expertise be helpful? What information do they have that you don’t? Campaigners would only know about issues on which they’ve researched voting behavior.
This seems unlikely. Door-to-door polling could be done by one person with a car/public transit in one day. Calling could make this even easier/quicker. Contact randomly selected, dispersed households, give them the same info they’ll see on the ballot, and ask how they’d vote. Your sample size can be quite small and still be informative.
This is very valuable input, thanks! In particular, I might have overestimated the cost of such polls. We’ll definitely look into this further and will strongly consider doing such polls.
As a result of this thread, we’re also in the process of obtaining data from a votematch tool. It’s not representative but contains demographic data, so we will be able to extrapolate.
It does mean something, but I’m now 1) less confident in the belief that people don’t do such polling, and 2) think that we have more reason to update away from that prior (how well-informed is it, really?).
I think polls are mostly helpful for strategic decisions such as how many resources to invest into the vote campaign. For refining our communication strategy, observing the media coverage and updating our strategy based on that will be more useful. Politicians also have intuitions for how people will react to various points, e.g. for the kinds of arguments opponents will bring up, or whether a particular party will support the initiative. They might be able to give us recommendations for how to make sure the parliament recommends accepting the initiative. This is especially true in the frequently-debated question of foreign aid. It’s not clear what a poll result like “45% plan to vote YES” means for the communication strategy.
I don’t think the last point is decisive though, we should simply do both.