More of a methodological question, but as I know you are doing quite rigorous analysis in idea selection I would be very curious to learn how you model the multiplier from policy advocacy vis-à-vis more direct forms of work.
For policy obviously the overall scale of the number of people/animals you can help is much bigger, but then we have to discount this based on the expected enforcement rate and also by the fact that we expect the overall probability of success of policy advocacy to be lower on average.
For the expected enforcement rate we look at existing enforcement rates of the policy in similar/neighbouring countries (if applicable) and the existing enforcement rates of similar policies (eg. when estimating the enforcement rate of seatbelt legislation we may look at the enforcement rates of speed limit legislation by looking at that % of people stick to the speed limit). This can be easier or more difficult to do depending on the intervention and the availability of information. In cases where there is less relevant information we might use proxies such as the World Justice Project’s Rule of Law Index.
For the probability of success we usually try to do a case study analysis of advocacy efforts by other organisations in other countries and take the average success rate of these campaigns, with a note that we are more likely to find information on successful campaigns so this might be somewhat of an overestimate on the overall probability of success.
More of a methodological question, but as I know you are doing quite rigorous analysis in idea selection I would be very curious to learn how you model the multiplier from policy advocacy vis-à-vis more direct forms of work.
Hi—sorry for the belated response!
For policy obviously the overall scale of the number of people/animals you can help is much bigger, but then we have to discount this based on the expected enforcement rate and also by the fact that we expect the overall probability of success of policy advocacy to be lower on average.
For the expected enforcement rate we look at existing enforcement rates of the policy in similar/neighbouring countries (if applicable) and the existing enforcement rates of similar policies (eg. when estimating the enforcement rate of seatbelt legislation we may look at the enforcement rates of speed limit legislation by looking at that % of people stick to the speed limit). This can be easier or more difficult to do depending on the intervention and the availability of information. In cases where there is less relevant information we might use proxies such as the World Justice Project’s Rule of Law Index.
For the probability of success we usually try to do a case study analysis of advocacy efforts by other organisations in other countries and take the average success rate of these campaigns, with a note that we are more likely to find information on successful campaigns so this might be somewhat of an overestimate on the overall probability of success.