TL;DR: Nobody seems to know what the value of mean years of impact should be, and I don’t see how this uncertainty could be reduced. I think that indirect effects are more important and it would be better to research them.
Good points and I’m happy you brought it up.
Firstly, I know you know this, but Lewis wrote that in his view, “the assumption that these campaigns only accelerated pledges by five years is very conservative.” It makes sense to use a conservative value when doing a point estimate like he did. And I did use a similar value (4 years) in my conservative estimate in Appendix 1. ACE did not really describe their choice for the value so I didn’t pay much attention to it. There’s also Capriati (2018) which assumed that THL’s cage-free and broiler campaigns moved policies forward by only one year (I just added the description. But this assumes that other organizations would have still done everything they did. And even then, I don’t think it is reasonable.
To be honest, I think that nobody has a clue about what value to use here. Hence, everybody uses random conservative values in order for the end result to be more believable, because the estimated cost-effectiveness of campaigns is unbelievably high even with a conservative values. I did that to a degree as well. I asked some people who work on corporate campaigns what value would they use for mean years of impact. They thought that my range was reasonable, but I think they would have said that about many different ranges because it’s difficult to think about. Only one person was able to say what range for mean years of impact would they use without looking at my range, and they said 40 to 100 years. If I weren’t anchored by other estimates and didn’t want to be a bit conservative to be more convincing to skeptics, I think I would have chosen a higher value as well, especially for the upper bound. In general, my impression is that all these “estimates” (including mine) are basically guesses that look more legitimate than they should because they are surrounded by neatly formatted text. We just can’t predict the future.
I also don’t see how to do more research on this topic. Perhaps seeing how long corporate commitments in other domains lasted could give a little bit information on that but only a little bit because situations are so different. If you have any ideas, I’d be interested to hear them.
But I’m uncertain if it’s worthwhile to put more effort into cost-effectiveness estimates of corporate campaigns. I don’t know what decisions depend on whether it’s 7.5-24 years or 9-120 years. I think it would be more decision-relevant to research indirect effects, or in which regions corporate campaigns are more cost-effective, or how corporate campaigns compare with legislative campaigns for welfare reforms.
your model and ACE’s model should be more uncertain about it.
I use a subjective confidence interval of 4 to 36 years with a log-normal distribution, so it’s already very uncertain. Perhaps I should have used a flatter custom distribution, but after experimenting, it seems that it wouldn’t change the answer much.
Additionally, I’d like to see reports like this contain a short display of which input parameters have the largest effect with the result. I think it can be the important information for seeing how robust the result is!
Interesting, I will consider it. Do you know of any report that I could use as an example of how to write such a thing?
TL;DR: Nobody seems to know what the value of mean years of impact should be, and I don’t see how this uncertainty could be reduced. I think that indirect effects are more important and it would be better to research them.
Good points and I’m happy you brought it up.
Firstly, I know you know this, but Lewis wrote that in his view, “the assumption that these campaigns only accelerated pledges by five years is very conservative.” It makes sense to use a conservative value when doing a point estimate like he did. And I did use a similar value (4 years) in my conservative estimate in Appendix 1. ACE did not really describe their choice for the value so I didn’t pay much attention to it. There’s also Capriati (2018) which assumed that THL’s cage-free and broiler campaigns moved policies forward by only one year (I just added the description. But this assumes that other organizations would have still done everything they did. And even then, I don’t think it is reasonable.
To be honest, I think that nobody has a clue about what value to use here. Hence, everybody uses random conservative values in order for the end result to be more believable, because the estimated cost-effectiveness of campaigns is unbelievably high even with a conservative values. I did that to a degree as well. I asked some people who work on corporate campaigns what value would they use for mean years of impact. They thought that my range was reasonable, but I think they would have said that about many different ranges because it’s difficult to think about. Only one person was able to say what range for mean years of impact would they use without looking at my range, and they said 40 to 100 years. If I weren’t anchored by other estimates and didn’t want to be a bit conservative to be more convincing to skeptics, I think I would have chosen a higher value as well, especially for the upper bound. In general, my impression is that all these “estimates” (including mine) are basically guesses that look more legitimate than they should because they are surrounded by neatly formatted text. We just can’t predict the future.
I also don’t see how to do more research on this topic. Perhaps seeing how long corporate commitments in other domains lasted could give a little bit information on that but only a little bit because situations are so different. If you have any ideas, I’d be interested to hear them.
But I’m uncertain if it’s worthwhile to put more effort into cost-effectiveness estimates of corporate campaigns. I don’t know what decisions depend on whether it’s 7.5-24 years or 9-120 years. I think it would be more decision-relevant to research indirect effects, or in which regions corporate campaigns are more cost-effective, or how corporate campaigns compare with legislative campaigns for welfare reforms.
I use a subjective confidence interval of 4 to 36 years with a log-normal distribution, so it’s already very uncertain. Perhaps I should have used a flatter custom distribution, but after experimenting, it seems that it wouldn’t change the answer much.
Interesting, I will consider it. Do you know of any report that I could use as an example of how to write such a thing?