Thanks for the suggestions! As we were discussing above, combining this estimate with a prior estimate using Bayes’ rule might be a solution here. Taking the uncertainty of the model into account, we indeed score this approach quite poorly when it comes to the evidence-base aspect of it. We have a different research template for approaches than the one you linked. I expect to publish the whole report on corporate outreach pretty soon.
When it comes to the step between research questions and the probability distribution, full research, answering each question, can be seen in the full report. In the report, we also address some of the concerns you have with the judgement calls on each of the “qualitative” parameters.
Each update incorporates the weight we put on this factor, the directionality and strength. Those factors, again, rely on other information. With the example, you cited ”what ACE thinks makes an effective campaign” vs “probability that all companies defect in a Prisoner’s Dilemma scenario”. For example, ACE’s opinion on the importance of public support when launching corporate campaigns is formed based on the intervention report they have researched in November 2014, and as they currently claim “is not up to our current standards.”. The landscape has changed since then. As of recent, we can observe that there is a strong track record of successful corporate campaigns in countries where the society didn’t have sympathetic views toward animals (e.g. Lithuania or Japan). I think we can rely more and more on rigorous and generalizable conclusions from research on real-life examples and on the application of game theory to predict the behaviour of the companies.
I agree I wish we had enough time to flesh out the reasoning for each of the factors. Sadly, due to limited time we are constantly having to make trade-offs about whether we should put time into explaining the reasoning more deeply to the broader community vs discussing with the CE candidates vs researching more to get a deeper internal understanding. We generally plan on going deeply into these factors with the specific entrepreneurs looking to start this project or others, who are going to work/are working in the field in the near term, but not publish much more on the topic publicly after our full report.
Thank you for the suggestion. I agree that we can’t extrapolate the conclusion about predicted follow-through rate based on what percentage of companies have followed through on the commitment so far. I looked at it again, and I think that if analyzed correctly it still provides valuable information, so I will leave it in the model, but I’ll move it to the section on updates based on qualitative data and change the values based on the information below. I think that a good proxy for the information is whether the top 20 biggest companies are making progress and will eventually switch is if they responded to EggTrack. We can break it down to:* Companies that passed their deadline: Whole Foods (2004) − 100% follow-throughCostco (2018) - as of July 2017, 78% and 100% converted to liquid*Those that didn’t respond to EggTrack are Walmart, Albertsons, McDonald’s, Target, Sysco, ALDI, Burger King, Tim Hortons, Southeastern, and Wendy’s. * Those that did respond and reported progress: Kroger (21% as of 2017), Publix (50%), SUPERVALU (as of 2016, 12%)* Those in 20 that I do not have information on US Foods, IGA, Inc., Associated Grocers of Florida.I will estimate the value in this cell based on this information unless you have info that could fill in the gaps. I generally think that with all very uncertain estimates, whatever the result, it should be only treated as a cautious update and be combined with prior estimates of value. As a meta comment, I think I’m less concerned about an error in one of the parameters than you seem to be because of the different goals of the research. My goal is to reach broadly good conclusions about which intervention should be executed from a given list of options given a limited amount of time, rather than get the right answer to a specific question, even if it takes me an extremely large amount of time. I think that using cluster approach is superior in such cases. If you are using cluster approach, the more perspectives you take into account the lower the odds of your decision being wrong, and so I trade other aspects (eg. number of interventions compared in a given time frame and how accurate a single estimate need to be) differently. One contradicting factor also cannot overpower the whole decision, etc. A completely different method should be used when we are trying to have as accurate beliefs about the world as possible vs getting to a good decision.
When evaluating cost-effectiveness of interventions or charities, GiveWell only looks at how the action affects the most important metric that the charity is trying to adress. For example, when analysing the cost-effectiveness of SMS vaccine reminders, they only take into account the effect on the vaccination rate, but not on breastfeeding rates, which is also promoted by the intervention. We look at the effect on multiple disperse metrics, including health effects, reduced birth rate, woman empowerment, effect on animal welfare and the environment, etc. Additionally, we have not yet determined that condom distribution is going to be the intervention the charity is going to pursue. We are also considering SMS for reproductive health, community reproductive education, advance provision of EC etc.
I agree that there is no obvious way to model it and the method would even depend on the goal of the model, and it might not necessarily cross-apply to seemingly similar cases.
The estimate reflects a probability distribution of the percentage of corporations that have pledged a welfare improvement that will follow through on those pledges. Note here that it doesn’t inform about what percentage of companies in a country that the organization operate will implement the improvement, but rather the percentage of companies out of companies that have already pledged. Here the 39% − 50% is the most plausible outcome, but the model also includes, for example, the small probability of just 5% of companies following-through. We are also trading the accuracy of the result for the value of the information it provides. Of course, I feel fully confident that the true outcome will be somewhere between 0% and 100%, but this result is not that informative when we need to make a call.
I was modelling in mostly having in mind CE’s asks recommendations: food fortification and management of DO levels. That enabled us to narrow it down and make it more generalizable. I agree it won’t be generalizable for other asks, like the one that you used or even for the broiler asks for the same reasons.
Given your aims, you can use my estimates but just give any prior estimate, given that presumably, your priors aren’t flat or 1.
An alternative to that method might be estimating number of animals affected rather than percentage of corporations since presumably animals aren’t distributed evenly across corporations and so it seems possible that you might hit >x% of animals with x% of corporations. That would require modelling it for a very specific case if you want to get a “usable” result.
Thanks for the suggestions. For our poverty year we mainly focused on GiveWell priority programs although considered some interventions in the hits based area. Next year we plan on writing up some views comparing hits based giving to evidence-based giving and how we think they compare in expected value.
Yes, we do have reports on both of those ideas. I will link it here as soon as they are published.
Thank you for the recommendation. Yes, we took some ideas or their variations from that book.
Thanks, I will definitely post the results. I also encourage others to test it as well, so we have more generalizable data.
Thank you for the suggestion. I’m always open for ideas on productivity improvements, especially if they directly affect charity entrepreneurs ;)
We generated a list of 100 ideas and prioritized them based on things like expected effect on the general population, on me and Joey, ease of testing, etc. As far as I remember, rotating positions from sitting on an office chair to standing to sit on a ball or laying on a couch are more strongly recommended than any single one of those. I think testing all of the tools you can use to be physically active would be an interesting separate experiment in itself. Have you ever tried a mini-stepper? How did you find the effects of a treadmill compared to a mini-stepper?
Thank you for compiling information on fish oil used in fish feed. As part of the research at Charity Entrepreneurship, I recently published a report exploring fish feed optimization as a potential intervention. We had mostly focused on fishmeal, so you might be interested in complementary research. A lot of crucial considerations that we’ve explored are also applicable to fish oil. You can find the whole report here.
Water quality including dissolved oxygen is affected by three main categories of causes; one of which is the biological loading and water treatment systems applied by the farmer that includes management of oxygen level. DO level is affected by multiple stable factors (like temperature) but also sporadic factors including overfeeding, swimming activity or CO2 increase, so it is important that the baseline dissolved oxygen level has a safety margin for temporary increases in DO requirements.
CE has rating that include wild fish and wild fish caught for human use.