We agree that the majority of our analysis should focus on the future work that would be enabled by ACE’s recommendation. However, forward-looking CEAs are inherently more subjective because they rely on projected metrics rather than actual past results. For this reason, we tend to create backward-looking CEAs and then assess whether there are any reasons to expect diminishing returns in the next two years (the duration of an ACE recommendation). When GWWC shared with us anonymized comments from the experts they consulted on this topic, the comments acknowledged these limitations of forward-looking CEAs. However, we also think there are cases where forward-looking CEAs can be helpful despite these limitations, for example when charities are planning new programs that are not currently funded.
We do not recommend charities if there is a large enough gap between their expected marginal cost-effectiveness and that of our other charities, and we do use the framing that you suggest when considering adding the next marginal charity. However, since we are unable to always fully quantify the impact on animals of charities’ work, this is partially based on qualitative arguments and judgments, so our decisions may not always appear consistent with the results of our CEAs.
In general, we quantify uncertainty within our CEA assessments and we also qualitatively assess the risk of each program. Additionally, we screen out applicants whose work is “too” uncertain based on their track record and whether or not the charities themselves are uncertain about where future funding would go. Our Movement Grants program does not have these bars and is willing to fund newer and more exploratory programs. However, we do agree that it’d be worthwhile to be clearer about how we weigh different types of risk in our decision-making, and we’ll consider adding this to our communications.
We do not recommend charities if there is a large enough gap between their expected marginal cost-effectiveness and that of our other charities
Your lower bound for the cost-effectiveness of Sinergia is 1.87 (= 217/116) times your upper bound for the cost-effectiveness of ÇHKD, which again points towards only Sinergia being recommended.
For this reason, we tend to create backward-looking CEAs and then assess whether there are any reasons to expect diminishing returns in the next two years (the duration of an ACE recommendation).
Makes sense. I very much agree the CEAs of past work are valuable. However, I suspect it would be good to be more quantitative/explicit about how that is used to inform your views about the cost-effectiveness of the additional funds caused by your recommendations. For example, you could determine the marginal cost-effectiveness of each organisation adding the contributions of their programs, determining each contribution multiplying:
The fraction of additional funds (which would be caused by your recommendation) going to the program i. You could ask the organisation about this.
The cost-effectiveness of additional funds going to the program as a fraction of its past cost-effectiveness. You currently consider this qualitatively.
The past cost-effectiveness of the program. You currently consider this quantitatively sometimes via backward-looking CEAs.
We do not recommend charities if there is a large enough gap between their expected marginal cost-effectiveness and that of our other charities, and we do use the framing that you suggest when considering adding the next marginal charity.
Great!
However, since we are unable to always fully quantify the impact on animals of charities’ work, this is partially based on qualitative arguments and judgments, so our decisions may not always appear consistent with the results of our CEAs.
Hi Vasco,
We agree that the majority of our analysis should focus on the future work that would be enabled by ACE’s recommendation. However, forward-looking CEAs are inherently more subjective because they rely on projected metrics rather than actual past results. For this reason, we tend to create backward-looking CEAs and then assess whether there are any reasons to expect diminishing returns in the next two years (the duration of an ACE recommendation). When GWWC shared with us anonymized comments from the experts they consulted on this topic, the comments acknowledged these limitations of forward-looking CEAs. However, we also think there are cases where forward-looking CEAs can be helpful despite these limitations, for example when charities are planning new programs that are not currently funded.
We do not recommend charities if there is a large enough gap between their expected marginal cost-effectiveness and that of our other charities, and we do use the framing that you suggest when considering adding the next marginal charity. However, since we are unable to always fully quantify the impact on animals of charities’ work, this is partially based on qualitative arguments and judgments, so our decisions may not always appear consistent with the results of our CEAs.
In general, we quantify uncertainty within our CEA assessments and we also qualitatively assess the risk of each program. Additionally, we screen out applicants whose work is “too” uncertain based on their track record and whether or not the charities themselves are uncertain about where future funding would go. Our Movement Grants program does not have these bars and is willing to fund newer and more exploratory programs. However, we do agree that it’d be worthwhile to be clearer about how we weigh different types of risk in our decision-making, and we’ll consider adding this to our communications.
Thanks, Vince
Your lower bound for the cost-effectiveness of Sinergia is 1.87 (= 217/116) times your upper bound for the cost-effectiveness of ÇHKD, which again points towards only Sinergia being recommended.
Thanks for the additional clarifications, Vince!
Makes sense. I very much agree the CEAs of past work are valuable. However, I suspect it would be good to be more quantitative/explicit about how that is used to inform your views about the cost-effectiveness of the additional funds caused by your recommendations. For example, you could determine the marginal cost-effectiveness of each organisation adding the contributions of their programs, determining each contribution multiplying:
The fraction of additional funds (which would be caused by your recommendation) going to the program i. You could ask the organisation about this.
The cost-effectiveness of additional funds going to the program as a fraction of its past cost-effectiveness. You currently consider this qualitatively.
The past cost-effectiveness of the program. You currently consider this quantitatively sometimes via backward-looking CEAs.
Great!
Have you described such judgements somewhere?