I don’t really have a strong view about LIC—as I’ve mentioned elsewhere in the comments, I’m skeptical in general that very EA donors should give to farmed vertebrate welfare issues in the near future. But I don’t find this level of evidence particularly compelling on its own. I think I feel confused about the example you’re giving because it isn’t about hypothetical cost-effectiveness, it’s about historic cost-effectiveness, where what matters are the counterfactuals.
I broadly think the critique is interesting, and again, seems like probably an issue with the methodology, but on its own doesn’t seem like reason to think that ACE isn’t identifying good donation opportunities, because things besides cost-effectiveness also matter here.
But I don’t find this level of evidence particularly compelling on its own.
You don’t find these facts particularly compelling evidence that LIC is not historically cost-effective?
LIC’s most cost-effective intervention was one in which they spent over $200,000, and the lawsuit was dismissed for failing to state a valid legal claim.
LIC received over a million dollars in funding prior to being reviewed
LIC existed for multiple years prior to being reviewed
LIC failed to secure any favorable legal outcomes, or file any lawsuit that stated a valid legal claim?
What would be compelling evidence for LIC not being historically cost-effective?
I think I feel confused about the example you’re giving because it isn’t about hypothetical cost-effectiveness, it’s about historic cost-effectiveness, where what matters are the counterfactuals.
ACE does 2 separate analyses for past cost-effectiveness, and room for future funding. For example, those two sections in ACE’s review of LIC are:
Cost Effectiveness: How much has Legal Impact for Chickens achieved through their programs?
Room For More Funding: How much additional money can Legal Impact for Chickens effectively use in the next two years?
Our review focuses on ACE’s Cost-Effectiveness analysis, not on their Room For More Funding analysis. In the future, we may evaluate ACE’s Room For More Funding Analysis, but that is not what our review focused on.
However, I would like to pose a question to you: Given the ACE often gives charities a worse historic cost-effectiveness rating for spending less money to achieve the exact same outcomes (see Problem 1), how confident do you feel in ACE’s ability to analyze future cost-effectiveness (which is inherently more difficult to analyze)?
I don’t find that evidence particularly compelling on its own, no. Lots of projects cost more than 1M or take more than a few years to have success. I don’t see why those things would be cause to dismiss a project out of hand. I don’t really buy social movement theories of change for animal advocacy, but many people do, and it just seems like many social movement-y things take a long time to build momentum, and legal and research-focused projects take forever to play out. Things I’d want to look at to form a view on this (though to be clear, I plausibly agree with you!):
How much lawsuits of this type typically cost
What the base rate for success is for this kind of work
How long this kind of work typically takes to get traction
Has anyone else tried similar work on misleading labelling or whatever? Was it effective or not?
Has LIC’s work inspired other lawsuits, as ACE reported might be a positive side effect?
I don’t think we disagree that much here, except how much these things matter — I don’t really care about ACE’s ability to analyze cost-effectiveness outside broad strokes because I think the primary benefits of organizations like ACE is shifting money to more cost-effective things within the animal space, which I do believe ACE does. I also don’t mind ACE endorsing speculative bets that don’t pay off — I think there are many things that were worth paying for in expectation that don’t end up helping any animals, and will continue to be, because we don’t really know very many effective ways to help animals so the information value of trying new things is high.
But to answer your question specifically, I’d be very skeptical of anyone’s numbers on future cost-effectiveness, ACE’s or yours or my own, because I think this is an issue that has historically been extremely difficult to estimate cost-effectiveness for. I’m not convinced that’s the right way to approach identifying effective animal interventions, in part because it is so hard to do well. I don’t really think ACE is making cost-effectiveness estimates here though—it seems much more like trying to get a rough sense of relative cost-effectiveness, which, putting aside the methodological issues you’ve raised, seems like the right approach to me, but only a small part of the information I’d want to know where money should move in animal advocacy.
What the base rate for success is for this kind of work
How long this kind of work typically takes to get traction
The Nonhuman Rights Project provides a possible point of comparison. From 2013 to 2023 they raised $13.2 Million. As far as I know, they have never won a case.
I don’t find that evidence particularly compelling on its own, no. Lots of projects cost more than 1M or take more than a few years to have success. I don’t see why those things would be cause to dismiss a project out of hand.
The question I asked was: “You don’t find these facts particularly compelling evidence that LIC is not historically cost-effective?”
The question was not about whether these facts are compelling evidence that LIC won’t be successful in the future, or if the project should be dismissed.
Wait, those are related to each other though—if we haven’t seen the full impact of their previous actions, we haven’t yet seen their historical cost-effectiveness in full! Also, you cite these as reasons the project should be dismissed in your post—you have a section literally called “Legal Impact for Chickens Did Not Achieve Any Favorable Legal Outcomes, Yet ACE Rated Them a Top Charity” which reads to me that you believe that it is bad they were rated a Top Charity, and make these same arguments (and no others) in the section, suggesting that you think this evidence means they should be dismissed.
Wait, those are related to each other though—if we haven’t seen the full impact of their previous actions, we haven’t yet seen their historical cost-effectiveness in full!
No, they are not. Historical cost-effectiveness refers to past actions and outcomes—what has already occurred.
All of LIC’s legal actions have already been either dismissed or rejected. What are you suggesting we need to wait for before we can analyze LIC’s historical cost-effectiveness in full?
You are conflating the issue of past cost-effectiveness with future potential.
Also, you cite these as reasons the project should be dismissed in your post—you have a section literally called “Legal Impact for Chickens Did Not Achieve Any Favorable Legal Outcomes, Yet ACE Rated Them a Top Charity” which reads to me that you believe that it is bad they were rated a Top Charity, and make these same arguments (and no others) in the section, suggesting that you think this evidence means they should be dismissed.
Did I claim that I don’t think LIC “should be dismissed”?
I don’t really have a strong view about LIC—as I’ve mentioned elsewhere in the comments, I’m skeptical in general that very EA donors should give to farmed vertebrate welfare issues in the near future. But I don’t find this level of evidence particularly compelling on its own. I think I feel confused about the example you’re giving because it isn’t about hypothetical cost-effectiveness, it’s about historic cost-effectiveness, where what matters are the counterfactuals.
I broadly think the critique is interesting, and again, seems like probably an issue with the methodology, but on its own doesn’t seem like reason to think that ACE isn’t identifying good donation opportunities, because things besides cost-effectiveness also matter here.
You don’t find these facts particularly compelling evidence that LIC is not historically cost-effective?
LIC’s most cost-effective intervention was one in which they spent over $200,000, and the lawsuit was dismissed for failing to state a valid legal claim.
LIC received over a million dollars in funding prior to being reviewed
LIC existed for multiple years prior to being reviewed
LIC failed to secure any favorable legal outcomes, or file any lawsuit that stated a valid legal claim?
What would be compelling evidence for LIC not being historically cost-effective?
ACE does 2 separate analyses for past cost-effectiveness, and room for future funding. For example, those two sections in ACE’s review of LIC are:
Cost Effectiveness: How much has Legal Impact for Chickens achieved through their programs?
Room For More Funding: How much additional money can Legal Impact for Chickens effectively use in the next two years?
Our review focuses on ACE’s Cost-Effectiveness analysis, not on their Room For More Funding analysis. In the future, we may evaluate ACE’s Room For More Funding Analysis, but that is not what our review focused on.
However, I would like to pose a question to you: Given the ACE often gives charities a worse historic cost-effectiveness rating for spending less money to achieve the exact same outcomes (see Problem 1), how confident do you feel in ACE’s ability to analyze future cost-effectiveness (which is inherently more difficult to analyze)?
I don’t find that evidence particularly compelling on its own, no. Lots of projects cost more than 1M or take more than a few years to have success. I don’t see why those things would be cause to dismiss a project out of hand. I don’t really buy social movement theories of change for animal advocacy, but many people do, and it just seems like many social movement-y things take a long time to build momentum, and legal and research-focused projects take forever to play out. Things I’d want to look at to form a view on this (though to be clear, I plausibly agree with you!):
How much lawsuits of this type typically cost
What the base rate for success is for this kind of work
How long this kind of work typically takes to get traction
Has anyone else tried similar work on misleading labelling or whatever? Was it effective or not?
Has LIC’s work inspired other lawsuits, as ACE reported might be a positive side effect?
I don’t think we disagree that much here, except how much these things matter — I don’t really care about ACE’s ability to analyze cost-effectiveness outside broad strokes because I think the primary benefits of organizations like ACE is shifting money to more cost-effective things within the animal space, which I do believe ACE does. I also don’t mind ACE endorsing speculative bets that don’t pay off — I think there are many things that were worth paying for in expectation that don’t end up helping any animals, and will continue to be, because we don’t really know very many effective ways to help animals so the information value of trying new things is high.
But to answer your question specifically, I’d be very skeptical of anyone’s numbers on future cost-effectiveness, ACE’s or yours or my own, because I think this is an issue that has historically been extremely difficult to estimate cost-effectiveness for. I’m not convinced that’s the right way to approach identifying effective animal interventions, in part because it is so hard to do well. I don’t really think ACE is making cost-effectiveness estimates here though—it seems much more like trying to get a rough sense of relative cost-effectiveness, which, putting aside the methodological issues you’ve raised, seems like the right approach to me, but only a small part of the information I’d want to know where money should move in animal advocacy.
How much lawsuits of this type typically cost
What the base rate for success is for this kind of work
How long this kind of work typically takes to get traction
The Nonhuman Rights Project provides a possible point of comparison. From 2013 to 2023 they raised $13.2 Million. As far as I know, they have never won a case.
The question I asked was: “You don’t find these facts particularly compelling evidence that LIC is not historically cost-effective?”
The question was not about whether these facts are compelling evidence that LIC won’t be successful in the future, or if the project should be dismissed.
Wait, those are related to each other though—if we haven’t seen the full impact of their previous actions, we haven’t yet seen their historical cost-effectiveness in full! Also, you cite these as reasons the project should be dismissed in your post—you have a section literally called “Legal Impact for Chickens Did Not Achieve Any Favorable Legal Outcomes, Yet ACE Rated Them a Top Charity” which reads to me that you believe that it is bad they were rated a Top Charity, and make these same arguments (and no others) in the section, suggesting that you think this evidence means they should be dismissed.
No, they are not. Historical cost-effectiveness refers to past actions and outcomes—what has already occurred.
All of LIC’s legal actions have already been either dismissed or rejected. What are you suggesting we need to wait for before we can analyze LIC’s historical cost-effectiveness in full?
You are conflating the issue of past cost-effectiveness with future potential.
Did I claim that I don’t think LIC “should be dismissed”?