VettedCauses
Hi Aidan, thank you for providing your input to the community.
I think it’s pretty safe to assume that the reality of most charities’ cost-effectiveness is less than they claim.
It appears we agree that Sinergia is making false claims about helping animals. ′
We are curious if you think this is proper grounds for not recommending them as a charity.
Thanks for your reply Jason—great to hear your perspective.
One problem is that your argument for importance also underscores how difficult and risky for an organization to “alter, fabricate, and/or destroy evidence” in the way you describe.
We understand it is risky, but could you clarify why it would be difficult for an organization to “alter, fabricate, and/or destroy evidence?” In many cases, wouldn’t it be as simple as changing values in a spreadsheet?
Thanks for clarifying, Neel, and for giving us the opportunity to clarify our statements. Quoting our original post:
It is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claims.
By “sufficient evidence” we did not mean proof eliminating all possible doubt. We meant evidence that would satisfy a reasonable person. For example, a reasonable person thinks there is sufficient evidence that Neil Armstrong landed on the moon, but some people do not.
Additionally, we do not expect charities to provide sufficient evidence for all of their publicly stated claims, just their important ones. For instance, if a charity claims to have provided 10,000 meals to homeless individuals, they should provide sufficient evidence that this actually happened. However, if they announce that their end-of-year party was at Disneyland, no additional evidence is necessary.
Re your point about charities should be incentivised to provide sufficient public evidence. I think this is an extremely unreasonably high standard.
Hi Neel, thanks for the reply. Could you clarify what standard you’re referring to?
Thanks for the tool! It seems very useful.
they have a lot to lose as soon as it’s confirmed that the archived data is not manipulated.
We think our team still has some disagreements with you over how effective disinformation campaigns can be (especially when the disinformation is technical and the audience is mostly non-technical). That being said, we really appreciate your insights—you’ve made some great points.
Thank you for your reply and technical insights, Lorenzo.
To clarify, we are actually not that concerned about archived documents being manipulated. From what we understand, this is extremely rare.
What we are quite concerned about is that we will be falsely accused of manipulating archives, and the charity accusing us will be given the benefit of the doubt. They could cite articles like the one we cited earlier, and most people do not have the technical expertise to evaluate disputes over archive integrity.
The aforementioned page states that they took action “to mitigate these attacks,” so from our understanding it is still possible to do.
Also, the organization who completed the study still cautions users who rely on Wayback Machine (the archive platform that was manipulated).[1]
- ^
https://rewritinghistory.cs.washington.edu/index.html See section “I rely on Wayback Machine—what should I do?”
- ^
Hi Michael, thanks for the reply.
https://web.archive.org/ seems good enough to me in most cases?
That archive service is great, we use them all the time. From our understand though, it is actually possible to manipulate web archives.
Thanks for the feedback, Vasco!
Risk 2: Unconscious biases from interacting with charity staff.
You can try correcting for this by not updating as much towards being positive about the organisation as naively warranted by the feedback you received from people working there. In addition, you can avoid meeting with people from the organisation, restricting yourself to sharing a doc with your review, and discussing it via messages.
These are fair points. We agree the risk of bias is likely minimal if this is how you handle it.
Hi Jason, thank you for your reply.
We believe we have addressed your questions here. Please let us know if there is anything we missed.
Hi Michael, thank you for your reply. We definitely agree with you that there are potential benefits to reaching out to a charity before writing a review on them; our answer to both of your questions is “no.”
We have provided more information regarding the thought process behind our decision here.
Hi David, thank you for taking the time to read our article. You make some interesting points.
In terms of the work Sinergia did to claim credit for helping millions of pigs, it was listed in Column W of this spreadsheet, but Column W was deleted in March of 2025.
Hi Marcus, thank you for your reply. We agree with you on a lot of what you’ve said. However, we would like to clarify something.
However much American companies follow corporate commitments, I expect less of Brazilian ones.
As we noted in our review, Sinergia allegedly secured a teeth clipping commitment from Pif Paf Alimentos (PPA).
Sinergia estimates that this commitment has helped over 1,000,000 piglets per year since 2023. Sinergia cites a source for their estimate, but the source states PPA slaughters only 750,000 pigs per year.[1] This makes it questionable if PPA even has 1,000,000 piglets that could be teeth clipped each year.
Thus, Sinergia’s estimate appears to rely on the assumption that without this commitment, PPA would have immediately started using teeth clipping on 100% of their piglets, but that with the commitment they will use teeth clipping on 0% of their piglets.
Sinergia appears to make this same assumption for all of the piglet/sow welfare commitments they allegedly secured.
Note: teeth clipping was also illegal in Brazil (PPA’s country) prior to the alleged commitment[2], and an archived version of the web page Sinergia cites for the commitment shows that the alleged commitment was already on PPA’s website prior to when the commitment was allegedly published[3].
- ^
This information can be found in Row 12 of Sinergia’s spreadsheet. Please let us know if there is anything we can clarify.
- ^
See Article 38Section 2 and Article 54 of Normative Instruction 113/2020.
- ^
Archived web page from PPA’s website from October 24, 2022.
- ^
Hi Abraham, thank you for taking the time to read our article and reply to this thread. We’re sorry for engaging with you negatively on our previous thread.
I worry that Google translating pages loses nuance that might matter here. Google Translating Brazilian law also seems like a huge stretch as evidence
The only thing we’d like to say for this thread is that we spent several hundred hours working on this review, and we can assure you that we did a lot of work to verify that what we said is true. We did not just Google translate documents and accept the translations as factual. For the “Brazilian law” you reference, Sinergia themselves stated in their 2023 report that “teeth clipping is prohibited” under Normative Instruction 113/2020.
If you’d like, we can provide additional sources stating this (one of our other sources cited in our article actually already does state this). However, the original writing of Normative Instruction 113/2020 is in only in Portuguese, so we are unable to provide an English version.
Thank you for your comment, Lorenzo.
We chose not to share the article with Sinergia or ACE before publishing it because:
The claims we critique were publicly made—we are evaluating what Sinergia has already presented to donors, not private internal data.
The evidence is verifiable—our conclusions are based on independently verifiable sources (archived web pages, legal regulations, etc.) that are cited throughout the article.
Independent analysis is critical—allowing organizations to review critiques before publication can introduce biases that weaken accountability.
With that being said, we’re open to engagement. If Sinergia, ACE or anyone else believes any specific fact is incorrect, we’d be happy to review their evidence.
Thank you for clarifying this. We agree that our comment above implied a greater degree of unreasonableness on behalf of THL than was warranted.
That being said, we still believe the law is likely in a worse place for animals compared to before the filing of the initial case. The interpretation offered by the Court appears to be quite strict, and we suspect it will be difficult for subsequent litigants to meet the evidentiary standard established by this case. Hopefully we are wrong about this.
We also still believe that THL’s communication surrounding this case was quite poor, and misleading. Notably, THL did not even disclose the name of the court case in this post, or in the post on their website.
I don’t think THL UK’s view of the case is as unreasonable as implied.
THL claimed “we believe that last week’s ruling on Franken chickens marks ‘the beginning of the end of this type of this type of outdated farming’”[1] and has a collaborative post on their Instagram page that states that this ruling marked “a significant victory for animal welfare.”[2][3]
The Court dismissed THL’s case, and provided an unfavorable interpretation of the animal welfare law in question.[4] All trial courts in the UK (except for those in Northern Ireland and Scotland) are now required to use this unfavorable interpretation of the animal welfare law in question.
Is this a significant victory for animal welfare?
The Humane League (THL) filed a lawsuit against the UK Secretary of State for Environment, Food and Rural Affairs (the Defra Secretary) alleging that the Defra Secretary’s policy of permitting farmers to farm fast-growing chickens unlawfully violated paragraph 29 of Schedule 1 to the Welfare of Farmed Animals (England) Regulations 2007.
Paragraph 29 of Schedule 1 to the Welfare of Farmed Animals (England) Regulations 2007 states the following:
“Animals may only be kept for farming purposes if it can reasonably be expected, on the basis of their genotype or phenotype, that they can be kept without any detrimental effect on their health or welfare.” [1]
THL’s case was dismissed.
THL appealed the dismissal, and again THL’s case was dismissed (this most recent dismissal is what THL’s post is about).
In this most recent dismissal, the Court clarified the meaning of Paragraph 29 as follows:
“Paragraph 29 was not concerned with the environmental conditions in which animals were kept; it was concerned with the characteristics of the breed, and with detriment which could not be mitigated by improving the animal’s environment”
“Accordingly, paragraph 29 was a prohibition on the keeping of farmed animals whose genotype and phenotype meant that, regardless of the conditions in which they were kept, they could not be kept without detriment to their health or welfare” [2]
Essentially, the Court ruled that Paragraph 29 is only violated if an animal is bred such that it cannot avoid genetically caused health/welfare problems even under perfect environmental conditions (i.e. giving the animal the best possible food/diet, a perfect living environment, and world class medical treatment). This allows farmers to continue to farm animals so long as their genetic issues can theoretically be mitigated by improving conditions, even if those conditions are unlikely to be implemented in practice.
For example, let’s say there is a genetically selected breed of chicken that under normal factory farming conditions grows so fast that their legs snap under their weight by the time they are a month old. Under the Court’s ruling, this would not violate Paragraph 29, so long as this problem (and other genetically caused problems) could theoretically be mitigated with better environmental conditions (i.e. giving the chicken the best possible food/diet, a perfect living environment, and world class medical treatment).
Since the Court offered this interpretation of Paragraph 29, all trial courts in the UK (except for those in Northern Ireland and Scotland) are now required to use this interpretation of Paragraph 29 when making rulings.
From our understanding, this is not a favorable interpretation of Paragraph 29, as it makes it extremely difficult to prove that a violation of Paragraph 29 has occurred. Under this ruling, the only way to prove that a Paragraph 29 violation has occurred is by proving the health/welfare problems encountered by an animal are completely unavoidable, even with absolutely perfect environmental conditions/treatment.
Because of this ruling, anyone who ever tries to claim a Paragraph 29 violation has occurred will have to meet this extremely high standard of evidence that the Court has laid out.
Hi,
Thank you for taking the time to read our review and for responding to each of our points. We really appreciate ACE’s willingness to engage with feedback and acknowledge problems.
Regarding your clarifications related to the calculation of Normalized Achievement Scores:
‘Charities can have 1,000,000 times the impact at the exact same price, and their Normalized Achievement Scores and Cost-Effectiveness Score can remain the same.’
This isn’t the case, but we didn’t publish the full details about our method for assessing the impact of books, podcasts, and other interventions, so we see why this wasn’t clear. Essentially for each intervention in our Menu of Interventions we identified proxies for its likely impact. For books, we had intended to include sales/views as well as a rating of the overall audience response/reviews. In practice, this wasn’t possible for various reasons given the wide variation in types of publication (e.g., some publications had not been released yet, or had been provided directly to the audience with no feedback collected), so we had to factor in such considerations on a more case-by-case basis in our Recommendations Decisions discussions.
We are glad to hear that ACE was accounting for these factors behind the scenes.
‘Charities can increase their Normalized Achievement Scores and Cost-Effectiveness Score by breaking down actions into smaller steps, even if the overall results remain unchanged.’
This actually isn’t the case (sorry if this wasn’t clear). Breaking down an achievement into smaller steps would drive up the ‘Achievement quantity’ score, but would be offset by lower ‘Achievement quality’ scores for each achievement. However, there was still a risk of this introducing inconsistency into the model, which is another reason why we updated our methods this year.
Thank you for clarifying this. From the publicly available rubrics for calculating Achievement Quality Scores, it did not seem like breaking down an achievement into smaller steps would decrease the Achievement Quality Score at all. However, given that ACE was accounting for factors outside of the publicly available rubrics, it makes sense that this decrease could occur.
That being said, we believe it is important for ACE to fully disclose its methodology to the public and avoid relying on hidden evaluation criteria. This transparency would allow people from outside the organization to understand how ACE’s charity evaluation metrics (i.e. Normalized Achievement Scores) were calculated.
We might also reach out to you via email in the coming weeks as we go through retrospectives and plan for next year’s evaluation. Because of the complexity of the animal welfare cause area, the many uncertainties and knowledge gaps in the field of charity evaluation, and the urgency and scope of suffering, we embrace productive collaboration.
We appreciate your openness to collaboration. Feel free to reach out to us at any time at hello@vettedcauses.com
Thanks for the insights Jason, we completely agree.
We are curious if you would you also agree that it is (1) reasonable to assert “Charity X didn’t feed 10K meals to homeless individuals” and (2) unreasonable to give Charity X the benefit of the doubt, if:
“Charity X has not provided sufficient, publicly available evidence to substantiate its claim to have provided 10K meals to homeless individuals” and
There is publicly available and sufficient evidence indicating Charity X did not provide 10K meals to homeless individuals.