GWWC board member, software engineer in Boston, parent, musician. Switched from earning to give to direct work in pandemic mitigation. Married to Julia Wise. Speaking for myself unless I say otherwise. Full list of EA posts: jefftk.com/ânews/âea
Jeff Kaufman đ¸
Karma: 16,470
SpendÂing on Ourselves
ďNuÂcleic Acid ObÂserÂvaÂtory UpÂdates, April 2025
I think a week from today is a good amount of review time, but I think the key thing is to be clearer in setting expectations in the future.
(Iâd also recommend, as I wrote above, apologizing to Sinergia for misunderstanding their earlier request and giving them a very short time to review.)
In the future the most reliable way to redact a screenshot is:
Open the screenshot and add the black bars
Save the edited screenshot as PNG
Since PNG doesnât support layers or history, the redacted information is reliably no longer present.
Uh oh! The emails you shared do include the full text, but with black bars added on top in Google Slides. This doesnât actually remove the information, and is a mistake people sometimes make when trying to redact things. Because I used an LLM to transcribe the emails, which used the underlying images, I ended up with the unredacted text. Iâve now gone through my comment above and put
[redacted]anywhere I see a black bar, but you should be aware that the information is still in your slides for anyone to extract.
Youâre taking their wording too literally. You wrote asking them if 24hr was enough notice, and this response was Sinergia asking if they could instead tell you how long theyâd need after reviewing the draft. You donât have to accept that â you could say youâre not willing to hold off on publishing for more than a week or something â but when you didnât respond to it Sinergia was right to expect that you wouldnât drop a draft on them with 24hr notice before publication.
@VettedCauses DMâd me asking if I could look at this situation (Iâve been pushing a norm of running things by orgs before publishing while also sometimes talking about downsides). I read the emails that VettedCauses shared below.
Overall, it seems to me that Sinergia is behaving pretty reasonably in this exchange, and Vetted Causes is not being clear enough in setting expectations about timelines and what opportunities Sinergia would have to correct things before publication. I especially think the âAfter Charity Didnât Objectâ in the title isnât right.
Under the circumstances, if I were VettedCauses I would write something like:
After discussing our emails on the EA Forum Iâd like to apologize for misunderstanding your requests around timing. We overlooked your request on deadlines in your message of the 27th, and weâre very sorry! Because we missed that, we had thought you didnât have issues with a 24 hour review period. While we canât in commit to unbounded review periods, weâre happy to have some discussion on a case-by-case basis, considering how much weâre asking for you to review.
With that in mind, would you like more time to prepare a public response before we publish the response weâd been targeting for today? For example, would Tuesday the 15th work?
Emails, with thoughts interspersed:
From: The Vetted Causes Team
To: [redacted], carolina
Date: Wed, Mar 26, 3:25 PMHello,
We saw your responses to our Sinergia Review, and wanting to respect your wishes, have decided to follow up with you in private before issuing a response.
We think there are some misunderstandings regarding our intentions, and would like to clear them up.
Our goal is to help animals by directing donations towards effective causes. In the past, our team has contributed tens of thousands of dollars in donations to ACEâs cause, and has explored ways to get more funding to ACE recommended charities. Given that some of these donations have been to ACEâs recommended charity fund, we would guess some of our contributions have even gone to Sinergia.
The text message below was sent by Adam Hebert (founder of Vetted Causes) to a current Vetted Causes teammate in 2020:
[screenshot of 2020-12-01 text message conversation showing three attached images without thumbnails, a link to Facebook Social Impactâs âYour Guide to Giving Seasonâ, and a message â3 of ACEâs top recommended charities took part in this event. I wonder if weâd be able to find events like these and match everyoneâs donationsâ]
We hope this makes it clear that Vetted Causes is motivated by a genuine desire to help animals. Weâd also like to note that later this year, we plan to publish positive reviews about some of the charities ACE has endorsed.We do plan to eventually release full responses to ACE and Sinergiaâs responses. In order to respect ACE and Sinergiaâs wishes, we will show this response to both of you at least 24 hours before publishing (or more if you would prefer). Additionally, we will send clarifying questions to both Sinergia and ACE prior to writing the responses in order to ensure there are no misunderstandings.
If Sinergia or ACE would like, weâd also be happy to have a phone call to clear anything else up. Please let us know if you have any questions.
Best,
The Vetted Causes Team
Basically reasonable, though âWe do plan to eventually release full responses to ACE and Sinergiaâs responses. In order to respect ACE and Sinergiaâs wishes, we will show this response to both of you at least 24 hours before publishing (or more if you would prefer).â shows less understanding of how organizations work than Iâd like to see from a charity evaluator, even a volunteer-run group. Better would be (a) default to an amount of time where the organization could plausibly review and respond without dropping everything, probably a week, (b) give more indication about when this might come than âeventuallyâ, and (c) clarify up front whether Vetted Causes is willing to engage in back-and-forth to improve the draft or whether theyâre just giving Sinergia the ability to prepare a public response.
From: Carolina Galvani
To: [redacted]
Date: Thu, Mar 27, 3:29 PMDear Adam and the Vetted Causes Team,
Thank you for reaching out and for your willingness to engage in further dialogue before publishing your posts and comments. We appreciate your consideration of our concerns, as well as those raised by the broader community regarding the importance of sharing the drafts of your posts in advance.
I am available to meet next week, and I may invite colleagues who have been involved in responding to your claims. You can schedule a meeting through the following link: https://ââcalendly.com/ââcarolina-galvani. We believe it is crucial to clarify your intentions regarding Sinergia and to speak about our request, according to the Forumâs norms, that you refrain from implying we act in bad faith.
To ensure a transparent and constructive process, we ask that you share the drafts of your publications about Sinergia Animal with us. After reviewing them, we will inform you of the time required to provide our response and any additional information, considering our internal priorities and the necessary level of analysis. We hope you understand that the manner in which this process has unfolded has disrupted our work and may have affected our reputation. Furthermore, members of the EA Forum have expressed a preference for this approach too, which we believe would contribute to saving everyoneâs time and a more informed discussion. Our goal is to minimize unnecessary miscommunications, disruptions, and potential harm moving forward.
Additionally, we seek clarification regarding your stated motivation for creating Vetted Causesâto direct donations to more effective causes. In the context of effective altruism, âmore effective causesâ typically refers to broad focus areas such as AI, climate change, existential risks, pandemic preparedness, and global health. Initially, we understood this to mean that your aim was to divert funding from animal welfare to these other areas. Could you please clarify your position?
We look forward to an open and constructive conversation.
Best regards,
Carolina
Also basically reasonable. In an ideal world Sinergia would be less prickly about whether VettedCauses is trying to redirect money between broad cause areas. Note that Sinergia is giving a counterproposal, requesting that the process be that first Vetted Causes shares drafts and then Sinergia says how long they will need. I donât think Vetted Causes has to agree to this, but unless if they didnât want to do it this way it would have been good to say something.
From: The Vetted Causes Team
To: adam, carolina, [redacted]
Date: Thu, Mar 27, 10:59 PMDear Carolina [redacted],
Thank you both for your kind replies. We hope there are no hard feelings, and look forward to resolving any misunderstandings.
Iâve scheduled a call with Carolina for next Friday via Calendly, and I look forward to speaking with you.
Carolina, regarding your question: Vetted Causes does not aim to divert funding from animal welfare to other cause areas. All of the charities we plan to recommend this year are animal-focused, and so far this year, weâve donated over $1,000 to animal charities and $0 to the other areas you mentioned.
Best,
AdamThis would have been a good time to mention the desire to record the call. This isnât a default in our society, and if Vetted Causes has a policy of requiring recordings that would be good to include up front.
Clarification on diversion is helpful.
From: Carolina Galvani
To: me, [redacted]
Date: Fri, Mar 28, 10:44 AM (13 days ago)Hi Adam,
Perhaps instead of saying, âMy sole motivation behind creating Vetted Causes is to direct donations to more effective causes,â you could say, âMy sole motivation behind creating Vetted Causes is to direct donations to more effective animal charities.â Just a small suggestion for consideration.
Looking forward to our conversation on Friday.
Best wishes,
I guess itâs fine for Sinergia to suggest this, though it would be better for the focus to stay on the review process.
From: Adam Hebert
To: Carolina, [redacted]
Date: Wed, Apr 2, 2:50 AM (8 days ago)Hi Carolina,
I hope this message finds you well. My teammate [redacted] would like to be part of our meeting this Friday, as he worked on the Sinergia review. If every email address needs to be approved to join the meeting, please approve both [Adamâs Email] and [[redacted]âs Email] to join the meeting.
As a standard policy, Vetted Causes records all meetings with charities. Please have everyone from your team who will participate in the meeting (including yourself) send [Adamâs Email] an email stating:
I consent to being recorded during the meeting between Sinergia and Vetted Causes on April 4, 2025. I understand that Vetted Causes will record every moment of this meeting from the moment a member of Vetted Causes joins the meeting to the moment every member of Vetted Causes has left the meeting, and I consent to this recording.
Unfortunately, if anyone joins the meeting under a different name or without submitting this written consent in advance, we will have to end the meeting immediately.
We look forward to speaking with you and your team!
Best, Adam Hebert
This is a weird message. It feels very aggressive and unprofessional, and itâs not taking into consideration that itâs asking for something unusual. It think it would have worked a lot better as something like:
Iâm sorry I forgot to mention this earlier, but Vetted Causes has a policy that all meetings we have with charities will be recorded. We use these recordings to ensure our notes are accurate, and we never share them outside Vetted Causes without the permission of the charity. Weâre also happy for you to record, as long as you similarly agree not to share outside your organization without our permission.
If this is ok with you, can you ensure everyone joining the meeting from your side is aware that weâll be recording?And then instead of asking for statements over email, doing the conventional thing of (a) confirming that Vetted Causes has consent to record before turning on recording and then (b) repeating that recording is happening after turning it on, so that itâs captured and itâs clear that anyone remaining in the call is aware theyâre being recorded.
(Iâm undecided on whether itâs reasonable for VettedCauses to have an all-charity-meetings-recorded policy)
From: Carolina Galvani
To: me, [redacted]
Date: Wed, Apr 2, 4:07 PM (8 days ago)Hi Adam and [redacted],
I hope you are well.
In this case, letâs keep written communications only.
I will cancel the meeting.
Best wishes, Carolina
Very reasonable response!
From: Adam Hebert
To: Carolina, [redacted]
Date: Thu, Apr 3, 11:11 AM (7 days ago)Hi Carolina,
No worries! Feel free to let me know if you change your mind. We also plan on starting a podcast soon, and youâd be welcome to come on as a guest.
Iâve attached a couple of questions weâd like to clarify for our response to Sinergia. Weâll also be sure to send our response to you prior to publishing it.
Thanks for your help!
Best, Adam Hebert
If someone has just said they donât want to be recorded, itâs pretty weird to immediately respond by inviting them on a podcast.
More clarity on âsend our response to you prior to publishing itâ would be better. Especially since Sinergia had requested a structure earlier (âafter reviewing them, we will inform you of the time required to provide our responseâ) and Vetted Causes hasnât indicated they have a problem with that.
From: Adam Hebert
To: Carolina, [redacted]
Date: Fri, Apr 4, 3:19 PM (6 days ago)Hi Carolina,
We wanted to let you know that we plan to post our response next Thursday (April 10th). Once you and ACE have sent us your answers to the questions we sent (two different questions were also sent to ACE), we will complete our response and send it to you within 24 hours for your review.
We want to incorporate your clarifications into our response to avoid any misunderstandings. We also want to post our response soon since people are likely wondering why we havenât responded yet.
Thanks for your understanding!
Best, Adam Hebert
Giving Sinergia a timeline on publishing is good, but the information on when theyâll be recieving the draft for comments is pretty messy. For example, setting aside the timing of Sinergiaâs responses, it sounds like if ACE doesnât give a response until, say, Wednesday April 9th then Sinergia might not get to review before publication. And if ACEâs response comes in on the 8th (as Vetted Causes says it did) then Sinergia only has a few hours to review before publication.
Ideally Sinergia would have pushed back on this timing.
From: The Vetted Causes Team
To: adam, [redacted]
Date: Wed, Apr 9, 6:34 AM (1 day ago)Hi everyone,
Attached [here] is a Google doc version of our response to Sinergia and ACEâs responses, which we plan to publish tomorrow (April 10th).
Please note that this Google doc does not have most of our citations, since we add citations at the time we convert our articles to TypeScript to be posted on our website. However, most our citations will be to statements from ourselves/âSinergia/âACE and Sinergia/âACEâs spreadsheets. Additionally, if you want to know what the citation is for any specific statement, you are welcome to send us an email and we will get back to you as soon as possible.
We would also like to note that we may make minor edits prior to publication, but we do not plan on making any major changes (we will be sure to notify you if we make one). Further, as noted in the introduction of our response, we will have a separate version of the article that doesnât use any summaries of what Sinergia said. This version will have close to the same text, but with exactly what Sinergia said in place of summaries (we are planning to just use pictures of Sinergiaâs response).
Please let us know if you have any questions or concerns. Thanks!
Best, The Vetted Causes Team
As above, I think 24hr is way too short to be practical. Sinergia did push back on this originally, but I think in a way that VettedCauses didnât understand was pushback (âafter reviewing them, we will inform you of the time required to provide our responseâ).
While itâs good that Vetted Causes is now making it clear that theyâre not planning on updating their article in response to issues Sinergia raises, it would have been much better (as I wrote above) to set these expectations from the beginning if that was indeed VettedCausesâ plan.
Sending without citations makes this way more work for Sinergia, since I expect a major thing they want to check is that theyâre not being misquoted or having quotes taken out of context. While this doesnât make sharing the draft with Sinergia useless, it does remove a lot of the value. Additionally âwe will get back to you as soon as possibleâ is pretty weak in a context of âweâre publishing tomorrowâ. While I think fixing the VettedCauses process to allow sharing cited drafts would be the way to go, saying something like âwithin 1hr if you request between 9am and 5pmâ would help indicate that VettedCauses is committed to doing their part to make this short deadline possible (setting aside that I disagree the urgency is necessary or helpful).
From: Carolina Galvani
To: me, [redacted]
Date: Wed, Apr 9, 2:35 PM (16 hours ago)Hi Adam,
Weâll be responding by the end of today. That said, we do plan to share in the forum our disappointment that our request to check on deadlines was not accepted. We are likely not to respond to further requests if they are done in the same manner.
We hope for a more constructive and respectful interaction moving forward.
Best regards, Carolina
Pretty reasonable!
From: Adam Hebert
To: Carolina, [redacted]
Date: Wed, Apr 9, 6:40 PM (11 hours ago)Hi Carolina,
Thank you for your message. Weâre sorry to hear that, but we donât believe we received a request to check deadlines.
For transparency to the public, would it be alright with you if Vetted Causes shared pictures of the emails that weâve sent to each other so far? This way, there is no misrepresentation of what happened. If so, Sinergia is welcome to do the same.
Also, weâd like to let you know that weâll be making a post on the forum asking for guidance on how to handle this situation. We hope you understand this is our first time sending a charity an article prior to publication, and we are doing our best to navigate it thoughtfully.
Thanks!
Best, Adam Hebert
Yes, it does sound like VettedCauses missed the significance of the âafter reviewing them, we will inform you of the time required to provide our responseâ.
I agree 1% high, and I wish it were lower. On the other hand, weâre specifically targeting stealth pathogens: ones where any distinctive symptoms come well after someone becomes contagious. Absent a monitoring system, you could be in a situation most people had been infected before anyone noticed there was something spreading. Flagging this sort of pathogen at 1% cumulative incidence still gives some time for rapid mitigations, though itâs definitely too late to nip it in the bud.
It helps that Iâm writing about stuff weâve discussed internally a lot! Thanks for the good questions!
I still feel confused how big a sewershed is relative to a city
A sewershed can vary dramatically in size: itâs the area that drains to some collection point (generally a treatment plant) and different cities are laid out differently. Iâm most familiar with Boston (after refreshing the MWRA Biobot Tracker intently during COVID-19) and here the main plant serves ~2M people divided between the North and South systems:
Some other cities have much smaller plants (and so smaller sewersheds), a few have larger ones.
Weâre not sure yet about the effect of size. Itâs possible that small ones are better because the waste is âfresherâ and you spend fewer of your observations (sequencing reads) on bacteria that replicates in the sewer. Or itâs possible that larger ones are better because they can support more observations (deeper sequencing).
if I read footnote 2 right, the implication is that by end of 2025, youâd aim to be able to detect a pathogen that sheds like Influenza A in cities you monitor before 2% of the population is infected?
Yes, thatâs right. Though sensitivity in practice could be higher or lower:
-
As we gather more data weâll get a better understanding of how easy or hard it is to detect Influenza A, along with other pathogens. Our influenza estimates are based on ~300 observations, but we now have the data to estimate for the 2024-2025 flu season with a lot more data. This is mostly a matter of someone taking the time to dig into it and put out an updated estimate.
-
Weâre still trying to increase sensitivity:
Testing better wet lab methods
Getting pooled airplane lavatory samples again, which have a ~20x higher human contribution
Figuring out which municipal sewersheds have the highest human contribution and focusing there
-
The projection is based on an assumption of 9d end to end time, and is relatively sensitive to timing: if your pathogen doubles every 3d then the difference between a 9d and 12d turnaround time is 2x sensitivity. Weâre currently well above 9d, but weâre on a track to get to ~7d via agreements with sequencing machine operators to reserve capacity and streamlining our processes. And then there are more expensive ways to get down to ~4d with serious investment in logistics (buy your own sequencer, run it daily, use the 10B flow cell for faster turnarounds, lab runs around the clock).
Which cities are you monitoring again?
Chicago IL, Riverside CA, and several others we hope to be able to name publicly soon.
I assume one weakness of this approach is in the geographic restrictions. Although Iâve vaguely heard of wastewater monitoring in a network of airports /â aircrafts as a way to get around this (I canât tell if thatâs just an idea right now or if itâs already being implemented, though.)
Yes, thatâs a real issue. Cosmopolitan US cities are not terrible from this perspective, especially if you have a bunch with different international connections, but theyâre still not good enough. Airplane lavatory sampling would be much better, not just because of this issue but also because (as I mentioned briefly above) theyâre much higher quality samples. Weâre working on this, but itâs much more difficult than bringing on municipal treatment plant partners.
Was the 2% threshold chosen for a particular reason?
No, itâs that 3x 25B is about the most weâre able to scale to at this stage. If we thought we could manage the scale 1% would have probably been our target, though 1% is still pretty arbitrary. Lower is better, since that means mitigations are more effective when deployed, but cost goes up dramatically as you lower your target.
-
Not a silly question, and not something where I think weâve talked about plans publicly yet. Some sort of red-teaming is something Iâd like to see us do in the second half of 2025. Most likely starting with fully computational spike-ins (much cheaper, faster to iterate on) and then real engineered viral particles.
ďScalÂing the NAOâs Stealth Pathogen Early-WarnÂing System
I object to your conclusion about present CEA
I donât think I gave any conclusion about CEA? I was pointing out that 80kâs past actions are primarily evidence about what we should expect from 80k in the future.
I think your comment is still pretty misleading: âCEA released âŚâ would be much clearer as â80k released âŚâ or perhaps â80k, at the time a sibling project of CEA, released âŚâ.
separate incident involving present CEA staff
FYI Iâm not getting into the separate incident because, as you point out, it involves my partner.
I do think there are downsides with sharing draft reviews with organizations ahead of time, but I think theyâre mostly different from the ones listed here. The biggest risk I see is that the organization could use the time to take an adversarial approach:
-
Trying to keep the review from being published. This could look like accusations of libel and threats to sue, or other kinds of retaliation (âis publishing this really in the best interest of your career...?â).
-
Preparing people to astroturf the comment section
-
Preparing a refutation that is seriously flawed but in a way that takes significant effort to investigate. This then risks turning into the opposite of the situation people usually worry about: instead of people seeing a negative review but not the orgâs follow-up with corrections they might see a negative review and a thorough refutation come out at the same time, and then never see the reviewerâs follow-up where they show that the refutation is misleading.
I also think what you list as risk 2, âUnconscious biases from interacting with charity staffâ, is a real risk. If people at an evaluator have been working with people at a charity, especially if they do this over long periods, they will naturally become more sympathetic. [1]
Of the other listed issues, however, I agree with the other commenters that theyâre avoidable:
-
There are many services for archiving web pages, and falsely claiming that archives have been tampered with is a pretty terrible strategy for a charity to take. If youâre especially concerned about this, however, you could publish your own archives of your evidence in advance (without checking with the org). The analogy to police is not a good one, because police have the ability get search warrants and learn additional things that are not already public.
-
If the charity says âVettedCausesâ review is about problems we have already addressedâ without acknowledging that they fixed the problems in response to your feedback I think that would look quite bad for them. There is risk of dispute over whether they made changes in response to your review or coincidentally, but if you give them a week to review and they claim they just happened to make the changes in that short time between their receiving the draft and you releasing it I think people would be quite skeptical.
-
On âIt is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claimsâ, I donât think youâve weighed how difficult this is. When I read through the funding appeals of even pretty careful and thoughtful charities I basically always notice claims that are not fully backed up by publicly stated evidence. While this does sound bad, organizations have a bunch of competing priorities and justifying their work to this level is rarely worth it.
Overall, I donât think these considerations appreciably change my view that you should run reviews by the orgs theyâre about.
[1] Charities can also trade access (allowing a more comprehensive evaluation) for more favorable coverage, generally not in an explicit way. I think this is related to why GiveWell and ACE have ended up with a policy that they only release reviews if charities are willing to see them released. This is a lot like access journalism. But this isnât related to whether you share drafts for review.
-
Iâm confused why youâre posting this?
Are you trying to say I should have included some sort of disclosure in my comment? Or trying to give this as an example of the kind of thing VettedCauses is worried about with sharing reviews before publication? Something else?
That sort of âitâs hard to archive things reliably long-termâ seems less relevant in the context of a review, where thereâs a pretty short time between sharing the doc with the charity and making the review public.
To the extent that you update against an org, of currently existing orgs this would be 80k, not CEA. At the time that this happened current CEA and current 80k were both independently managed efforts under the umbrella organization then known as CEA and now known as EV (more).
Separately, I agree this editing was bad, but doing it in the context of a review would be much worse.
The motivation for focusing on global catastrophic risks is that these could dramatically limit humanityâs potential. If, per your population ethics, such a limitation wouldnât be concerning, then itâs not surprising that you wouldnât find work aiming to avert or mitigate such risks compelling.
I think the post would be clearer if it were explicit about this up front: the disagreement here isnât about the relative scale of biorisk vs factory farming, but instead about how much value there is in averting civilizational collapse and/âor extinction.
I donât think impartiality to the extent of not caring more about the people one loves is a core value for very many EAs? Yes, itâs pretty central to EA that most people are excessively partial, but I donât recall ever seeing someone advocate full impartiality.