Overall this seems like a sensible, and appropriately skeptical, way of using LLM’s in this sort of work.
In regards to improving the actual AI output, it looks like there is insufficient sourcing of claims in what it puts out, which is going to slow you down when you actually try and check the output. I’m looking at the red team output here on water turpidity. This was highlighted as a real contribution by the AI, but the output has zero sourcing on it’s claims, which presumably made it much harder to actually check for validity. If you were to get this critique from a real, human, red-teamer, they would make it signficantly more easy to check that the critique was valid and sourced.
One question I have to ask is whether you are measuring how much time and effort is being extended into managing the output of these LLM’s and sifting out the actually useful recommendations? When assessing whether the techniques are a success, you have to consider the counterfactual case where that time was replaced by human research time looking more closely at the literature, for example.
On your first point, if researchers found any of the critiques raised by the AI to be credible, we would follow up in the same chat to ask for sourcing or additional information (like for the water turbidity point). We found this more successful than asking for heavy citations in the initial output, which led to lower quality critiques (presumably because we were giving too many instructions). The most recent generation of models are better at following complex instructions, so I expect we should revisit this.
Human researchers spent ~75 minutes reviewing AI output per intervention. We haven’t rigorously compared this to having researchers look into the literature themselves, but my sense is that currently using AI is worth it for areas where we haven’t done as much research (family planning, syphilis treatment) and not worth it in areas where we’ve done a lot of research (bed nets, vaccinations). Given the pace of improvement we’ve seen over the past year, I expect a preliminary AI red teaming pass will be worthwhile in almost all cases by the end of 2026.
Overall this seems like a sensible, and appropriately skeptical, way of using LLM’s in this sort of work.
In regards to improving the actual AI output, it looks like there is insufficient sourcing of claims in what it puts out, which is going to slow you down when you actually try and check the output. I’m looking at the red team output here on water turpidity. This was highlighted as a real contribution by the AI, but the output has zero sourcing on it’s claims, which presumably made it much harder to actually check for validity. If you were to get this critique from a real, human, red-teamer, they would make it signficantly more easy to check that the critique was valid and sourced.
One question I have to ask is whether you are measuring how much time and effort is being extended into managing the output of these LLM’s and sifting out the actually useful recommendations? When assessing whether the techniques are a success, you have to consider the counterfactual case where that time was replaced by human research time looking more closely at the literature, for example.
Thanks for the feedback!
On your first point, if researchers found any of the critiques raised by the AI to be credible, we would follow up in the same chat to ask for sourcing or additional information (like for the water turbidity point). We found this more successful than asking for heavy citations in the initial output, which led to lower quality critiques (presumably because we were giving too many instructions). The most recent generation of models are better at following complex instructions, so I expect we should revisit this.
Human researchers spent ~75 minutes reviewing AI output per intervention. We haven’t rigorously compared this to having researchers look into the literature themselves, but my sense is that currently using AI is worth it for areas where we haven’t done as much research (family planning, syphilis treatment) and not worth it in areas where we’ve done a lot of research (bed nets, vaccinations). Given the pace of improvement we’ve seen over the past year, I expect a preliminary AI red teaming pass will be worthwhile in almost all cases by the end of 2026.