An important upcoming study that I read about is a robustness study or RCT on the effectiveness of AMF or distributions of malaria nets.
(I think I read about this study in a post here. I am typing in mobile and can’t immediately find/post a link).
This RCT or result would be really important to EAs and maybe EA in general. Many people have donated a lot to AMF. The act of donating, the belief in AMF, as well as the method and process used, is part of EA identity.
Maybe EAs should not be worried about finding the underlying truth as a result of the study, but maybe there should be worry or attention to clunky or simplistic presentation or socialization of the results.
For example, there are many reasons to see superficially negative results (that suggest cost inefficiency or ineffectiveness of nets) but that doesn’t reflect a reality that that undermines AMF. For example, just unmeasured rainfall, or noise from any number of sources, could cause an issue in seeing an effect (issues with statistical inference or identification)
I think explaining causal inference beforehand (as opposed to after) might be useful.
A reasonable guess is that an RCT might only be 50% reliable (including claiming to be reliable but having a hidden defect). The chance of some negative event from this study (other than the consequence of showing the truth) might be ~5-15%.
It seems valuable, but it doesn’t seem to be an RCT. I can’t immediately tell what it is but it looks like collecting trend data without a control group. (To onlookers, I know that sounds frowned upon but it’s a real thing and probably judging the value of the design requires great domain knowledge.)
So it looks a lot less pivotal than an “RCT on AMF”.
So my original answer above might have been misleading.
EDIT: Contrary to the text below, the study is not an RCT. Instead there is a GiveWell funded study collecting and exploring trend data for LLIN (malaria nets). https://www.givewell.org/research/incubation-grants/Malaria-Consortium-monitoring-Ondo-July-2021#Risks_and_reservations
An important upcoming study that I read about is a robustness study or RCT on the effectiveness of AMF or distributions of malaria nets.
(I think I read about this study in a post here. I am typing in mobile and can’t immediately find/post a link).
This RCT or result would be really important to EAs and maybe EA in general. Many people have donated a lot to AMF. The act of donating, the belief in AMF, as well as the method and process used, is part of EA identity.
Maybe EAs should not be worried about finding the underlying truth as a result of the study, but maybe there should be worry or attention to clunky or simplistic presentation or socialization of the results.
For example, there are many reasons to see superficially negative results (that suggest cost inefficiency or ineffectiveness of nets) but that doesn’t reflect a reality that that undermines AMF. For example, just unmeasured rainfall, or noise from any number of sources, could cause an issue in seeing an effect (issues with statistical inference or identification)
I think explaining causal inference beforehand (as opposed to after) might be useful.
A reasonable guess is that an RCT might only be 50% reliable (including claiming to be reliable but having a hidden defect). The chance of some negative event from this study (other than the consequence of showing the truth) might be ~5-15%.
This seems very relevant; if you find the name/link, please do add it in the Airtable or just here in the comments. Thanks.
Ok I think I found my “source”: https://www.givewell.org/research/incubation-grants/Malaria-Consortium-monitoring-Ondo-July-2021
It seems valuable, but it doesn’t seem to be an RCT. I can’t immediately tell what it is but it looks like collecting trend data without a control group. (To onlookers, I know that sounds frowned upon but it’s a real thing and probably judging the value of the design requires great domain knowledge.)
So it looks a lot less pivotal than an “RCT on AMF”.
So my original answer above might have been misleading.