I (Jacy) was asked a good question privately that I wanted to log my answer to here, about how our RCT approach compares with that of academic social science RCTs, which I also discussed some in my response to Jay.
While there are many features of academic social science research we hope to emulate, e.g. peer review, I think academia also has a lot of issues that we want to avoid. For example, some good science practices, e.g. preregistration, are still uncommon in academia and there are strong incentives other than scientific accuracy, e.g. publish or perish, that we hope to minimize. I’d venture a speculative guess that the RCTs ran by nonprofit researchers in the EA community, e.g. the Mercy For Animals online ads RCT, are higher-quality than most academic RCTs. The most recurrent issue in EA RCTs is low sample size, which seems like more of a funding issue than a skillset/approach issue. (It could be a skillset/approach issue in some ways, e.g. if EA nonprofits should be running fewer RCTs so they can get the higher sample size on the same budget, which I tentatively agree with and think is the current trend.)
With our Research Network, we’re definitely happy to support high-quality academic research. We’d also be happy to hire academics interested in switching to nonprofit research, though we worry that few would be willing to work for the relatively low salaries.
In terms of communicating our research, our lack of PhDs and academic appointments on staff has been at the top of our list of concerns. Unfortunately there’s just not a good fix available. Ideally, once we’re able to make our first hire, we’d find a PhD who’s willing to work for a nonprofit EA salary, but that seems unlikely. We do already have PhDs/academics in our advisory/review network. I’ve also considered personally going back to school for a PhD, but everyone I’ve consulted with thinks this wouldn’t be worth the time cost.
I (Jacy) was asked a good question privately that I wanted to log my answer to here, about how our RCT approach compares with that of academic social science RCTs, which I also discussed some in my response to Jay.
While there are many features of academic social science research we hope to emulate, e.g. peer review, I think academia also has a lot of issues that we want to avoid. For example, some good science practices, e.g. preregistration, are still uncommon in academia and there are strong incentives other than scientific accuracy, e.g. publish or perish, that we hope to minimize. I’d venture a speculative guess that the RCTs ran by nonprofit researchers in the EA community, e.g. the Mercy For Animals online ads RCT, are higher-quality than most academic RCTs. The most recurrent issue in EA RCTs is low sample size, which seems like more of a funding issue than a skillset/approach issue. (It could be a skillset/approach issue in some ways, e.g. if EA nonprofits should be running fewer RCTs so they can get the higher sample size on the same budget, which I tentatively agree with and think is the current trend.)
With our Research Network, we’re definitely happy to support high-quality academic research. We’d also be happy to hire academics interested in switching to nonprofit research, though we worry that few would be willing to work for the relatively low salaries.
In terms of communicating our research, our lack of PhDs and academic appointments on staff has been at the top of our list of concerns. Unfortunately there’s just not a good fix available. Ideally, once we’re able to make our first hire, we’d find a PhD who’s willing to work for a nonprofit EA salary, but that seems unlikely. We do already have PhDs/academics in our advisory/review network. I’ve also considered personally going back to school for a PhD, but everyone I’ve consulted with thinks this wouldn’t be worth the time cost.