When implementing their career advice program, for example, AAC pre-registered a study to evaluate the effects of career advising calls on animals because of a lack of systematic evidence for this intervention (5). This study would not only help AAC evaluate their impact, but provide valuable data to other EA meta charities.
Yeah, this sounds useful. Currently my independent impression is that it’s weird that EA orgs don’t do (lower-effort) versions that sort of thing more often—e.g., for a subset of applicants to receive advice or go through a program or whatever, randomise who gets it, then send out surveys afterwards to both groups. (I think this has been discussed—maybe including by me? - in various places before, but I can’t remember where.)
But maybe there are good reasons I’m not aware of. And I know EA orgs do often at least do things like sending out surveys, just not necessarily hearing from people they didn’t end up giving the “treatment” to with (for comparison) nor randomising who gets the “treatment”.
Thanks, I found this post interesting.
Yeah, this sounds useful. Currently my independent impression is that it’s weird that EA orgs don’t do (lower-effort) versions that sort of thing more often—e.g., for a subset of applicants to receive advice or go through a program or whatever, randomise who gets it, then send out surveys afterwards to both groups. (I think this has been discussed—maybe including by me? - in various places before, but I can’t remember where.)
But maybe there are good reasons I’m not aware of. And I know EA orgs do often at least do things like sending out surveys, just not necessarily hearing from people they didn’t end up giving the “treatment” to with (for comparison) nor randomising who gets the “treatment”.