I’d be very interested in seeing a continuation in regards to outcomes (maybe career changes could be a proxy for impact?)
Yes, I think career changes and additional effective donations would be better proxies for impact than outputs like quality-adjusted attendances and calls. Relatedly:
Animal Advocacy Careers (AAC) ran two longitudinal studies aiming to compare and test the cost-effectiveness of our one-to-one advising calls and our online course. Various forms of these two types of careers advice service have been used by people seeking to build the effective altruism (EA) movement for years, and we expect the results to be informative to EA movement builders, as well as to AAC.
We interpret the results as tentative evidence of positive effects from both services, but the effects of each seem to be different. Which is more effective overall depends on your views about which sorts of effects are most important; our guess is that one-to-one calls are slightly more effective per participant, but not by much. One-to-one calls seem substantially more costly per participant, which makes the service harder to scale.
There therefore seems to be a tradeoff between costs and apparent effects per participant. We’d guess that the online course was (and will be, once scaled up) slightly more cost-effective, all things considered, but the services might just serve different purposes, especially since the applicants might be different for the different services.
Also, curious how you would think about the added value of a career call or participation in a program? Given that a person made a career change, obviously the career call with 80k isn’t 100% responsible for the change, but probably not 0% either (if the call was successful).
AAC’s studies had a control group, so they provide evidence about the counterfactual impact of their one-to-one advising calls and online course. 80,000 Hours’ has a metric called discounted impact-adjusted peak years (DIPYs) which accounts for which fraction of the career change was caused by them.
Thanks for the comment, Ezrah!
Yes, I think career changes and additional effective donations would be better proxies for impact than outputs like quality-adjusted attendances and calls. Relatedly:
AAC’s studies had a control group, so they provide evidence about the counterfactual impact of their one-to-one advising calls and online course. 80,000 Hours’ has a metric called discounted impact-adjusted peak years (DIPYs) which accounts for which fraction of the career change was caused by them.