Again, I’ll give a pretty brief answer since this is a question about broader 80,000 Hours strategy rather than advising specifically.
Around the end of the year we do an annual impact review. At that point we follow up with people we think have made a significant plan change, and people who we knew about from the past who had made large plan changes to see how those panned out. For cases where it seemed we made a particularly large impact we’ve continued following up for years, in order to update how much impact the plan change had. It’s not viable for us to do this with everyone who reports any kind of change (our impact survey gets more than 1000 answers each year), so we just do it with the cases where we seemed to have had the most counterfactual impact on the person’s career (as opposed to, say, the people who just say they read something on our website and it made them slightly more likely to follow a different path)
Thanks! Makes sense that 80k would only do this for “rated-1o” and “rated-1oo” plan changes.
For cases where it seemed we made a particularly large impact we’ve continued following up for years, in order to update how much impact the plan change had.
Is data about these longer term follow-ups publicly available somewhere? Didn’t see it in my quick read of the 2018 review.
I would expect not, since it would be hard to give much information which isn’t identifiable to individuals. The longer term follow up is factored into our overall impact numbers though, so in that sense it is.
You might be interested in this section though, which says how many plan changes were rated 10 in previous years, but have subsequently been downgraded.
Does 80k do longterm follow-up with folks who’ve attributed an impact-adjusted significant plan change (IASPC) to 80k advice?
I’m imagining following up 12 months later (and also 24 months later, 36 months later, if ambitious), to see:
how things are going after the change
if they still think the change was a good idea
if they still attribute the change to the same factors
etc.
Again, I’ll give a pretty brief answer since this is a question about broader 80,000 Hours strategy rather than advising specifically.
Around the end of the year we do an annual impact review. At that point we follow up with people we think have made a significant plan change, and people who we knew about from the past who had made large plan changes to see how those panned out. For cases where it seemed we made a particularly large impact we’ve continued following up for years, in order to update how much impact the plan change had. It’s not viable for us to do this with everyone who reports any kind of change (our impact survey gets more than 1000 answers each year), so we just do it with the cases where we seemed to have had the most counterfactual impact on the person’s career (as opposed to, say, the people who just say they read something on our website and it made them slightly more likely to follow a different path)
Thanks! Makes sense that 80k would only do this for “rated-1o” and “rated-1oo” plan changes.
Is data about these longer term follow-ups publicly available somewhere? Didn’t see it in my quick read of the 2018 review.
I would expect not, since it would be hard to give much information which isn’t identifiable to individuals. The longer term follow up is factored into our overall impact numbers though, so in that sense it is.
You might be interested in this section though, which says how many plan changes were rated 10 in previous years, but have subsequently been downgraded.
Thanks!
Those are plan changes that have been downgraded after 80k learned more about the situation?