Part of this is caused by (over)use of a metric called impact-adjusted significant career plan changes. In a way, you get exactly what you optimise for. Quoting from 80k website
A typical plan change scored 10 is someone who, in large part due to us, switched to working at a highly effective organisation like GiveWell, became a major donor (>$100k/year) to effective organisations, or become a major advocate of effective causes.
A typical plan change scored 1 is someone who has taken the Giving What We Can pledge or decided to earn to give in a medium income career. We also award 1s to people who want to work on the most pressing problems and who switch to build better career capital in order to do this, for instance doing quantitative grad studies or pursuing consulting; people who have become much more involved in the effective altruism community in a way that has changed their career, and people who switch into policy or research in pressing problem areas.
A typical plan change scored 0.1 is someone shifting to gain better career capital but where they’re less obviously focused on the most pressing problems, or where they’ve switched into an option that is less obviously higher impact than what they were planning before.
Scoring the options you recommend
Skills building in non-EA organisations such as start-ups … scores either 0.1 or 1, so 10x or 100x less valuable, in comparison to changing plan to work in GiveWell
Earning to give … scores 10x less valuable
Speculative options … 10x −100x less valuable
It’s worth to emphasise the metric which is optimised is changing plans. How the difference between where someone actually switched, vs. switched just the plan, is handled, is likely inconsistent across places and orgs.
Taken literally, the best thing for a large number of people under this metric is to switch plan to working for OpenPhil, and consider other options as failure.
Taken literally, the best thing to do for student group community builders is to convince everyone to switch plans in this way, and count that as success.
So it is not a bias. Quite the opposite: it is a very literal interpretation of the objective function, which was explicitly specified several years ago.
The meta-level point people should take from this is:
If you are in a position of influence, you should be super-careful before you introduce anything like a quantitative metric into EA culture. EAs love measuring impact, are optimisers, and will Goodhart hard
I think IASPCs handle these things well, and think there’s some misinterpretation going on. What makes a strong plan change under this metric is determined by whatever 80,000 Hours thinks is most important, and currently this includes academic, industry, EA org and government roles. These priorities also change in response to new information and needs. The problem Sebastian is worried about seems more of a big deal: maybe some orgs / local groups are defining their metrics mostly in terms of one category, or that it’s easy to naively optimise for one category at the expense of the others.
The part about counting impact from skill-building and direct work differently simply seems like correct accounting: EA orgs should credit themselves with substantially more impact for a plan change which has led to impact as one which might do so in the future, most obviously because the latter has a <100% probability of turning into the former.
I also think the metric works fine with Sebastian’s point that quant trading can be competitive with other priority paths. You seem to imply that the use of IASPCs contradicts his advice, but you point to a non-priority rating for ‘earn to give in a medium income career’, which is not quant trading!† 80,000 Hours explicitly list quant trading as a priority path (as Seb pointed out in the post), so if an org uses IASPCs as one of their metrics they should be excited to see people with those particular skills go down that route. (If any readers land quant jobs in London, please do say hi :) )
I agree that misapplication of this or similar metrics is dangerous, and that if e.g. some local groups are just optimising for EA-branded orgs instead of at least the full swathe of priority paths, there’s a big opportunity to improve. All the normal caveats about using metrics sensibly continue to apply.
All views my own.
†As a former trader, I felt the need to put an exclamation mark somewhere in this paragraph.
Ultimately the more you ground the metric in “what some sensible people thing is important and makes sense right now”, the more nuance it has, and the more is it tracking reality. The text in my quote is verbatim copy from the page describing the metric from 2015, so I think it’s highly relevant for understanding how IASPCs were understood. I agree that 80k career guides as a whole actually has much more nuance, and suggests approach like “figure out what will be needed in future and prepare for that”.
The whole accounting still seems wrong: per definition, what’s counted is … caused them to change the career path they intend to pursue, … ; this is still several steps away from impact: if someone changes their intentions to pursue jobs in EA orgs, it is counted as impact, even if the fraction of the people making such plans who will succeed is low.
For specificity, would you agree that someone who was 2 years away from graduation in 2016, deciding to change career plan to pursuing a job in CEA, would have been counted as impact 10, while someone switching from a plan going to industry to pursuing PhD in econ would have been counted as 1, and someone deciding to stay in, let’s say, cognitive neuroscience, would have been counted as 0?
I just wrote a bit more about how we measure IASPCs in another comment on this thread. We don’t use a precise formula and the details are important so I can’t say exactly how we’d rate a particular change at this level of generality.
That said, we take into account someone’s degree of follow through when we score their plan change, such that very few of our highest rated plan changes (rated-100 or more) are from people who are not currently doing impactful work.
30% have changed their plans but not yet passed a major “milestone” in their shift. Most of these people have applied to a new graduate programme but not yet received an offer.
30% have reached a milestone, but are still building career capital (e.g. entered graduate school, or taken a high-earning job but not yet donated much).
40% have already started having an impact (e.g. have published research, taken a non-profit job).
I agree if we didn’t take follow through into account it would lead to some scores that were far removed from expected impact such as the hypothetical you’ve described.
thanks very much for the clarifying discussion. The fact that there is this discussion (also looking at the high number of votes for the comments) illustrates that there is at least some confusion around rating EA org vs. non-EA org careers, which is a bit concerning in itself.
FWIW my original claim was not that people (neither 80k nor community members) get the rational analysis part wrong. And a career path where actual impact is a few years off should totally get a reduced expected value & rating. (My claim in the initial post is that many of the other paths are still competitive with EA org roles.) There is little actual disagreement that quant trading is a great career.
My worry is that many soft factors may cause people to develop preferences that are not in line with the EV reasoning, and that may reduce motivation and/or lead to people overly focused on jobs at explicit EA employers.
Also, you lack a ‘stamp of approval’ from 80k when you pursue some of these careers that you kind of don’t need when doing a ‘standard’ path like working at CEA/FHI/80k/OPP or do a top ML PhD, even if all of them were rated 10. (In coaching days this was better, because you could just tell your doubting student group leader that this is what 80k wants you to do :) )
I’m sympathetic to your comment. The fact that (I think) 80k is not making this particular mistake in its IASPC system does not imply that there’s nothing to be concerned about. I think your post as well as some of the comments in other threads do a good job of laying out many of the factors pushing people toward jobs at explicitly EA orgs.
Hi Jan, thanks for your thoughts. Kit’s response is fairly close to our views.
The most important thing we want to emphasize is that at 80,000 Hours we definitely don’t think that working at an EA org is the only valuable thing for people to do. I think that taken as a whole, our writing reflects that.
The best way to quickly get sense of our views is reading through our high impact careers article, especially the list of 10 priority paths. Only one of these is working at an EA org.
I think our job board, problem profiles, podcast and so on give a similar sense of how much we value people working outside EA orgs.
A second key point is that when we score plan changes, we do not have a strict formula. We score changes based on our overall views of which paths are high-impact and assess many of the plan changes, especially the larger ones, on an individual basis, rather than simply putting them in a category. As an approximation, those we most prioritise are those represented by the 10 priority paths.
Of our top rated plan changes, only 25% involve people working at EA orgs
Fortunately, scoring on a case by case basis makes our scoring less vulnerable to Goodharting. Unfortunately, it means that it’s difficult for us to communicate exactly how we score plan changes to others. When we do so, it’s generally a few sentences, which are just aimed at giving people a sense of how impactful 80,000 Hours is as an organisation. These explanations are not intended to be career advice and it would be a shame if people have been taking them as such.
The specific sentences you quote are a bit out of date and we explain the categories differently in a draft of our annual review, which we hope to publish in the coming months. For example, we often score a plan change as rated-10 if somebody takes up a particularly valuable skill-building opportunity within one of our priority paths.
Of our top rated plan changes, only 25% involve people working at EA orgs
For what it’s worth, given how few EA orgs there are in relation to the number of highly dedicated EAs and how large the world outside of EA is (e.g. in terms of institutions/orgs that work in important areas or are reasonably good at teaching important skills), 25% actually strikes me as a high figure. Even if this was right, there might be good reasons for the figure being that high, e.g. it’s natural and doesn’t necessarily reflect any mistake that 80K knows more about which careers at EA orgs are high-impact, can do a better job at finding people for them etc. However, I would be surprised if as the EA movement becomes more mature the optimal proportion was as high.
(I didn’t read your comment as explicitly agreeing or disagreeing with anything in the above paragraph, just wanted to share my intuitive reaction.)
Thank you for your comments here, they’ve helped me understand 80K’s current thinking on the issue raised by the OP.
Thanks for the thoughts, Max. As you suggest in your parenthetical, we aren’t saying that 25% of the community ought to be working at EA orgs. The distribution of the plan changes we cause is also affected by things like our network being strongest within EA. That figure is also calculated from a fairly small number of our highest impact plan changes so it could easily change a lot over time.
Personally, I agree with your take that the optimal percentage of the community working at EA orgs is less than 25%.
To clarify the concern, I’m generally not much more worried about how you use it internally, but about other people using the metric. It was probably not clear from my comment.
I understand it was probably never intended as something which other should use either for guiding their decisions or evaluating their efforts.
Have you thought about making this into a post? This is the first I’ve heard about this and find it really compelling and interesting and totally worth a larger discussion.
Part of this is caused by (over)use of a metric called impact-adjusted significant career plan changes. In a way, you get exactly what you optimise for. Quoting from 80k website
Scoring the options you recommend
Skills building in non-EA organisations such as start-ups … scores either 0.1 or 1, so 10x or 100x less valuable, in comparison to changing plan to work in GiveWell
Earning to give … scores 10x less valuable
Speculative options … 10x −100x less valuable
It’s worth to emphasise the metric which is optimised is changing plans. How the difference between where someone actually switched, vs. switched just the plan, is handled, is likely inconsistent across places and orgs.
Taken literally, the best thing for a large number of people under this metric is to switch plan to working for OpenPhil, and consider other options as failure.
Taken literally, the best thing to do for student group community builders is to convince everyone to switch plans in this way, and count that as success.
So it is not a bias. Quite the opposite: it is a very literal interpretation of the objective function, which was explicitly specified several years ago.
The meta-level point people should take from this is:
If you are in a position of influence, you should be super-careful before you introduce anything like a quantitative metric into EA culture. EAs love measuring impact, are optimisers, and will Goodhart hard
I think IASPCs handle these things well, and think there’s some misinterpretation going on. What makes a strong plan change under this metric is determined by whatever 80,000 Hours thinks is most important, and currently this includes academic, industry, EA org and government roles. These priorities also change in response to new information and needs. The problem Sebastian is worried about seems more of a big deal: maybe some orgs / local groups are defining their metrics mostly in terms of one category, or that it’s easy to naively optimise for one category at the expense of the others.
The part about counting impact from skill-building and direct work differently simply seems like correct accounting: EA orgs should credit themselves with substantially more impact for a plan change which has led to impact as one which might do so in the future, most obviously because the latter has a <100% probability of turning into the former.
I also think the metric works fine with Sebastian’s point that quant trading can be competitive with other priority paths. You seem to imply that the use of IASPCs contradicts his advice, but you point to a non-priority rating for ‘earn to give in a medium income career’, which is not quant trading!† 80,000 Hours explicitly list quant trading as a priority path (as Seb pointed out in the post), so if an org uses IASPCs as one of their metrics they should be excited to see people with those particular skills go down that route. (If any readers land quant jobs in London, please do say hi :) )
I agree that misapplication of this or similar metrics is dangerous, and that if e.g. some local groups are just optimising for EA-branded orgs instead of at least the full swathe of priority paths, there’s a big opportunity to improve. All the normal caveats about using metrics sensibly continue to apply.
All views my own.
†As a former trader, I felt the need to put an exclamation mark somewhere in this paragraph.
Ultimately the more you ground the metric in “what some sensible people thing is important and makes sense right now”, the more nuance it has, and the more is it tracking reality. The text in my quote is verbatim copy from the page describing the metric from 2015, so I think it’s highly relevant for understanding how IASPCs were understood. I agree that 80k career guides as a whole actually has much more nuance, and suggests approach like “figure out what will be needed in future and prepare for that”.
The whole accounting still seems wrong: per definition, what’s counted is … caused them to change the career path they intend to pursue, … ; this is still several steps away from impact: if someone changes their intentions to pursue jobs in EA orgs, it is counted as impact, even if the fraction of the people making such plans who will succeed is low.
For specificity, would you agree that someone who was 2 years away from graduation in 2016, deciding to change career plan to pursuing a job in CEA, would have been counted as impact 10, while someone switching from a plan going to industry to pursuing PhD in econ would have been counted as 1, and someone deciding to stay in, let’s say, cognitive neuroscience, would have been counted as 0?
Hi Jan,
I just wrote a bit more about how we measure IASPCs in another comment on this thread. We don’t use a precise formula and the details are important so I can’t say exactly how we’d rate a particular change at this level of generality.
That said, we take into account someone’s degree of follow through when we score their plan change, such that very few of our highest rated plan changes (rated-100 or more) are from people who are not currently doing impactful work.
Of the rated 10’s, our analysis in 2017 found:
30% have changed their plans but not yet passed a major “milestone” in their shift. Most of these people have applied to a new graduate programme but not yet received an offer.
30% have reached a milestone, but are still building career capital (e.g. entered graduate school, or taken a high-earning job but not yet donated much).
40% have already started having an impact (e.g. have published research, taken a non-profit job).
I agree if we didn’t take follow through into account it would lead to some scores that were far removed from expected impact such as the hypothetical you’ve described.
Hope this clarifies things.
Hey Jan and Howie,
thanks very much for the clarifying discussion. The fact that there is this discussion (also looking at the high number of votes for the comments) illustrates that there is at least some confusion around rating EA org vs. non-EA org careers, which is a bit concerning in itself.
FWIW my original claim was not that people (neither 80k nor community members) get the rational analysis part wrong. And a career path where actual impact is a few years off should totally get a reduced expected value & rating. (My claim in the initial post is that many of the other paths are still competitive with EA org roles.) There is little actual disagreement that quant trading is a great career.
My worry is that many soft factors may cause people to develop preferences that are not in line with the EV reasoning, and that may reduce motivation and/or lead to people overly focused on jobs at explicit EA employers.
Also, you lack a ‘stamp of approval’ from 80k when you pursue some of these careers that you kind of don’t need when doing a ‘standard’ path like working at CEA/FHI/80k/OPP or do a top ML PhD, even if all of them were rated 10. (In coaching days this was better, because you could just tell your doubting student group leader that this is what 80k wants you to do :) )
Hey Sebastian,
I’m sympathetic to your comment. The fact that (I think) 80k is not making this particular mistake in its IASPC system does not imply that there’s nothing to be concerned about. I think your post as well as some of the comments in other threads do a good job of laying out many of the factors pushing people toward jobs at explicitly EA orgs.
Hi Jan, thanks for your thoughts. Kit’s response is fairly close to our views.
The most important thing we want to emphasize is that at 80,000 Hours we definitely don’t think that working at an EA org is the only valuable thing for people to do. I think that taken as a whole, our writing reflects that.
The best way to quickly get sense of our views is reading through our high impact careers article, especially the list of 10 priority paths. Only one of these is working at an EA org.
I think our job board, problem profiles, podcast and so on give a similar sense of how much we value people working outside EA orgs.
A second key point is that when we score plan changes, we do not have a strict formula. We score changes based on our overall views of which paths are high-impact and assess many of the plan changes, especially the larger ones, on an individual basis, rather than simply putting them in a category. As an approximation, those we most prioritise are those represented by the 10 priority paths.
Of our top rated plan changes, only 25% involve people working at EA orgs
Fortunately, scoring on a case by case basis makes our scoring less vulnerable to Goodharting. Unfortunately, it means that it’s difficult for us to communicate exactly how we score plan changes to others. When we do so, it’s generally a few sentences, which are just aimed at giving people a sense of how impactful 80,000 Hours is as an organisation. These explanations are not intended to be career advice and it would be a shame if people have been taking them as such.
The specific sentences you quote are a bit out of date and we explain the categories differently in a draft of our annual review, which we hope to publish in the coming months. For example, we often score a plan change as rated-10 if somebody takes up a particularly valuable skill-building opportunity within one of our priority paths.
I hope that helps answer your concern!
For what it’s worth, given how few EA orgs there are in relation to the number of highly dedicated EAs and how large the world outside of EA is (e.g. in terms of institutions/orgs that work in important areas or are reasonably good at teaching important skills), 25% actually strikes me as a high figure. Even if this was right, there might be good reasons for the figure being that high, e.g. it’s natural and doesn’t necessarily reflect any mistake that 80K knows more about which careers at EA orgs are high-impact, can do a better job at finding people for them etc. However, I would be surprised if as the EA movement becomes more mature the optimal proportion was as high.
(I didn’t read your comment as explicitly agreeing or disagreeing with anything in the above paragraph, just wanted to share my intuitive reaction.)
Thank you for your comments here, they’ve helped me understand 80K’s current thinking on the issue raised by the OP.
Thanks for the thoughts, Max. As you suggest in your parenthetical, we aren’t saying that 25% of the community ought to be working at EA orgs. The distribution of the plan changes we cause is also affected by things like our network being strongest within EA. That figure is also calculated from a fairly small number of our highest impact plan changes so it could easily change a lot over time.
Personally, I agree with your take that the optimal percentage of the community working at EA orgs is less than 25%.
To clarify the concern, I’m generally not much more worried about how you use it internally, but about other people using the metric. It was probably not clear from my comment.
I understand it was probably never intended as something which other should use either for guiding their decisions or evaluating their efforts.
Have you thought about making this into a post? This is the first I’ve heard about this and find it really compelling and interesting and totally worth a larger discussion.