Ultimately the more you ground the metric in “what some sensible people thing is important and makes sense right now”, the more nuance it has, and the more is it tracking reality. The text in my quote is verbatim copy from the page describing the metric from 2015, so I think it’s highly relevant for understanding how IASPCs were understood. I agree that 80k career guides as a whole actually has much more nuance, and suggests approach like “figure out what will be needed in future and prepare for that”.
The whole accounting still seems wrong: per definition, what’s counted is … caused them to change the career path they intend to pursue, … ; this is still several steps away from impact: if someone changes their intentions to pursue jobs in EA orgs, it is counted as impact, even if the fraction of the people making such plans who will succeed is low.
For specificity, would you agree that someone who was 2 years away from graduation in 2016, deciding to change career plan to pursuing a job in CEA, would have been counted as impact 10, while someone switching from a plan going to industry to pursuing PhD in econ would have been counted as 1, and someone deciding to stay in, let’s say, cognitive neuroscience, would have been counted as 0?
I just wrote a bit more about how we measure IASPCs in another comment on this thread. We don’t use a precise formula and the details are important so I can’t say exactly how we’d rate a particular change at this level of generality.
That said, we take into account someone’s degree of follow through when we score their plan change, such that very few of our highest rated plan changes (rated-100 or more) are from people who are not currently doing impactful work.
30% have changed their plans but not yet passed a major “milestone” in their shift. Most of these people have applied to a new graduate programme but not yet received an offer.
30% have reached a milestone, but are still building career capital (e.g. entered graduate school, or taken a high-earning job but not yet donated much).
40% have already started having an impact (e.g. have published research, taken a non-profit job).
I agree if we didn’t take follow through into account it would lead to some scores that were far removed from expected impact such as the hypothetical you’ve described.
thanks very much for the clarifying discussion. The fact that there is this discussion (also looking at the high number of votes for the comments) illustrates that there is at least some confusion around rating EA org vs. non-EA org careers, which is a bit concerning in itself.
FWIW my original claim was not that people (neither 80k nor community members) get the rational analysis part wrong. And a career path where actual impact is a few years off should totally get a reduced expected value & rating. (My claim in the initial post is that many of the other paths are still competitive with EA org roles.) There is little actual disagreement that quant trading is a great career.
My worry is that many soft factors may cause people to develop preferences that are not in line with the EV reasoning, and that may reduce motivation and/or lead to people overly focused on jobs at explicit EA employers.
Also, you lack a ‘stamp of approval’ from 80k when you pursue some of these careers that you kind of don’t need when doing a ‘standard’ path like working at CEA/FHI/80k/OPP or do a top ML PhD, even if all of them were rated 10. (In coaching days this was better, because you could just tell your doubting student group leader that this is what 80k wants you to do :) )
I’m sympathetic to your comment. The fact that (I think) 80k is not making this particular mistake in its IASPC system does not imply that there’s nothing to be concerned about. I think your post as well as some of the comments in other threads do a good job of laying out many of the factors pushing people toward jobs at explicitly EA orgs.
Ultimately the more you ground the metric in “what some sensible people thing is important and makes sense right now”, the more nuance it has, and the more is it tracking reality. The text in my quote is verbatim copy from the page describing the metric from 2015, so I think it’s highly relevant for understanding how IASPCs were understood. I agree that 80k career guides as a whole actually has much more nuance, and suggests approach like “figure out what will be needed in future and prepare for that”.
The whole accounting still seems wrong: per definition, what’s counted is … caused them to change the career path they intend to pursue, … ; this is still several steps away from impact: if someone changes their intentions to pursue jobs in EA orgs, it is counted as impact, even if the fraction of the people making such plans who will succeed is low.
For specificity, would you agree that someone who was 2 years away from graduation in 2016, deciding to change career plan to pursuing a job in CEA, would have been counted as impact 10, while someone switching from a plan going to industry to pursuing PhD in econ would have been counted as 1, and someone deciding to stay in, let’s say, cognitive neuroscience, would have been counted as 0?
Hi Jan,
I just wrote a bit more about how we measure IASPCs in another comment on this thread. We don’t use a precise formula and the details are important so I can’t say exactly how we’d rate a particular change at this level of generality.
That said, we take into account someone’s degree of follow through when we score their plan change, such that very few of our highest rated plan changes (rated-100 or more) are from people who are not currently doing impactful work.
Of the rated 10’s, our analysis in 2017 found:
30% have changed their plans but not yet passed a major “milestone” in their shift. Most of these people have applied to a new graduate programme but not yet received an offer.
30% have reached a milestone, but are still building career capital (e.g. entered graduate school, or taken a high-earning job but not yet donated much).
40% have already started having an impact (e.g. have published research, taken a non-profit job).
I agree if we didn’t take follow through into account it would lead to some scores that were far removed from expected impact such as the hypothetical you’ve described.
Hope this clarifies things.
Hey Jan and Howie,
thanks very much for the clarifying discussion. The fact that there is this discussion (also looking at the high number of votes for the comments) illustrates that there is at least some confusion around rating EA org vs. non-EA org careers, which is a bit concerning in itself.
FWIW my original claim was not that people (neither 80k nor community members) get the rational analysis part wrong. And a career path where actual impact is a few years off should totally get a reduced expected value & rating. (My claim in the initial post is that many of the other paths are still competitive with EA org roles.) There is little actual disagreement that quant trading is a great career.
My worry is that many soft factors may cause people to develop preferences that are not in line with the EV reasoning, and that may reduce motivation and/or lead to people overly focused on jobs at explicit EA employers.
Also, you lack a ‘stamp of approval’ from 80k when you pursue some of these careers that you kind of don’t need when doing a ‘standard’ path like working at CEA/FHI/80k/OPP or do a top ML PhD, even if all of them were rated 10. (In coaching days this was better, because you could just tell your doubting student group leader that this is what 80k wants you to do :) )
Hey Sebastian,
I’m sympathetic to your comment. The fact that (I think) 80k is not making this particular mistake in its IASPC system does not imply that there’s nothing to be concerned about. I think your post as well as some of the comments in other threads do a good job of laying out many of the factors pushing people toward jobs at explicitly EA orgs.