This makes me think of the contrast between systems analysis and net assessment/strategy. Yes, Fermi calculations are a valuable input into the discussion, but the nature of the problem is likely too complex to give sole weight to that calculation (in most circumstances—I think your example of comparing different research papers is a relatively simple/constrained environment).
In net assessment/strategy, the nature of the choice and your assessment of that nature determine the best methods of analysis. In systems analysis, a one-size-fits-all approach of measurement and analysis is taken. In certain circumstances that method of measurement is hugely valuable (why it became so popular in the US DoD), but in many other circumstances (like the Vietnam War) that method is deeply flawed.
It takes a solid assessment of the problem space to identify the key aspects of the competition. From there you can assess which methods and measures are most useful to judge the decision upon. And from there you can search for the data available to match your assessment.
Although choosing a career isn’t a competition against a competitor, your search to do the most good is a clearer objective than most adversaries can articulate regarding their objectives for long-term competition. The same meta-methods of analysis (to determine the best methods of analysis to fit the problem) apply. And then you can go on to finding the right data to fit those methods you’ve deemed most effective (maybe a Fermi calculation, but maybe 5 expert opinions in your career field or maybe your own gut feeling). I also think that in a career decision (unless it’s super major and clearly between two choices), the data you are going to be looking for is closer to a startup’s lean experiment than a longitudinal study. As you mentioned, Fermi calculations being readily available and usable doesn’t mean that is an effective tool for your decision.
Looking back at my answer, this didn’t really answer your question. The most effective technique I have found for deciding career choices though is to talk to as many career-related people as you can—because 1. they’ll be able to help you better understand the nature of the decision; and 2. they’ll open up doors for future opportunities, potentially even avoiding the need to make your current decision.
This makes me think of the contrast between systems analysis and net assessment/strategy. Yes, Fermi calculations are a valuable input into the discussion, but the nature of the problem is likely too complex to give sole weight to that calculation (in most circumstances—I think your example of comparing different research papers is a relatively simple/constrained environment).
In net assessment/strategy, the nature of the choice and your assessment of that nature determine the best methods of analysis. In systems analysis, a one-size-fits-all approach of measurement and analysis is taken. In certain circumstances that method of measurement is hugely valuable (why it became so popular in the US DoD), but in many other circumstances (like the Vietnam War) that method is deeply flawed.
It takes a solid assessment of the problem space to identify the key aspects of the competition. From there you can assess which methods and measures are most useful to judge the decision upon. And from there you can search for the data available to match your assessment.
Although choosing a career isn’t a competition against a competitor, your search to do the most good is a clearer objective than most adversaries can articulate regarding their objectives for long-term competition. The same meta-methods of analysis (to determine the best methods of analysis to fit the problem) apply. And then you can go on to finding the right data to fit those methods you’ve deemed most effective (maybe a Fermi calculation, but maybe 5 expert opinions in your career field or maybe your own gut feeling). I also think that in a career decision (unless it’s super major and clearly between two choices), the data you are going to be looking for is closer to a startup’s lean experiment than a longitudinal study. As you mentioned, Fermi calculations being readily available and usable doesn’t mean that is an effective tool for your decision.
Looking back at my answer, this didn’t really answer your question. The most effective technique I have found for deciding career choices though is to talk to as many career-related people as you can—because 1. they’ll be able to help you better understand the nature of the decision; and 2. they’ll open up doors for future opportunities, potentially even avoiding the need to make your current decision.