This is an area I should probably write more about, but I have a harder time being pithy, and haven’t tried to distill my thoughts enough. But since you asked....
As a first approximation, you want to first consider the plausible value of the decision. If it’s choosing a career, for example, the difference between a good choice and a bad one is plausibly a couple million dollars. You almost certainly don’t want to spend more than a small fraction of that gathering information, but you do want to spend up to, say, 5% on thinking about the decision. (Yes, I’d say spending a year or two exploring the options before picking a career is worthwhile, if you’re really uncertain—but you shouldn’t need to be. See below.)
Once you have some idea of what the options are, you should pick what about the different options are good or bad—or uncertain. This should form the basis of at least a pro/con list—which is often enough by itself. (See my simulation here.) If you see that one option is winning on that type of list, you should probably just pick it—unless there are uncertainties that would change your mind.
Next, list those key uncertainties. In the career example, these might include: Will I enjoy doing the work?, How likely am I to be successful in the area?, How likely is the field to continue to be viable in the coming decades?, and How easy or hard is it to transition into/out of?
Notice that some of the uncertainties matter significantly, and others don’t. We have a tool that’s useful for this, which is the theoretical maximum of VoI, called Value of Perfect Information. This is the difference in value between knowing the answer with certainty, and the current decision. (Note: not knowing the future with certainty, but rather knowing the correct answer to the uncertainty. For example, knowing that you would have a 70% chance of being successful and making tons of money in finance.) Now ask yourself: If I knew the answer, would it change my decision? If the answer is no, drop it from the list of key uncertainties. If a relatively small probability of success would still leave finance as your top option, because of career capital and the potentially huge payoff, maybe this doesn’t matter. Alternatively, if even a 95% chance of success wouldn’t matter because you don’t know if you’d enjoy it, it still doesn’t matter—so move on to other questions.
If the answer is that knowing the answer would change your mind, you need to ask what information you could plausibly get about the question, and how expensive it is. For instance, you currently think there’s a 50% chance you’d enjoy working in finance. Spending a summer interning would make you sure one way or the other—but the cost in time is very high. It might be worth it, but there are other possibilities. Spending 15 minutes talking to someone in the field won’t make you certain, but will likely change your mind to think the odds are 90% or 10% - and in the former case, you can still decide to apply for a summer internship, and in the latter case, you can drop the idea now.
You should continue with this process of finding key things that would change your mind until you either think that you’re unlikely to change your mind further, or the cost of more certainty is high enough compared to the value of the decision that it’s not obviously worth the investment of time and money. (If it’s marginal or unclear, unless the decision is worth tens or hundreds of millions of dollars, significant further analysis is costly enough that you may not want to do it. If you’re unsure which way to decide at that point, you should flip a coin about the eventual decision—and if you’re uncertain enough to use a coin flip, then just do the riskier thing.)
This is an area I should probably write more about, but I have a harder time being pithy, and haven’t tried to distill my thoughts enough. But since you asked....
As a first approximation, you want to first consider the plausible value of the decision. If it’s choosing a career, for example, the difference between a good choice and a bad one is plausibly a couple million dollars. You almost certainly don’t want to spend more than a small fraction of that gathering information, but you do want to spend up to, say, 5% on thinking about the decision. (Yes, I’d say spending a year or two exploring the options before picking a career is worthwhile, if you’re really uncertain—but you shouldn’t need to be. See below.)
Once you have some idea of what the options are, you should pick what about the different options are good or bad—or uncertain. This should form the basis of at least a pro/con list—which is often enough by itself. (See my simulation here.) If you see that one option is winning on that type of list, you should probably just pick it—unless there are uncertainties that would change your mind.
Next, list those key uncertainties. In the career example, these might include: Will I enjoy doing the work?, How likely am I to be successful in the area?, How likely is the field to continue to be viable in the coming decades?, and How easy or hard is it to transition into/out of?
Notice that some of the uncertainties matter significantly, and others don’t. We have a tool that’s useful for this, which is the theoretical maximum of VoI, called Value of Perfect Information. This is the difference in value between knowing the answer with certainty, and the current decision. (Note: not knowing the future with certainty, but rather knowing the correct answer to the uncertainty. For example, knowing that you would have a 70% chance of being successful and making tons of money in finance.) Now ask yourself: If I knew the answer, would it change my decision? If the answer is no, drop it from the list of key uncertainties. If a relatively small probability of success would still leave finance as your top option, because of career capital and the potentially huge payoff, maybe this doesn’t matter. Alternatively, if even a 95% chance of success wouldn’t matter because you don’t know if you’d enjoy it, it still doesn’t matter—so move on to other questions.
If the answer is that knowing the answer would change your mind, you need to ask what information you could plausibly get about the question, and how expensive it is. For instance, you currently think there’s a 50% chance you’d enjoy working in finance. Spending a summer interning would make you sure one way or the other—but the cost in time is very high. It might be worth it, but there are other possibilities. Spending 15 minutes talking to someone in the field won’t make you certain, but will likely change your mind to think the odds are 90% or 10% - and in the former case, you can still decide to apply for a summer internship, and in the latter case, you can drop the idea now.
You should continue with this process of finding key things that would change your mind until you either think that you’re unlikely to change your mind further, or the cost of more certainty is high enough compared to the value of the decision that it’s not obviously worth the investment of time and money. (If it’s marginal or unclear, unless the decision is worth tens or hundreds of millions of dollars, significant further analysis is costly enough that you may not want to do it. If you’re unsure which way to decide at that point, you should flip a coin about the eventual decision—and if you’re uncertain enough to use a coin flip, then just do the riskier thing.)
Thanks—it’s useful to see your take on this!