A technique I’ve found useful in making complex decisions where you gather lots of evidence over time—for example, deciding what to do after your graduation, or whether to change jobs, etc., where you talk to lots of different people and weigh lots of considerations—is to make a spreadsheet of all the arguments you hear, each with a score for how much it supports each decision.
For example, this summer, I was considering the options of “take the Open Phil job,” “go to law school,” and “finish the master’s.” I put each of these options in columns. Then, I’d hear an argument like “being in school delays your ability to take a full-time job, which is where most of your impact will happen”; I’d add a row for this argument. I thought this was a very strong consideration, so I gave the Open Phil job 10 points, law school 0, and the master’s 3 (since it was one more year of school instead of 3 years). Later, I’d hear an argument like “legal knowledge is actually pretty useful for policy work,” which I thought was a medium-strength consideration, and I’d give these options 0, 5, and 0.
I wouldn’t take the sum of these as a final answer, but it was useful for a few reasons:
In complicated decisions, it’s hard to hold all of the arguments in your head at a time. This might be part of why I noticed a strong recency bias, where the most recent handful of considerations raised to me seemed the most important. By putting them all in one place, I could feel like I was properly accounting for all the things I was aware of.
Relatedly, it helped me avoid double-counting arguments. When I’d talk to a new person, and they’d give me an opinion, I could just check whether their argument was basically already in the spreadsheet; sometimes I’d bump a number from 4 to 5, or something, based on them being persuasive, but sometimes I’d just say, “Oh, right, I guess I already knew this and shouldn’t really update from it.”
I also notice a temptation to simplify the decision down to a single crux or knockdown argument, but usually cluster thinking is a better way to make these decisions, and the spreadsheet helps aggregate things such that an overall balance of evidence can carry the day.
A technique I’ve found useful in making complex decisions where you gather lots of evidence over time—for example, deciding what to do after your graduation, or whether to change jobs, etc., where you talk to lots of different people and weigh lots of considerations—is to make a spreadsheet of all the arguments you hear, each with a score for how much it supports each decision.
For example, this summer, I was considering the options of “take the Open Phil job,” “go to law school,” and “finish the master’s.” I put each of these options in columns. Then, I’d hear an argument like “being in school delays your ability to take a full-time job, which is where most of your impact will happen”; I’d add a row for this argument. I thought this was a very strong consideration, so I gave the Open Phil job 10 points, law school 0, and the master’s 3 (since it was one more year of school instead of 3 years). Later, I’d hear an argument like “legal knowledge is actually pretty useful for policy work,” which I thought was a medium-strength consideration, and I’d give these options 0, 5, and 0.
I wouldn’t take the sum of these as a final answer, but it was useful for a few reasons:
In complicated decisions, it’s hard to hold all of the arguments in your head at a time. This might be part of why I noticed a strong recency bias, where the most recent handful of considerations raised to me seemed the most important. By putting them all in one place, I could feel like I was properly accounting for all the things I was aware of.
Relatedly, it helped me avoid double-counting arguments. When I’d talk to a new person, and they’d give me an opinion, I could just check whether their argument was basically already in the spreadsheet; sometimes I’d bump a number from 4 to 5, or something, based on them being persuasive, but sometimes I’d just say, “Oh, right, I guess I already knew this and shouldn’t really update from it.”
I also notice a temptation to simplify the decision down to a single crux or knockdown argument, but usually cluster thinking is a better way to make these decisions, and the spreadsheet helps aggregate things such that an overall balance of evidence can carry the day.