This is an interesting post! I agree with most of what you write. But when I saw the graph, I was suspicious. The graph is nice, but the world is not.
I tried to create a similar graph to yours:
In this case, fun work is pretty close to impactful toll. In fact, the impact value for it is only about 30% less than the impact value of impactful toll. This is definitely sizable, and creates some of the considerations above. But mostly, everywhere on the pareto frontier seems like a pretty reasonable place to be.
But there’s a problem: why is the graph so nice? To be more specific: why are the x and y axes so similarly scaled?
Why doesn’t it look like this?
Here I just replaced x in the ellipse equation with log(x). It seems pretty intuitive that our impact would be power law distributed, with a small number of possible careers making up the vast majority of our possible impact. A lot of the time when people are trying to maximize something it ends up power law distributed (money donated, citations for researchers, lives saved, etc.). Multiplicative processes, as Thomas Kwa alluded to, will also make something power law distributed. This doesn’t really look power law distributed quite yet though. Maybe I’ll take the log again:
Now, fun work is unfortunately 100x less impactful than impactful toll. That would be unfortunate. Maybe the entire pareto frontier doesn’t look so good anymore.
I think this is an inherently fatal flaw with attempts to talk about trading off impact and other personal factors in making choices. If your other personal factors are your ability to have fun, have good friendships, etc., you now have to make the claim that those things are also power-law distributed, and that your best life with respect to those other values is hundreds of times better than your impact maximizing life. If you don’t make that claim, then either you have to give your other values an extremely high weight compared with impact, or you have to let impact guide every decision.
In my view, the numbers for most people are probably pretty clear that impact should be the overriding factor. But I think there can be problems with thinking that way about everything. Some of those problems are instrumental: if you think impact is all that matters, you might try to do the minimum of self-care, but that’s dangerous.
I think people should think in the frame of the original graph most of the time, because the graph is nice, and a reminder that you should be nice to yourself. If you had one of the other graphs in your head, you wouldn’t really have any good reason to be nice to yourself that isn’t arbitrary or purely instrumental.
But every so often, when you face down a new career decision with fresh eyes, it can help to remember that the world is not so nice.
I know that “give your other values an extremely high weight compared with impact” is an accurate description of how I behave in practice. I’m kind of tempted to bite that same bullet when it comes to my extrapolated volition—but again, this would definitely be biting a bullet that doesn’t taste very good (do I really endorse caring about the log of my impact?). I should think more about this, thanks!
This isn’t a well thought-out argument, but something is bugging me in your claim. The real impact for your work may have some distribution, but I think the expected impact given career choices can be distributed very differently. Maybe, for example, the higher you aim, the more uncertainty you have, so your expectation doesn’t grow as fast.
I find it hard to believe that in real life you face choices that are reflected much better by your graph than Eric’s.
I share some of that intuition as well, but I have trouble conveying it numerically. Suppose that among realistic options that we might consider, we think ex post impact varies by 9 OOMs (as Thomas’ graph implies). Wouldn’t it be surprising if we have so little information that we only have <10^-9 confidence that our best choice is better than our second best choice?
I’m not very confident in my argument, but the particular scenario you describe sounds plausible to me.
Trying to imagine it in a simpler, global health setting—you could ask which of many problems to try to solve (e.g. malaria, snake bites, cancer), some of which may cause several orders of magnitude more suffering than others every year. If the solutions require things that are relatively straightforward—funding, scaling up production of something, etc. - it could be obvious which one to pick. But if the solutions require more difficult things, like research, or like solving decades-old distribution problems in Africa, then maybe the uncertainty can be strong enough to influence your decision noticeably.
This is tricky, because it’s really an empirical claim for which we need empirical evidence. I don’t currently have such evidence about anyone’s counterfactual choices. But I think even if you zoom in on the top 10% of a skewed distribution, it’s still going to be skewed. Within the top 10% (or even 1%) of researchers, nonprofits, it’s likely only a small subset are making most of the impact.
I think it’s true that “the higher we aim, the higher uncertainty we have” but you make it seem as if that uncertainty always washes out. I don’t think it does. I think higher uncertainty often is an indicator that you might be able to make it into the tails. Consider the monetary EV of starting a really good startup or working at a tech company. A startup has more uncertainty, but that’s because it creates the possibility of tail gains.
Anecdotally I think that certain choices I’ve made have changed the EV of my work by orders of magnitude. It’s important to note that I didn’t necessarily know this at the time, but I think it’s true retrospectively. But I do agree it’s not necessarily true in all cases.
I just copied the data manually since I don’t have that many comments, doesn’t seem like it would be too hard to throw together a scraper for it though.
Issa Rice’s post and his thoughtful design of providing raw queries from his website (mentioned in that post) are very helpful to quickly generate these queries.
I’m confused about how to square this with specific counterexamples. Say theoretical alignment work: P(important safety progress) probably scales with time invested, but not 100x by doubling your work hours. Any explanations here?
Idk if this is because uncertainty/ probabilistic stuff muddles the log picture. E.g. we really don’t know where the hits are, so many things are ‘decent shots’. Maybe after we know the outcomes, the outlier good things would be quite bad on the personal-liking front. But that doesn’t sound exactly correct either
Another complication: we want to select for people who are good fits for our problems, e.g. math kids, philosophy research kids, etc. To some degree, we’re selecting for people with personal-fun functions that match the shape of the problems we’re trying to solve (where what we’d want them to do is pretty aligned with their fun)
I think your point applies with cause selection, “intervention strategy”, or decisions like “moving to Berkeley”. Confused more generally
I want to point out two things that I think work in Eric’s favor in a more sophisticated model like the one you described.
First, I like the model that impact follows an approximately log distribution. But I would draw a different conclusion from this.
It seems to me that there is some set of current projects, S (this includes the project of “expand S”). They have impact given by some variable that I agree is closer to log normal than normal. Now one could posit two models: one idealized model in which people know (and agree on) magnitue of impacts and a second, more realistic model, where impact is extremely uncertain, with standard deviation on the same order as the potential impact. In the idealized model, you would maximize impact by working on the most impactful project, and get comparatively much less impact by working on a random project you happen to enjoy. But in the realistic world with very large uncertainty, you would maximize expected value by working on a project on the fuzzy Pareto frontier of “potentially very impactful projects”, but within this set you would prioritize projects that you have the largest competitive advantage in (which I think is also log distributed to a large extent). Presumably “how much you enjoy a subject” is correlated to “how much log advantage you have over the average person”, which makes me suspicious of the severe impact/enjoyment trade-off in your second graph.
I think a strong argument against this point would be to claim that the log difference between individual affinities is much less than the log difference between impacts. I intuitively think this is likely, but the much greater knowledge people have of their comparative strengths over the (vastly uncertain) guesses about impact will counteract this. Here I would enjoy an analysis of a model of impact vs. personal competitive advantage that takes both of these things into account.
Another point, which I think is somewhat orthogonal to the discussion of “how enjoyable is the highest-impact job”, and which I think is indirectly related to Eric’s point, is nonlinearity of effort.
Namely, there is a certain amount of nonlinearity in how “amount of time dedicated to cause X” correlates with “contribution to X”. There is some superlinearity at low levels (where at first most of your work goes into gaining domain knowledge and experience), and some sublinearity at high levels (where you run the risk of burnout as well as saturation potential if you chose a narrow topic). Because of the sublinearity at high levels, I think it makes sense for most people to have at least two “things” they do.
If you buy this I think it makes a lot of sense to make your second “cause” some version of “have fun” (or related things like “pursue personal growth for its own sake”). There are three reasons I believe this. First, this is a neglected cause: unless you’re famous or rich, no one else will work on it, which means that no one else will even try to pick the low-hanging fruit. Second, it’s a cause where you are an expert and, from your position, payoff is easy to measure and unambiguous. And third, if you are genuinely using a large part of your energy to have a high-impact career, being someone who has fun (and on a meta level, being a community that actively encourages people to have fun) will encourage others to be more likely to follow your career path.
I should caveat the third point: there are bad/dangerous arguments that follow similar lines, that result in people convincing themselves that they are being impactful by being hedonistic, or pursuing their favorite pet project. People are rationalizers and love coming up with stories that say “making myself happy is also the right thing to do”. But while this is something to be careful of, I don’t think it makes arguments of this type incorrect.
This is an interesting post! I agree with most of what you write. But when I saw the graph, I was suspicious. The graph is nice, but the world is not.
I tried to create a similar graph to yours:
In this case, fun work is pretty close to impactful toll. In fact, the impact value for it is only about 30% less than the impact value of impactful toll. This is definitely sizable, and creates some of the considerations above. But mostly, everywhere on the pareto frontier seems like a pretty reasonable place to be.
But there’s a problem: why is the graph so nice? To be more specific: why are the x and y axes so similarly scaled?
Why doesn’t it look like this?
Here I just replaced x in the ellipse equation with log(x). It seems pretty intuitive that our impact would be power law distributed, with a small number of possible careers making up the vast majority of our possible impact. A lot of the time when people are trying to maximize something it ends up power law distributed (money donated, citations for researchers, lives saved, etc.). Multiplicative processes, as Thomas Kwa alluded to, will also make something power law distributed. This doesn’t really look power law distributed quite yet though. Maybe I’ll take the log again:
Now, fun work is unfortunately 100x less impactful than impactful toll. That would be unfortunate. Maybe the entire pareto frontier doesn’t look so good anymore.
I think this is an inherently fatal flaw with attempts to talk about trading off impact and other personal factors in making choices. If your other personal factors are your ability to have fun, have good friendships, etc., you now have to make the claim that those things are also power-law distributed, and that your best life with respect to those other values is hundreds of times better than your impact maximizing life. If you don’t make that claim, then either you have to give your other values an extremely high weight compared with impact, or you have to let impact guide every decision.
In my view, the numbers for most people are probably pretty clear that impact should be the overriding factor. But I think there can be problems with thinking that way about everything. Some of those problems are instrumental: if you think impact is all that matters, you might try to do the minimum of self-care, but that’s dangerous.
I think people should think in the frame of the original graph most of the time, because the graph is nice, and a reminder that you should be nice to yourself. If you had one of the other graphs in your head, you wouldn’t really have any good reason to be nice to yourself that isn’t arbitrary or purely instrumental.
But every so often, when you face down a new career decision with fresh eyes, it can help to remember that the world is not so nice.
Great comment, I think that’s right.
I know that “give your other values an extremely high weight compared with impact” is an accurate description of how I behave in practice. I’m kind of tempted to bite that same bullet when it comes to my extrapolated volition—but again, this would definitely be biting a bullet that doesn’t taste very good (do I really endorse caring about the log of my impact?). I should think more about this, thanks!
This isn’t a well thought-out argument, but something is bugging me in your claim. The real impact for your work may have some distribution, but I think the expected impact given career choices can be distributed very differently. Maybe, for example, the higher you aim, the more uncertainty you have, so your expectation doesn’t grow as fast.
I find it hard to believe that in real life you face choices that are reflected much better by your graph than Eric’s.
I share some of that intuition as well, but I have trouble conveying it numerically. Suppose that among realistic options that we might consider, we think ex post impact varies by 9 OOMs (as Thomas’ graph implies). Wouldn’t it be surprising if we have so little information that we only have <10^-9 confidence that our best choice is better than our second best choice?
I’m not very confident in my argument, but the particular scenario you describe sounds plausible to me.
Trying to imagine it in a simpler, global health setting—you could ask which of many problems to try to solve (e.g. malaria, snake bites, cancer), some of which may cause several orders of magnitude more suffering than others every year. If the solutions require things that are relatively straightforward—funding, scaling up production of something, etc. - it could be obvious which one to pick. But if the solutions require more difficult things, like research, or like solving decades-old distribution problems in Africa, then maybe the uncertainty can be strong enough to influence your decision noticeably.
This is tricky, because it’s really an empirical claim for which we need empirical evidence. I don’t currently have such evidence about anyone’s counterfactual choices. But I think even if you zoom in on the top 10% of a skewed distribution, it’s still going to be skewed. Within the top 10% (or even 1%) of researchers, nonprofits, it’s likely only a small subset are making most of the impact.
I think it’s true that “the higher we aim, the higher uncertainty we have” but you make it seem as if that uncertainty always washes out. I don’t think it does. I think higher uncertainty often is an indicator that you might be able to make it into the tails. Consider the monetary EV of starting a really good startup or working at a tech company. A startup has more uncertainty, but that’s because it creates the possibility of tail gains.
Anecdotally I think that certain choices I’ve made have changed the EV of my work by orders of magnitude. It’s important to note that I didn’t necessarily know this at the time, but I think it’s true retrospectively. But I do agree it’s not necessarily true in all cases.
I had similar thoughts, discussed here after I tweeted about this post and somebody replied mentioning this comment.
(Apologies for creating a circular link loop, as my tweet links to this post, which now has a comment linking to my tweet)
Fittingly, this comment is in the tail of comments I’ve written on the forum:
Did you plot this manually or is there a tool for this? :O
Looks to me like it was created with one of the popular R plotting libraries.
Oh sorry I was more referring to the data.
I just copied the data manually since I don’t have that many comments, doesn’t seem like it would be too hard to throw together a scraper for it though.
The following GraphQL query gives a score for all of your comments in JSON form[1].
The same query for Linch.
Issa Rice’s post and his thoughtful design of providing raw queries from his website (mentioned in that post) are very helpful to quickly generate these queries.
Note that
userId
is trivially publicly available and should offer no utility or access.I’m confused about how to square this with specific counterexamples. Say theoretical alignment work: P(important safety progress) probably scales with time invested, but not 100x by doubling your work hours. Any explanations here?
Idk if this is because uncertainty/ probabilistic stuff muddles the log picture. E.g. we really don’t know where the hits are, so many things are ‘decent shots’. Maybe after we know the outcomes, the outlier good things would be quite bad on the personal-liking front. But that doesn’t sound exactly correct either
Another complication: we want to select for people who are good fits for our problems, e.g. math kids, philosophy research kids, etc. To some degree, we’re selecting for people with personal-fun functions that match the shape of the problems we’re trying to solve (where what we’d want them to do is pretty aligned with their fun)
I think your point applies with cause selection, “intervention strategy”, or decisions like “moving to Berkeley”. Confused more generally
I want to point out two things that I think work in Eric’s favor in a more sophisticated model like the one you described.
First, I like the model that impact follows an approximately log distribution. But I would draw a different conclusion from this.
It seems to me that there is some set of current projects, S (this includes the project of “expand S”). They have impact given by some variable that I agree is closer to log normal than normal. Now one could posit two models: one idealized model in which people know (and agree on) magnitue of impacts and a second, more realistic model, where impact is extremely uncertain, with standard deviation on the same order as the potential impact. In the idealized model, you would maximize impact by working on the most impactful project, and get comparatively much less impact by working on a random project you happen to enjoy. But in the realistic world with very large uncertainty, you would maximize expected value by working on a project on the fuzzy Pareto frontier of “potentially very impactful projects”, but within this set you would prioritize projects that you have the largest competitive advantage in (which I think is also log distributed to a large extent). Presumably “how much you enjoy a subject” is correlated to “how much log advantage you have over the average person”, which makes me suspicious of the severe impact/enjoyment trade-off in your second graph.
I think a strong argument against this point would be to claim that the log difference between individual affinities is much less than the log difference between impacts. I intuitively think this is likely, but the much greater knowledge people have of their comparative strengths over the (vastly uncertain) guesses about impact will counteract this. Here I would enjoy an analysis of a model of impact vs. personal competitive advantage that takes both of these things into account.
Another point, which I think is somewhat orthogonal to the discussion of “how enjoyable is the highest-impact job”, and which I think is indirectly related to Eric’s point, is nonlinearity of effort.
Namely, there is a certain amount of nonlinearity in how “amount of time dedicated to cause X” correlates with “contribution to X”. There is some superlinearity at low levels (where at first most of your work goes into gaining domain knowledge and experience), and some sublinearity at high levels (where you run the risk of burnout as well as saturation potential if you chose a narrow topic). Because of the sublinearity at high levels, I think it makes sense for most people to have at least two “things” they do.
If you buy this I think it makes a lot of sense to make your second “cause” some version of “have fun” (or related things like “pursue personal growth for its own sake”). There are three reasons I believe this. First, this is a neglected cause: unless you’re famous or rich, no one else will work on it, which means that no one else will even try to pick the low-hanging fruit. Second, it’s a cause where you are an expert and, from your position, payoff is easy to measure and unambiguous. And third, if you are genuinely using a large part of your energy to have a high-impact career, being someone who has fun (and on a meta level, being a community that actively encourages people to have fun) will encourage others to be more likely to follow your career path.
I should caveat the third point: there are bad/dangerous arguments that follow similar lines, that result in people convincing themselves that they are being impactful by being hedonistic, or pursuing their favorite pet project. People are rationalizers and love coming up with stories that say “making myself happy is also the right thing to do”. But while this is something to be careful of, I don’t think it makes arguments of this type incorrect.