Thanks a lot, I feel like this post could prove to be really usefuly for me, especially with giving this pattern a nice handle. I very much relate to stressing myself about having impact with my research. This led to me feeling averse towards trying to think about new “useful” research projects, which plausible decreases my research productivity quite a bit.
Relatedly, I’m currently reading “Why Greatness Cannot Be Planned: The Myth of the Objective” by Ken Stanley and Joel Lehman [1], where they argue that, among others, innovation and research is best achieved by aiming for what’s interesting and not what makes progress on a more concrete objective. I don’t yet have formed an opinion if I should avoid having impact at the forefront of my day-to-day thinking about research, but I found the idea refreshing that I might just focus on my interest and apply the impact-filter much more sparsely.
they argue that, among others, innovation and research is best achieved by aiming for what’s interesting and not what makes progress on a more concrete objective.
I don’t yet have a confident stance on how often/strongly I should, when picking and pursuing research directions, have relatively explciit, concrete plans for how my research would improve the world in mind. (It sounds like you’re in a similar boat.) But in thinking about that question, I found Galef’s post useful. I also found some of the links I collected here useful—perhaps especially How to do research that matters and (the answers provided to) Do research organisations make theory of change diagrams? Should they?
[I]f the expected concrete value of two interventions is similar, we should generally favor investing in interventions that have less evidence supporting them.
I like Askell’s talk and think this is an important point. Though when making the point without the full context of the talk, it also seems worth noting that:
As an empirical matter, one’s naive/early/quick analyses of how good (or cost-effective, or whatever) something is seem to often be overly optimistic.
Though I don’t know precisely why this is the case, and I imagine it varies by domain.
Additionally, there’s the optimizer’s curse. This is essentially a reason why one is likelier to be overestimating the value of something if one thinks that thing is unusually good. The curse is larger the more uncertainty one has.
For both reasons, if you see X and Y as unusually good, but you have less evidence re X, then that should update you towards thinking you’re being (more) overly optimistic about X, and thus that Y is actually better.
I think your comment is completely valid if we imagine that the two options “look to be equally good” even after adjusting for these tendencies. But I think people often don’t adjust for these tendencies, so it seems worth making it explicit.
(Also, even if X does currently seem somewhat less good than Y, that can be outweighed by the value of information consideration such that it’s worth further investigation of X rather than Y anyway.)
As an empirical matter, one’s naive/early/quick analyses of how good (or cost-effective, or whatever) something is seem to often be overly optimistic.
One possible reason is completely rational: if we’re estimating expected value of an intervention with a 1% chance to be highly valuable, then 99% of the time we realize the moonshot won’t work and revise the expected value downward.
That definitely can happen, and makes me realise my comment wasn’t sufficiently precise.
An expected value estimate can be reasonable even if there’s a 99% chance it would be revised downwards given more info, if there’s also a 1% chance it would be revised upwards by enough to offset the potential downward revisions. If an estimator makes such an estimate and is well-calibrated, I wouldn’t say they’re making a mistake, and thus probably wouldn’t say they’re being “overly optimistic”.
The claim I was making was that one’s naive/early/quick analyses of how good (or cost-effective, or whatever) tend to not be well-calibrated, systematically erring towards optimism in a way that means that it’s best to adjust the expected value downwards to account for this (unless one has already made such an adjustment).
But I’m not actually sure how true that claim is (I’m just basing it on my memory of GiveWell posts I read in the past). Maybe most things that look like that situation are either actually the optimiser’s curse or actually the sort of situation you describe.
Thanks for the link! I’m adding it to my to listen to list :)
A few rambly reactions/thoughts in response to your message in case they are helpful:
I wonder if doing some experiments here would be useful. e.g. do you seem to have more impact if you assess your impact/usefulness daily, weekly, monthly, yearly? (or whatever intervals feel worth a test to you) What happens if you experiment with just leaning into your interests (rather than usefulness)? Seems worth trying some things out and then taking time to reflect on how they went. I’ve benefited a lot from experimentation of this form.
FWIW I find assessing my impact more than a couple of times a year to be quite stressful and not very helpful (often a distraction from actually doing things). Interestingly and perhaps counterintuitively, I find prioritizing based on expected value to be useful in my role daily. Somehow the frame of prioritization enables me to make those trade-offs without kicking up this sort of desperation hamster wheel/stress cycle. I wonder if trying on different frames could be useful for you too? Prioritization depersonalizes it a bit for me in a useful way.
Thanks a lot, I feel like this post could prove to be really usefuly for me, especially with giving this pattern a nice handle. I very much relate to stressing myself about having impact with my research. This led to me feeling averse towards trying to think about new “useful” research projects, which plausible decreases my research productivity quite a bit.
Relatedly, I’m currently reading “Why Greatness Cannot Be Planned: The Myth of the Objective” by Ken Stanley and Joel Lehman [1], where they argue that, among others, innovation and research is best achieved by aiming for what’s interesting and not what makes progress on a more concrete objective. I don’t yet have formed an opinion if I should avoid having impact at the forefront of my day-to-day thinking about research, but I found the idea refreshing that I might just focus on my interest and apply the impact-filter much more sparsely.
[1] a nice interview about the book can be found here: https://braininspired.co/podcast/86/
[Maybe a tangent]
This reminds me of Julia Galef’s Can we intentionally improve the world? Planners vs. Hayekians.
I don’t yet have a confident stance on how often/strongly I should, when picking and pursuing research directions, have relatively explciit, concrete plans for how my research would improve the world in mind. (It sounds like you’re in a similar boat.) But in thinking about that question, I found Galef’s post useful. I also found some of the links I collected here useful—perhaps especially How to do research that matters and (the answers provided to) Do research organisations make theory of change diagrams? Should they?
Also related is the idea that the moral value of additional information is high when there is relatively low resilience in your credence that the current intervention is best. This leads to the (to me) rather unintuitive conclusion that if you have two research paths that both look to be equally good to look into for potentially improving the world, then, ceteris paribus, it may be better to invest in the research path for which you have less evidence that it is a good research path to follow. From Amanda Askell in the link:
I like Askell’s talk and think this is an important point. Though when making the point without the full context of the talk, it also seems worth noting that:
As an empirical matter, one’s naive/early/quick analyses of how good (or cost-effective, or whatever) something is seem to often be overly optimistic.
Though I don’t know precisely why this is the case, and I imagine it varies by domain.
See also Why We Can’t Take Expected Value Estimates Literally (Even When They’re Unbiased)
Additionally, there’s the optimizer’s curse. This is essentially a reason why one is likelier to be overestimating the value of something if one thinks that thing is unusually good. The curse is larger the more uncertainty one has.
For both reasons, if you see X and Y as unusually good, but you have less evidence re X, then that should update you towards thinking you’re being (more) overly optimistic about X, and thus that Y is actually better.
I think your comment is completely valid if we imagine that the two options “look to be equally good” even after adjusting for these tendencies. But I think people often don’t adjust for these tendencies, so it seems worth making it explicit.
(Also, even if X does currently seem somewhat less good than Y, that can be outweighed by the value of information consideration such that it’s worth further investigation of X rather than Y anyway.)
One possible reason is completely rational: if we’re estimating expected value of an intervention with a 1% chance to be highly valuable, then 99% of the time we realize the moonshot won’t work and revise the expected value downward.
That definitely can happen, and makes me realise my comment wasn’t sufficiently precise.
An expected value estimate can be reasonable even if there’s a 99% chance it would be revised downwards given more info, if there’s also a 1% chance it would be revised upwards by enough to offset the potential downward revisions. If an estimator makes such an estimate and is well-calibrated, I wouldn’t say they’re making a mistake, and thus probably wouldn’t say they’re being “overly optimistic”.
The claim I was making was that one’s naive/early/quick analyses of how good (or cost-effective, or whatever) tend to not be well-calibrated, systematically erring towards optimism in a way that means that it’s best to adjust the expected value downwards to account for this (unless one has already made such an adjustment).
But I’m not actually sure how true that claim is (I’m just basing it on my memory of GiveWell posts I read in the past). Maybe most things that look like that situation are either actually the optimiser’s curse or actually the sort of situation you describe.
Thanks for the link! I’m adding it to my to listen to list :)
A few rambly reactions/thoughts in response to your message in case they are helpful:
I wonder if doing some experiments here would be useful. e.g. do you seem to have more impact if you assess your impact/usefulness daily, weekly, monthly, yearly? (or whatever intervals feel worth a test to you) What happens if you experiment with just leaning into your interests (rather than usefulness)? Seems worth trying some things out and then taking time to reflect on how they went. I’ve benefited a lot from experimentation of this form.
FWIW I find assessing my impact more than a couple of times a year to be quite stressful and not very helpful (often a distraction from actually doing things). Interestingly and perhaps counterintuitively, I find prioritizing based on expected value to be useful in my role daily. Somehow the frame of prioritization enables me to make those trade-offs without kicking up this sort of desperation hamster wheel/stress cycle. I wonder if trying on different frames could be useful for you too? Prioritization depersonalizes it a bit for me in a useful way.