Addendum to the post: an exercise in giving myself advice
The ideas below aren’t new or very surprising, but I found it useful to sit down and write out some lessons for myself. Consider doing something similar; if you’re up for sharing what you write as a comment on this post, I’d be interested and grateful.
(1) Figure out my reasons for working on (major) projects and outline situations in which I would want myself to leave, ideally before getting started on the projects
I plan on trying to do this for any project that gives me any (ethical) doubts, and/or will take up at least 3 months of my full-time work. When possible, I also want to try sharing my notes with someone I trust. (I just did this. If you want to use my template / some notes, you can find it in this footnote.[1] Related: Staring into the abyss.)
(2) Notice potential (epistemic and moral) “risk factors” in my environment
In many ways, the environment at Los Alamos seemed to elevate the risk that participants would ignore their ethical concerns (probably partly by design). Besides the fact that they were working on a deadly weapon,
There was a lot of secrecy, and connections to people outside of the project were suspended
There was a decent amount of ~blanket admiration for the leaders of the project (and for some of the more senior scientists)
Relatedly, there was a sense of urgency and a collective mission (and there was a relatively clear set of “enemies” — this was during a war)
Based on how people wrote about Los Alamos later, there seemed to be something playful or adventurous about how many treated their work; the bomb was being called a “gadget,” their material needs were taken care of, etc.
And many of the participants were relatively young
All else equal, I would like to avoid environments that exhibit these kinds of factors. But shunning them entirely seems impractical, so it seems worth finding ways to create some guardrails. Simply noticing that an environment poses these risks seems like it might already be quite useful. I think it would give me the chance to put myself on “high alert,” using that as a prompt to check back in with myself, talk to mentors, etc.
(3) Train some habits and mental motions
Reading about all of this made me want to do the following things more (and more deliberately):
Talking to people who aren’t embedded in my local/professional environment
And talking to people who think very independently or in unusual (relative to my immediate circles) ways, seriously considering their arguments and conclusions
(Also: remembering that I almost always have fallback options, and nurturing those options)
Explicitly prompting myself to take potential outcomes of my work seriously
Imagine my work’s impact is more significant than I expected it to be. How do I feel? Is there anything I’m embarrassed about, or that I wish I had done differently?
Training myself to say — and be willing to believe/entertain — things that might diminish my social status in my community
(This includes articulating confusion or uncertainty.)
I don’t have time right now to set up systems that could help me with these things, but I also just an event to my calendar, to try to figure out how I can do more of this. (E.g. I might want to add something like this to one of my weekly templates or just set up 1-2 recurring events.) Consider doing the same, if you’re in a similar situation.
And these ideas were generated very quickly — I’m sure there are more and likely better recommendations, so suggestions are welcome!
[Core worldview + 1-3 key goals, ideally including something that’s specific enough that people you know and like might disagree with it]
(2) Situations in which I would want myself to leave [these are not necessarily things that I (or you, if you’re filling this out) think are plausible!]
(2a) Specific red lines — I’d definitely leave if...
(2b) Red flags: very seriously consider leaving if...
(2c) [Optional] Other general notes on this (e.g. how changes in circumstances might affect why I’d leave)
My notes included hypothetical situations like learning something that would cause me to significantly update on the integrity of the people in charge of my organization, situations in which important sources of feedback (sources of correction) seemed to be getting getting closed off, etc.
I plan on trying to do this for any project that gives me any (ethical) doubts, and/or will take up at least 3 months of my full-time work.
Choosing ‘and’ or ‘or’ feels important here since they seem quite different! Maybe our rough model should be cause-for-introspection = ethical qualms * length of project
Addendum to the post: an exercise in giving myself advice
The ideas below aren’t new or very surprising, but I found it useful to sit down and write out some lessons for myself. Consider doing something similar; if you’re up for sharing what you write as a comment on this post, I’d be interested and grateful.
(1) Figure out my reasons for working on (major) projects and outline situations in which I would want myself to leave, ideally before getting started on the projects
I plan on trying to do this for any project that gives me any (ethical) doubts, and/or will take up at least 3 months of my full-time work. When possible, I also want to try sharing my notes with someone I trust. (I just did this. If you want to use my template / some notes, you can find it in this footnote.[1] Related: Staring into the abyss.)
(2) Notice potential (epistemic and moral) “risk factors” in my environment
In many ways, the environment at Los Alamos seemed to elevate the risk that participants would ignore their ethical concerns (probably partly by design). Besides the fact that they were working on a deadly weapon,
There was a lot of secrecy, and connections to people outside of the project were suspended
There was a decent amount of ~blanket admiration for the leaders of the project (and for some of the more senior scientists)
Relatedly, there was a sense of urgency and a collective mission (and there was a relatively clear set of “enemies” — this was during a war)
Based on how people wrote about Los Alamos later, there seemed to be something playful or adventurous about how many treated their work; the bomb was being called a “gadget,” their material needs were taken care of, etc.
And many of the participants were relatively young
(Related themes are also discussed in “Are you really in a race?”)
All else equal, I would like to avoid environments that exhibit these kinds of factors. But shunning them entirely seems impractical, so it seems worth finding ways to create some guardrails. Simply noticing that an environment poses these risks seems like it might already be quite useful. I think it would give me the chance to put myself on “high alert,” using that as a prompt to check back in with myself, talk to mentors, etc.
(3) Train some habits and mental motions
Reading about all of this made me want to do the following things more (and more deliberately):
Talking to people who aren’t embedded in my local/professional environment
And talking to people who think very independently or in unusual (relative to my immediate circles) ways, seriously considering their arguments and conclusions
(Also: remembering that I almost always have fallback options, and nurturing those options)
Explicitly prompting myself to take potential outcomes of my work seriously
Imagine my work’s impact is more significant than I expected it to be. How do I feel? Is there anything I’m embarrassed about, or that I wish I had done differently?
Training myself to say — and be willing to believe/entertain — things that might diminish my social status in my community
(This includes articulating confusion or uncertainty.)
I don’t have time right now to set up systems that could help me with these things, but I also just an event to my calendar, to try to figure out how I can do more of this. (E.g. I might want to add something like this to one of my weekly templates or just set up 1-2 recurring events.) Consider doing the same, if you’re in a similar situation.
And these ideas were generated very quickly — I’m sure there are more and likely better recommendations, so suggestions are welcome!
Here’s the rough format I just used:
(1) Why I’m doing what I’m doing
[Core worldview + 1-3 key goals, ideally including something that’s specific enough that people you know and like might disagree with it]
(2) Situations in which I would want myself to leave [these are not necessarily things that I (or you, if you’re filling this out) think are plausible!]
(2a) Specific red lines — I’d definitely leave if...
(2b) Red flags: very seriously consider leaving if...
(2c) [Optional] Other general notes on this (e.g. how changes in circumstances might affect why I’d leave)
My notes included hypothetical situations like learning something that would cause me to significantly update on the integrity of the people in charge of my organization, situations in which important sources of feedback (sources of correction) seemed to be getting getting closed off, etc.
Choosing ‘and’ or ‘or’ feels important here since they seem quite different! Maybe our rough model should be cause-for-introspection = ethical qualms * length of project