Since you asked, here are my agreements and disagreements, mostly presented without argument:
As someone who is roughly in the target audience (I am involved in hiring for senior ops roles, though it’s someone else’s core responsibility), I think I disagree with much of this post (eg I think this isn’t as big a problem as you think, and the arguments around hiring from outside EA are weak), but in my experience it’s somewhat costly and quite low value to publicly disagree with posts like this, so I didn’t write anything.
It’s costly because people get annoyed at me.
It’s low value because inasmuch as think your advice is bad, I don’t really need to persuade you you’re wrong, I just need to persuade the people who this article is aimed at that you’re wrong. It’s generally much easier to persuade third parties than people who already have a strong opinion. And I don’t think that it’s that useful for the counterarguments to be provided publicly.
And if someone was running an org and strongly agreed with you, I’d probably shrug and say “to each their own” rather than trying that hard to talk them out of it: if a leader really feels passionate about shaping org culture a particular way, that’s a reasonable argument for them making the culture be that way.
For some of the things you talk about in this post (e.g. “The priority tasks are often mundane, not challenging”, ‘The role is mostly positioned as “enabling the existing leadership team” to the extent that it seems like “do all the tasks that we don’t like”’) I agree that it is bad inasmuch as EA orgs do this as egregiously as you’re describing. I’ve never seen this happen in an EA org as blatantly as you’re describing, but find it easy to believe that it happens.
However, if we talked through the details I think there’s a reasonable chance that I’d end up thinking that you were being unfair in your description.
I think one factor here is that some candidates are actually IMO pretty unreasonably opposed to ever doing grunt work. Sometimes jobs involve doing repetitive things for a while when they’re important. For example, I spoke to 60 people or so when I was picking applicants for the first MLAB, which was kind of repetitive but also seemed crucial. It’s extremely costly to accidentally hire someone who isn’t willing to do this kind of work, and it’s tricky to correctly communicate both “we’d like you to not mostly do repetitive work” and “we need you to sometimes do repetitive work, as we all do, because the most important tasks are sometimes repetitive”.
I think our main disagreement is that you’re more optimistic about getting people who “aren’t signed up to all EA/long-termist ideas” to help out with high level strategy decisions than I am. In my experience, people who don’t have a lot of the LTist context often have strong opinions about what orgs should do that don’t really make sense given more context.
For example, some strategic decisions I currently face are:
Should I try to hire more junior vs more senior researchers?
Who is the audience of our research?
Should I implicitly encourage or discourage working on weekends?
I think that people who don’t have experience in a highly analogous setting will often not have the context required to assess this, because these decisions are based on idiosyncrasies of our context and our goals. Senior people without relevant experience will have various potentially analogous experience, and I really appreciate the advice that I get from senior people who don’t have the context, but I definitely have to assess all of their advice for myself rather than just following their best practices (except on really obvious things).
If I was considering hiring a senior person who didn’t have analogous experience and also wanted to have a lot of input into org strategy, I’d be pretty scared if they didn’t seem really on board with the org leadership sometimes going against their advice, and I would want to communicate this extremely clearly to the candidate, to prevent mismatched expectations.
I think that the decisions that LTist orgs make are often predicated on LTist beliefs (obviously), and people who don’t agree with LTist beliefs are going to systematically disagree about what to do, and so if the org hires such a person, they need that person to be okay with getting overruled a bunch on high level strategy. I don’t really see how you could avoid this.
In general, I think that a lot of your concerns might be a result of orgs trying to underpromise and overdeliver: the orgs are afraid that you will come in expecting to have a bunch more strategic input than they feel comfortable promising you, and much less mundane work than you might occasionally have. (But probably some also comes from orgs making bad decisions.)
I really appreciate your honest response—thanks for sticking your neck out.
I think you’ve layered on nuance / different perspectives which enables a richer understanding in some regards, and in some others I think we diverge mostly on how big a risk we perceive non-EAs in senior roles as presenting relative to value they bring. I think we might have fundamental disagreements about ‘the value of outside perspectives’ Vs. ‘the need for context to add value’; or put another way ‘the risk of an echo chamber from too-like-minded people’ Vs. ‘the risk of fracture and bad decision-making from not-like-minded-enough people’.
I’m going to have a think about what the most interesting / useful way to respond might be; I suspect it would be a bit dull / less useful to just rebutt point by point rather than get deeper, but don’t want to infer your drivers too much. Will likey build on this later in the week.
I think we might have fundamental disagreements about ‘the value of outside perspectives’ Vs. ‘the need for context to add value’; or put another way ‘the risk of an echo chamber from too-like-minded people’ Vs. ‘the risk of fracture and bad decision-making from not-like-minded-enough people’.
First off, again lauding you and CarolineJ for throwing some challenge. I’ve tried to think about how to counter-argue by being critical of your arguments / their implications without sounding like I’m being critical of you two personally. I think it’s hard to do this, especially through this medium, so I apologise if I get this wrong.
So main counter-arguments...:
For some of the things you talk about in this post (e.g. “The priority tasks are often mundane, not challenging”, ‘The role is mostly positioned as “enabling the existing leadership team” to the extent that it seems like “do all the tasks that we don’t like”’) I agree that it is bad inasmuch as EA orgs do this as egregiously as you’re describing. I’ve never seen this happen in an EA org as blatantly as you’re describing, but find it easy to believe that it happens.
However, if we talked through the details I think there’s a reasonable chance that I’d end up thinking that you were being unfair in your description.
I think one factor here is that some candidates are actually IMO pretty unreasonably opposed to ever doing grunt work. Sometimes jobs involve doing repetitive things for a while when they’re important. For example, I spoke to 60 people or so when I was picking applicants for the first MLAB, which was kind of repetitive but also seemed crucial. It’s extremely costly to accidentally hire someone who isn’t willing to do this kind of work, and it’s tricky to correctly communicate both “we’d like you to not mostly do repetitive work” and “we need you to sometimes do repetitive work, as we all do, because the most important tasks are sometimes repetitive”.
My point here wasn’t about quantifying how often it happens, or whether it’s a “fair” claim. It’s about wanting people to write job descriptions that will attract more / better candidates than they’re currently doing. Even if it doesn’t apply in 90% of cases, I thought it was important to make the point so people could think about the signals it sends out, as I laid out in the post.
I agree though that all jobs will involve some ‘grunt work’ or enabling others even at the cost of your own projects, and it’s important to signal that; the issue as I outlined is that a good candidate will think “they really don’t know how to get the best of me” if the JD is mostly that.
I think our main disagreement is that you’re more optimistic about getting people who “aren’t signed up to all EA/long-termist ideas” to help out with high level strategy decisions than I am. In my experience, people who don’t have a lot of the LTist context often have strong opinions about what orgs should do that don’t really make sense given more context.
I think that is often the value they will add—paradigmatic challenge as well as practical insight. It’s not like the entire EA / LT house is built on solid epistemic foundations, let alone very clear theories of change / impact behind every single intervention. (I do think this is a dirty secret we don’t do well of owning up to, but maybe this is for another post). And if there were, it’s not as if people with outside experience wouldn’t do a good job of red-teaming them. I outlined examples in my original post about where outsiders would bring value in emerging / pre-paradigmatic fields, even if they are not fuller EA / LT signed up; I think they should be more strongly considered.
For me personally, I find perspectives of people outside of my field helpful for challenging my fundamental assumptions about how innovation actually works. Them being in the room makes it more likely a) they’ll get listened to, b) they have access to the same materials / research as me so we can see why we’re diverging on what appears to be the same information, and then check / challenge each others assumptions.
I know I should really listen when I feel uncomfortable; when I feel annoyed at what another team member is saying. It usually indicates they’ve struck a nerve of my uncertainty, or pointed out some fundamental assumption I’m resting too heavily upon. I’m not always good at doing it, but when I do it does make our work stronger.
For example, some strategic decisions I currently face are:
Should I try to hire more junior vs more senior researchers?
Who is the audience of our research?
Should I implicitly encourage or discourage working on weekends?
There are some structural / cultural issues which pop up again and again in different workplaces. EA / LT might have more astronomically massive goals than most other orgs, but the mechanics of achieving them will have more in common with other teams / orgs than idiosyncracies; especially given most planning periods / real decisions never go beyond 5-10 years.
I can’t imagine why someone outside EA / LT would be constrained by ‘not being signed up enough’ from contributing to the strategic decision you mention? Some of them are pretty classic whatever team you’re in; like senior / junior researchers. The cultural ones—e.g. implicitly encouraging / discouraging weekend working—would especially benefit from outsiders who have experienced lots of team / org environments, and have better intuition for good / bad workplace cultures. I think so because, again, lots of EA orgs have really messed up their own cultures and left a trail of burn-out from it.
I think that people who don’t have experience in a highly analogous setting will often not have the context required to assess this, because these decisions are based on idiosyncrasies of our context and our goals. Senior people without relevant experience will have various potentially analogous experience, and I really appreciate the advice that I get from senior people who don’t have the context, but I definitely have to assess all of their advice for myself rather than just following their best practices (except on really obvious things).
Reading this makes me think two things:
Isn’t that an argument for bringing outsiders into the organisation; so they can acquire the wider context, weigh it up along with their experience from other situations—some analogous along the lines you think and some analogous along different lines than you’d expect—and then add their thoughts to yours to come down on a decision? My bet is that would be more valuable to you than someone more similar to you doing the same to make those strategic decisions collaboratively.
I’m always sceptical about arguments that can be boiled down to “our context is different” or “our organisation is unique”. We all think that about our teams / orgs; we all think our constraints and challenges are very specific, but in reality most things going well / badly can be explained by some combination of clarity of purpose / vision, clarity of roles and responsibilities, and how people are actually working together[1].
If I was considering hiring a senior person who didn’t have analogous experience and also wanted to have a lot of input into org strategy, I’d be pretty scared if they didn’t seem really on board with the org leadership sometimes going against their advice, and I would want to communicate this extremely clearly to the candidate, to prevent mismatched expectations.
But I hope this would be a two-way street? To be fair, I don’t know of any recruitment where this kind of negotiation of terms doesn’t happen to begin with. That’s normal. People recruit ‘outsiders’, different levels of autonomy are given, they see if they can work together, if it doesn’t they part ways; usually with the newbie leaving but sometimes with the incumbent leaving as the vision of the newbie carries more sway...
Working well with people you don’t 100% agree with is very possible, esp. if you’re optimised your hiring for certain qualities like openness and abilities to argue but maintain cohesion. It’s also just a really important leadership skill to build for most contexts.
I think that the decisions that LTist orgs make are often predicated on LTist beliefs (obviously), and people who don’t agree with LTist beliefs are going to systematically disagree about what to do, and so if the org hires such a person, they need that person to be okay with getting overruled a bunch on high level strategy. I don’t really see how you could avoid this.
Reading this, I felt a little like “what’s wrong about that?” Or more specifically “what’s wrong about having systematic disagreements?” Conflict / disagreements are good things! The more fundamental the better, provided you’re not rehashing the same ground on repeat (and there is a subtle difference between seeing the same argument ad nauseum compared with the same justifiable tensions being played out in different problem / solution spaces).
However, if your assumption is that the non-LT person would be overruled consistently, then yes that would be a problem because then you’re sacrificing the opportunity for synthesis or steel-manning. I feel that if I’m making really good LT arguments with a good theory of change, I should be able to convince someone who isn’t 100% signed up to that to come on the journey with me. If that person was being overruled again and again without any form of synthesis emerging, I’d take that as a sign that I was doing a rubbish job of listening and understanding another person’s perspective.
I think if I was working in such an LT org I would be terrified of not having a detractor in the room. This is because I think a lot of LT Vs. near-term conflict happens because of a lack of concrete theories of change being put forward by LTists. I would want a detractor challenging the assumptions behind my LT decisions, steel-manning at every step of the way. Personally, I would want them jumping up and down at me if it sounded like I was willing to sacrifice a lot of near-term high probability impact in the service of more spurious long-term gains. If I am going to do that, I want to be really confident I made that decision for the right reasons; I want to be so confident that the detractor can plausibly agree with me, and even leave the org on good terms if they could understand the decision being made but couldn’t abide by it.
I am sceptical that an advisor from outside the org would give that level of challenge; they don’t have the full context, they are not as invested in thinking about these things, they won’t have the bandwidth to really make the case. And I’m very sceptical that just really smart, conscientious people who also happen to share the same set of assumptions as me will do as good a job steelmanning. In practice, they should be able to, but in reality assumptions / blindspots need people who don’t have them to point them out.
I think two additional cruxes I’m realising we have might be:
I think really good organisational leadership is hard, and requires experience—ideally lots of experience, doing well and badly and reflecting on yourself and what it’s like to work with you. I think the leadership in many EA orgs have not had those opportunities, which I think is a big risk. But I think you and perhaps others see this as less the case?
I trust my judgement less if I am not getting serious challenge from people on strategic decisions, that shakes me both on the process and on the paradigm / values-system. Because I don’t think it’s possible I know everything, but that wisdom of crowds should get to better solution space. I think I might be unusually strong in this tendency, but I won’t put words in yours / others mouths on what the opposite looks like.
In fact, last two weeks I received a tonne of highly critical challenge fundamentally about whether my programme had a plausible theory of change. Though initially frustrating, it’s helped me see my work for what it is; ultimately experimental, and if it doesn’t work it must be killed off.
To finish, this is the type of thing I’d usually chat over in a tone / setting that’s relaxed and open and less point-by-point rebuttaly; because I think this type of topic is better as an open, reflective conversation but this medium sets it up to feel more confrontational. Ultimately, I don’t know what’s going on in anyone else’s head, or their full context, so I can only make observations / ask questions / pose challenge and hope they feel useful. Because I don’t think the medium can do this topic justice, I would be open to exploring in a different medium if need be.
And yes, part of my deciding to go for the delicate core is inspired by Scott Alexander’s recent post, and reinforced by Helen’s more recent post.
I totally over-summarised this bit in haste, but there’s tonnes of org literature on this, much of the best I read is by McKinsey on organisational health.
This counter-response updated my views quite a bit, thanks! Maybe a way to reconcile the two views in a specific example:
Should I implicitly encourage or discourage working on weekends?
Strategical alignment/understanding: Ultimately at some level the answer to this question depends on what the organisation is trying to achieve and the philosophical trade-offs its leadership are willing to make.
Experience: But in order to make that decision, you have to know what the effects of more or less working on the weekends would actually be. And effects on organisational culture can be gradual, so you need to have observed this process to be able to predict it well.
You don’t necessarily need to have both capacities in the same person, as long as there’s appreciation of the respective areas of expertise.
tl;dr: I suspect selecting for speaking fluent LessWrong jargon is anti-correlated with being exceptionally good at ops.
Double-cruxing in the comments is so satisfying to read. I’ve outlined another possible related component in this comment that this post might have been pointing to (I find the below gets easily conflated in my head unless I do work to tease out the distinct pieces and so I thought this might plausibly be happening a bit here too).
I am curious how much actual values matter versus being good at EA/rationalist signalling mattering.
Eg. I think people take what I say a lot more seriously when I use jargon compared to when I say the same core idea in plain English or with a more accessible framing (which usually is a lot more work: saying what I believe is true in language I think a smart newcomer with a different academic background could understand is hard but often seems worth it to me).
I can imagine someone who gets vibed as a “normie” because of the language and framing they use (because they’ve spent more time in the real world than the EA world where different wording gets social reinforcement) being dismissed despite caring about the long-term future and believing AI is a really big deal. Would you even get many applicants to orgs you’d be hiring at that didn’t buy into this at least a little?
The reason I find the use of language being a bigger deal than actual values plausible is that I have humanities friends who are quicker than me at picking up new concepts (teaching maths to them is both extremely fun and a tiny bit depressing because they pick things up much quicker than I ever did, turns out group theory is easy, who knew? As an aside, group theory is my favourite thing to teach to smart people with very little maths background because there are quite a few hard-ish exercises that require very little pre-requisite knowledge).
They have great epistemics in conversations with me where there are high enough levels of trust that very different worldviews don’t make “the other person is trolling/talking in bad faith because they can’t see what just obviously logically follows” the number 1 most plausible hypothesis when everyone is talking past each other.
I couldn’t bring them to an EA event because when I simulate their experience, I see them being dismissed as having poor reasoning skills because they don’t speak fluent STEM or analytic philosophy.
I know that speaking rationalist is a good signal—I think speaking like a LessWronger tells you the person has probably read a lot of LessWrong and, therefore, you are likely to have a lot of common ground with them. However, it does limit the pool of candidates a lot if finding LessWrong fun to read is a prerequisite requirement (which probably requires both being exceptionally analytical and exceptionally unbothered by confrontational communication styles, among probably other personal traits that might not correlate well with ops skills).
I suspect for ops roles, being exceptionally analytical might not be fundamentally needed to be amazing (and I don’t find it implausible that being exceptionally analytical is anti-correlated with being mindblowingly incredible at ops).
I don’t normally think you should select for speaking fluent LessWrong jargon, and I have advocated for hiring senior ops staff who have read relatively little LessWrong.
Great (and also unsurprising so I’m now trying to work out why I felt the need to write the initial comment)
I think I wrote the initial comment less because I expected anyone to reflectively disagree and more because I think we all make snap judgements that maybe take conscious effort to notice and question.
I don’t expect anyone to advocate for people because they speak more jargon (largely because I think very highly of people in this community). I do expect it to be harder to understand someone who comes from a different cultural bubble and, therefore, harder to work out if they are aligned with your values enough. Jargon often gives precision that makes people more legible. Also human beings are pretty instinctively tribal and we naturally trust people who indicate in some way (e.g. in their language) they are more like us. I think it’s also easy for these things to get conflated (it’s hard to tell where a gut feeling comes from and once we have a gut feeling, we naturally are way more likely to have supporting arguments pop into our heads than opposing ones).
Anyway, I feel there is something I’m pointing to even if I’ve failed to articulate it.
Obviously EA hiring is pretty good because big things are getting accomplished and have already happened. I probably should have said initially that this does feel quite marginal. My guess as an outsider is that hiring is, overall, done quite a bit better than at the median non-profit organisation.
I think the reason it’s tempting to criticize EA orgs is because we’re all more invested in them being as good as they can possibly can be and so want to point out perceived flaws to improve them (though this instinct might often be counter-productive because it takes up scarce attention, so sorry about that!).
Part of why I think selecting for this trait is particularly bad if you want to find amazing ops people is I have a hunch that executive dysfunction is very bad for ops roles. I also suspect that executive dysfunction makes it easier to trick people in conversation/interviews into thinking you’re really smart and good at analytic reasoning.
I actually think that executive dysfunction and interest in EA are so well correlated that we could use ASD and ADHD diagnostic tools to identify people who are pre-disposed to falling down the EA/LessWrong/longtermism rabbit-hole. (I’m only half-joking, I legitimately think finding people without any executive dysfunctional traits who are interested in EA might be the root cause of some talent bottlenecks)
I am biased because I think I’m quite good at “tricking” people into thinking I’m smart in conversation and that halo-effecting me in EA has been bad and could have been a lot worse (cos at age 20 you just believe people when they tell you your doubts are imposter syndrome and not just self-awareness).
Don’t get me wrong, I love me (well, I actually have a very up and down relationship with myself but I generally have a lot more ups than downs and I have an incredible psychologist so hopefully I won’t need a caveat like this in future 🤣🤞).
I just think that competency is more multi-dimensional than many rationalist people seem to alieve.
(Writing quickly, sorry if I’m unclear)
Since you asked, here are my agreements and disagreements, mostly presented without argument:
As someone who is roughly in the target audience (I am involved in hiring for senior ops roles, though it’s someone else’s core responsibility), I think I disagree with much of this post (eg I think this isn’t as big a problem as you think, and the arguments around hiring from outside EA are weak), but in my experience it’s somewhat costly and quite low value to publicly disagree with posts like this, so I didn’t write anything.
It’s costly because people get annoyed at me.
It’s low value because inasmuch as think your advice is bad, I don’t really need to persuade you you’re wrong, I just need to persuade the people who this article is aimed at that you’re wrong. It’s generally much easier to persuade third parties than people who already have a strong opinion. And I don’t think that it’s that useful for the counterarguments to be provided publicly.
And if someone was running an org and strongly agreed with you, I’d probably shrug and say “to each their own” rather than trying that hard to talk them out of it: if a leader really feels passionate about shaping org culture a particular way, that’s a reasonable argument for them making the culture be that way.
For some of the things you talk about in this post (e.g. “The priority tasks are often mundane, not challenging”, ‘The role is mostly positioned as “enabling the existing leadership team” to the extent that it seems like “do all the tasks that we don’t like”’) I agree that it is bad inasmuch as EA orgs do this as egregiously as you’re describing. I’ve never seen this happen in an EA org as blatantly as you’re describing, but find it easy to believe that it happens.
However, if we talked through the details I think there’s a reasonable chance that I’d end up thinking that you were being unfair in your description.
I think one factor here is that some candidates are actually IMO pretty unreasonably opposed to ever doing grunt work. Sometimes jobs involve doing repetitive things for a while when they’re important. For example, I spoke to 60 people or so when I was picking applicants for the first MLAB, which was kind of repetitive but also seemed crucial. It’s extremely costly to accidentally hire someone who isn’t willing to do this kind of work, and it’s tricky to correctly communicate both “we’d like you to not mostly do repetitive work” and “we need you to sometimes do repetitive work, as we all do, because the most important tasks are sometimes repetitive”.
I think our main disagreement is that you’re more optimistic about getting people who “aren’t signed up to all EA/long-termist ideas” to help out with high level strategy decisions than I am. In my experience, people who don’t have a lot of the LTist context often have strong opinions about what orgs should do that don’t really make sense given more context.
For example, some strategic decisions I currently face are:
Should I try to hire more junior vs more senior researchers?
Who is the audience of our research?
Should I implicitly encourage or discourage working on weekends?
I think that people who don’t have experience in a highly analogous setting will often not have the context required to assess this, because these decisions are based on idiosyncrasies of our context and our goals. Senior people without relevant experience will have various potentially analogous experience, and I really appreciate the advice that I get from senior people who don’t have the context, but I definitely have to assess all of their advice for myself rather than just following their best practices (except on really obvious things).
If I was considering hiring a senior person who didn’t have analogous experience and also wanted to have a lot of input into org strategy, I’d be pretty scared if they didn’t seem really on board with the org leadership sometimes going against their advice, and I would want to communicate this extremely clearly to the candidate, to prevent mismatched expectations.
I think that the decisions that LTist orgs make are often predicated on LTist beliefs (obviously), and people who don’t agree with LTist beliefs are going to systematically disagree about what to do, and so if the org hires such a person, they need that person to be okay with getting overruled a bunch on high level strategy. I don’t really see how you could avoid this.
In general, I think that a lot of your concerns might be a result of orgs trying to underpromise and overdeliver: the orgs are afraid that you will come in expecting to have a bunch more strategic input than they feel comfortable promising you, and much less mundane work than you might occasionally have. (But probably some also comes from orgs making bad decisions.)
FWIW, this is also roughly my take on this post (and I felt a similar hesitation to Buck to objecting, plus being pretty busy the last few days).
I really appreciate your honest response—thanks for sticking your neck out.
I think you’ve layered on nuance / different perspectives which enables a richer understanding in some regards, and in some others I think we diverge mostly on how big a risk we perceive non-EAs in senior roles as presenting relative to value they bring. I think we might have fundamental disagreements about ‘the value of outside perspectives’ Vs. ‘the need for context to add value’; or put another way ‘the risk of an echo chamber from too-like-minded people’ Vs. ‘the risk of fracture and bad decision-making from not-like-minded-enough people’.
I’m going to have a think about what the most interesting / useful way to respond might be; I suspect it would be a bit dull / less useful to just rebutt point by point rather than get deeper, but don’t want to infer your drivers too much. Will likey build on this later in the week.
I agree that this is probably the crux.
First off, again lauding you and CarolineJ for throwing some challenge. I’ve tried to think about how to counter-argue by being critical of your arguments / their implications without sounding like I’m being critical of you two personally. I think it’s hard to do this, especially through this medium, so I apologise if I get this wrong.
So main counter-arguments...:
My point here wasn’t about quantifying how often it happens, or whether it’s a “fair” claim. It’s about wanting people to write job descriptions that will attract more / better candidates than they’re currently doing. Even if it doesn’t apply in 90% of cases, I thought it was important to make the point so people could think about the signals it sends out, as I laid out in the post.
I agree though that all jobs will involve some ‘grunt work’ or enabling others even at the cost of your own projects, and it’s important to signal that; the issue as I outlined is that a good candidate will think “they really don’t know how to get the best of me” if the JD is mostly that.
I think that is often the value they will add—paradigmatic challenge as well as practical insight. It’s not like the entire EA / LT house is built on solid epistemic foundations, let alone very clear theories of change / impact behind every single intervention. (I do think this is a dirty secret we don’t do well of owning up to, but maybe this is for another post). And if there were, it’s not as if people with outside experience wouldn’t do a good job of red-teaming them. I outlined examples in my original post about where outsiders would bring value in emerging / pre-paradigmatic fields, even if they are not fuller EA / LT signed up; I think they should be more strongly considered.
For me personally, I find perspectives of people outside of my field helpful for challenging my fundamental assumptions about how innovation actually works. Them being in the room makes it more likely a) they’ll get listened to, b) they have access to the same materials / research as me so we can see why we’re diverging on what appears to be the same information, and then check / challenge each others assumptions.
I know I should really listen when I feel uncomfortable; when I feel annoyed at what another team member is saying. It usually indicates they’ve struck a nerve of my uncertainty, or pointed out some fundamental assumption I’m resting too heavily upon. I’m not always good at doing it, but when I do it does make our work stronger.
There are some structural / cultural issues which pop up again and again in different workplaces. EA / LT might have more astronomically massive goals than most other orgs, but the mechanics of achieving them will have more in common with other teams / orgs than idiosyncracies; especially given most planning periods / real decisions never go beyond 5-10 years.
I can’t imagine why someone outside EA / LT would be constrained by ‘not being signed up enough’ from contributing to the strategic decision you mention? Some of them are pretty classic whatever team you’re in; like senior / junior researchers. The cultural ones—e.g. implicitly encouraging / discouraging weekend working—would especially benefit from outsiders who have experienced lots of team / org environments, and have better intuition for good / bad workplace cultures. I think so because, again, lots of EA orgs have really messed up their own cultures and left a trail of burn-out from it.
Reading this makes me think two things:
Isn’t that an argument for bringing outsiders into the organisation; so they can acquire the wider context, weigh it up along with their experience from other situations—some analogous along the lines you think and some analogous along different lines than you’d expect—and then add their thoughts to yours to come down on a decision? My bet is that would be more valuable to you than someone more similar to you doing the same to make those strategic decisions collaboratively.
I’m always sceptical about arguments that can be boiled down to “our context is different” or “our organisation is unique”. We all think that about our teams / orgs; we all think our constraints and challenges are very specific, but in reality most things going well / badly can be explained by some combination of clarity of purpose / vision, clarity of roles and responsibilities, and how people are actually working together[1].
But I hope this would be a two-way street? To be fair, I don’t know of any recruitment where this kind of negotiation of terms doesn’t happen to begin with. That’s normal. People recruit ‘outsiders’, different levels of autonomy are given, they see if they can work together, if it doesn’t they part ways; usually with the newbie leaving but sometimes with the incumbent leaving as the vision of the newbie carries more sway...
Working well with people you don’t 100% agree with is very possible, esp. if you’re optimised your hiring for certain qualities like openness and abilities to argue but maintain cohesion. It’s also just a really important leadership skill to build for most contexts.
Reading this, I felt a little like “what’s wrong about that?” Or more specifically “what’s wrong about having systematic disagreements?” Conflict / disagreements are good things! The more fundamental the better, provided you’re not rehashing the same ground on repeat (and there is a subtle difference between seeing the same argument ad nauseum compared with the same justifiable tensions being played out in different problem / solution spaces).
However, if your assumption is that the non-LT person would be overruled consistently, then yes that would be a problem because then you’re sacrificing the opportunity for synthesis or steel-manning. I feel that if I’m making really good LT arguments with a good theory of change, I should be able to convince someone who isn’t 100% signed up to that to come on the journey with me. If that person was being overruled again and again without any form of synthesis emerging, I’d take that as a sign that I was doing a rubbish job of listening and understanding another person’s perspective.
I think if I was working in such an LT org I would be terrified of not having a detractor in the room. This is because I think a lot of LT Vs. near-term conflict happens because of a lack of concrete theories of change being put forward by LTists. I would want a detractor challenging the assumptions behind my LT decisions, steel-manning at every step of the way. Personally, I would want them jumping up and down at me if it sounded like I was willing to sacrifice a lot of near-term high probability impact in the service of more spurious long-term gains. If I am going to do that, I want to be really confident I made that decision for the right reasons; I want to be so confident that the detractor can plausibly agree with me, and even leave the org on good terms if they could understand the decision being made but couldn’t abide by it.
I am sceptical that an advisor from outside the org would give that level of challenge; they don’t have the full context, they are not as invested in thinking about these things, they won’t have the bandwidth to really make the case. And I’m very sceptical that just really smart, conscientious people who also happen to share the same set of assumptions as me will do as good a job steelmanning. In practice, they should be able to, but in reality assumptions / blindspots need people who don’t have them to point them out.
I think two additional cruxes I’m realising we have might be:
I think really good organisational leadership is hard, and requires experience—ideally lots of experience, doing well and badly and reflecting on yourself and what it’s like to work with you. I think the leadership in many EA orgs have not had those opportunities, which I think is a big risk. But I think you and perhaps others see this as less the case?
I trust my judgement less if I am not getting serious challenge from people on strategic decisions, that shakes me both on the process and on the paradigm / values-system. Because I don’t think it’s possible I know everything, but that wisdom of crowds should get to better solution space. I think I might be unusually strong in this tendency, but I won’t put words in yours / others mouths on what the opposite looks like.
In fact, last two weeks I received a tonne of highly critical challenge fundamentally about whether my programme had a plausible theory of change. Though initially frustrating, it’s helped me see my work for what it is; ultimately experimental, and if it doesn’t work it must be killed off.
To finish, this is the type of thing I’d usually chat over in a tone / setting that’s relaxed and open and less point-by-point rebuttaly; because I think this type of topic is better as an open, reflective conversation but this medium sets it up to feel more confrontational. Ultimately, I don’t know what’s going on in anyone else’s head, or their full context, so I can only make observations / ask questions / pose challenge and hope they feel useful. Because I don’t think the medium can do this topic justice, I would be open to exploring in a different medium if need be.
And yes, part of my deciding to go for the delicate core is inspired by Scott Alexander’s recent post, and reinforced by Helen’s more recent post.
I totally over-summarised this bit in haste, but there’s tonnes of org literature on this, much of the best I read is by McKinsey on organisational health.
This counter-response updated my views quite a bit, thanks! Maybe a way to reconcile the two views in a specific example:
Strategical alignment/understanding: Ultimately at some level the answer to this question depends on what the organisation is trying to achieve and the philosophical trade-offs its leadership are willing to make.
Experience: But in order to make that decision, you have to know what the effects of more or less working on the weekends would actually be. And effects on organisational culture can be gradual, so you need to have observed this process to be able to predict it well.
You don’t necessarily need to have both capacities in the same person, as long as there’s appreciation of the respective areas of expertise.
tl;dr: I suspect selecting for speaking fluent LessWrong jargon is anti-correlated with being exceptionally good at ops.
Double-cruxing in the comments is so satisfying to read. I’ve outlined another possible related component in this comment that this post might have been pointing to (I find the below gets easily conflated in my head unless I do work to tease out the distinct pieces and so I thought this might plausibly be happening a bit here too).
I am curious how much actual values matter versus being good at EA/rationalist signalling mattering.
Eg. I think people take what I say a lot more seriously when I use jargon compared to when I say the same core idea in plain English or with a more accessible framing (which usually is a lot more work: saying what I believe is true in language I think a smart newcomer with a different academic background could understand is hard but often seems worth it to me).
I can imagine someone who gets vibed as a “normie” because of the language and framing they use (because they’ve spent more time in the real world than the EA world where different wording gets social reinforcement) being dismissed despite caring about the long-term future and believing AI is a really big deal. Would you even get many applicants to orgs you’d be hiring at that didn’t buy into this at least a little?
The reason I find the use of language being a bigger deal than actual values plausible is that I have humanities friends who are quicker than me at picking up new concepts (teaching maths to them is both extremely fun and a tiny bit depressing because they pick things up much quicker than I ever did, turns out group theory is easy, who knew? As an aside, group theory is my favourite thing to teach to smart people with very little maths background because there are quite a few hard-ish exercises that require very little pre-requisite knowledge).
They have great epistemics in conversations with me where there are high enough levels of trust that very different worldviews don’t make “the other person is trolling/talking in bad faith because they can’t see what just obviously logically follows” the number 1 most plausible hypothesis when everyone is talking past each other.
I couldn’t bring them to an EA event because when I simulate their experience, I see them being dismissed as having poor reasoning skills because they don’t speak fluent STEM or analytic philosophy.
I know that speaking rationalist is a good signal—I think speaking like a LessWronger tells you the person has probably read a lot of LessWrong and, therefore, you are likely to have a lot of common ground with them. However, it does limit the pool of candidates a lot if finding LessWrong fun to read is a prerequisite requirement (which probably requires both being exceptionally analytical and exceptionally unbothered by confrontational communication styles, among probably other personal traits that might not correlate well with ops skills).
I suspect for ops roles, being exceptionally analytical might not be fundamentally needed to be amazing (and I don’t find it implausible that being exceptionally analytical is anti-correlated with being mindblowingly incredible at ops).
I don’t normally think you should select for speaking fluent LessWrong jargon, and I have advocated for hiring senior ops staff who have read relatively little LessWrong.
Great (and also unsurprising so I’m now trying to work out why I felt the need to write the initial comment)
I think I wrote the initial comment less because I expected anyone to reflectively disagree and more because I think we all make snap judgements that maybe take conscious effort to notice and question.
I don’t expect anyone to advocate for people because they speak more jargon (largely because I think very highly of people in this community). I do expect it to be harder to understand someone who comes from a different cultural bubble and, therefore, harder to work out if they are aligned with your values enough. Jargon often gives precision that makes people more legible. Also human beings are pretty instinctively tribal and we naturally trust people who indicate in some way (e.g. in their language) they are more like us. I think it’s also easy for these things to get conflated (it’s hard to tell where a gut feeling comes from and once we have a gut feeling, we naturally are way more likely to have supporting arguments pop into our heads than opposing ones).
Anyway, I feel there is something I’m pointing to even if I’ve failed to articulate it.
Obviously EA hiring is pretty good because big things are getting accomplished and have already happened. I probably should have said initially that this does feel quite marginal. My guess as an outsider is that hiring is, overall, done quite a bit better than at the median non-profit organisation.
I think the reason it’s tempting to criticize EA orgs is because we’re all more invested in them being as good as they can possibly can be and so want to point out perceived flaws to improve them (though this instinct might often be counter-productive because it takes up scarce attention, so sorry about that!).
I have related thoughts on over-selecting for one single good-but-not-the-be-all-and-end-all trait (being exceptionally analytic) in the EA community in response to the ridiculously competent CEO of GWWC’s comment here: https://forum.effectivealtruism.org/posts/x9Rn5SfapcbbZaZy9/ea-for-dumb-people?commentId=noHWAsztWeJvijbGC
Part of why I think selecting for this trait is particularly bad if you want to find amazing ops people is I have a hunch that executive dysfunction is very bad for ops roles. I also suspect that executive dysfunction makes it easier to trick people in conversation/interviews into thinking you’re really smart and good at analytic reasoning.
I actually think that executive dysfunction and interest in EA are so well correlated that we could use ASD and ADHD diagnostic tools to identify people who are pre-disposed to falling down the EA/LessWrong/longtermism rabbit-hole. (I’m only half-joking, I legitimately think finding people without any executive dysfunctional traits who are interested in EA might be the root cause of some talent bottlenecks)
I am biased because I think I’m quite good at “tricking” people into thinking I’m smart in conversation and that halo-effecting me in EA has been bad and could have been a lot worse (cos at age 20 you just believe people when they tell you your doubts are imposter syndrome and not just self-awareness).
Don’t get me wrong, I love me (well, I actually have a very up and down relationship with myself but I generally have a lot more ups than downs and I have an incredible psychologist so hopefully I won’t need a caveat like this in future 🤣🤞).
I just think that competency is more multi-dimensional than many rationalist people seem to alieve.