Severin—thanks for an interesting post, and a useful distinction between community-building and movement-building.
I would just offer a couple of cautions about the Google Project Aristotle, which has many of the same methodological, statistical, and theoretical problems of other social psychology and organizational psychology research that I’m familiar with, that try to assess the ‘key ingredients’ of successful groups.
First, when Google finds (as summarized by you) that ‘It is not the fanciness of degrees or individual intelligence or agentyness or any other property of the individual team members, but five factors:....‘, it’s crucial to bear in mind that Google is one of the hardest companies in the world to work for. They’re incredibly selective about the general intelligence, conscientiousness, and social skills of their employees. Basically, their workers are all pretty ‘close to ceiling’ in terms of the cognitive and personality traits that predict work success. So, lacking much variance in those traits (e.g. a severe ‘restriction of range’ with respect to individual intelligence), of course they’ll find that differences in individual intelligence across groups don’t predict much (although it’s not clear to me they actually assessed this quantitatively in their study.)
Second, there are strong ideological and theoretical biases in organizational behavior research to show that group organization, norms, management, and corporate culture have big effects on group efficiency, and to downplay the role of individual differences (e.g. intelligence, Big Five personality traits, empathizing vs. systemizing styles, etc). This bias has been a problem for many decades. For example, intelligence researchers tend to find that the average level of individual intelligence within a group has a huge effect on group effectiveness, and that group-level dynamics (such as ‘psychological safety’) aren’t as important. Likewise, personality psychologists often find that the average levels of Conscientiousness, Agreeableness, Emotional Stability, and Openness within a group strongly predict group-level dynamics.
It’s also worth noting that some of the things that sound like group-level dynamics are actually aggregates of individual traits—e.g. ‘dependability’ is basically a sum of individual levels of the Big Five trait Conscientiousness, and ‘structure and clarity’ (‘An individual’s understanding of job expectations, the process for fulfilling these expectations, and the consequences of one’s performance’) sounds heavily dependent individual levels of general intelligence.
So, overall, I would just suggest some caution in how we interpret studies down by organizational psychologists inside large corporations—especially if the corporation is extremely selective about who they hire. (Counterpoint: many EAs might be about as smart as Google employees, so the restriction-of-range issue might apply as strongly within EA as within Google.)
Thank you so much for your comments and your expertise on this subject.
Some mild-moderate disagreements I have with your claims/framing in your comment above (which should be understood locally as disagreements with this comment, not an endorsement of the original post or Project Aristotle):
First, when Google finds (as summarized by you) that ‘It is not the fanciness of degrees or individual intelligence or agentyness or any other property of the individual team members, but five factors:....’, it’s crucial to bear in mind that Google is one of the hardest companies in the world to work for. They’re incredibly selective about the general intelligence, conscientiousness, and social skills of their employees.
I have a few disagreements here. The first is as you say, both EA-in-practice and Google are quite selective on some traits like intelligence. So to the extent that Google’s data is only useful in a range-restricted setting, they should be somewhat applicable to EA as well.
FWIW it is not at all ex ante obvious to me that (eg) variation in intelligence is less predictive of work output than group dynamics, for employees above Google’s bar (certainly I’ve heard many people argue the opposite).
Secondly I’m anecdotally pretty skeptical that Google is selective on social skills of their employees, particularly among the engineers. In my time there(2018-2019), I didn’t notice people being particularly astute or sociable, compared to a less restricted sample like random Uber drivers. I expect there’s some truth to classical stereotypes of engineers/software developers.
I’m also somewhat skeptical that they select heavily on conscientiousness in the Big Five sense of hardworkingness or orderliness. In Silicon Valley, there are a lot of memes and jokes about how nobody gets any work done in Big Tech in general and Google specifically. Obviously exaggerated, but certainly there’s some truth to it. (Also they couldn’t be that selective on hard work since they hired me lol).
I think what I say above is noticeably more true for American employees than immigrant ones, at least anecdotally. Also, there are good theoretical reasons for this. (I expect second-gen people to be somewhere in between).
Second, there are strong ideological and theoretical biases in organizational behavior research to show that group organization, norms, management, and corporate culture have big effects on group efficiency, and to downplay the role of individual differences (e.g. intelligence, Big Five personality traits, empathizing vs. systemizing styles, etc). This bias has been a problem for many decades. For example, intelligence researchers tend to find that the average level of individual intelligence within a group has a huge effect on group effectiveness, and that group-level dynamics (such as ‘psychological safety’) aren’t as important. Likewise, personality psychologists often find that the average levels of Conscientiousness, Agreeableness, Emotional Stability, and Openness within a group strongly predict group-level dynamics.
I don’t know if you intended your comment this way, but of course your critique for ideological or incentive biases of organizational behavior research should also apply to intelligence and personality researchers (we have straightforward theoretical reasons to think intelligence researchers will overstate the impact of intelligence on work performance, personality researchers will overstate the impact of personality traits, etc). So a bias argument unfortunately isn’t enough for outsiders to judge the truth here without digging into the details. :/
Insofar as both Google and EA are quite range-restricted in terms of people having very high intelligence, conscientiousness, & openness, organizational behavior lessons from the former might transfer to the latter, if they’re empirically solid.
I would quibble with your claim that Google doesn’t select much on ‘conscientiousness in the Big Five sense of hardworkingness or orderliness’. I think there’s a tendency of people who are moderately high on conscientiousness to compare themselves to the most driven, workaholic, ambitious people they know, and to feel like they fall short of what they could be doing.
However, everyone who’s taught students at a large state university (like me), or managed a business, or hired contractors to renovate a house, will be better-calibrated to the full range of conscientiousness out there in the population. Compared to that distribution, I would bet that almost everybody selected to work at Google will be in the top 10% of conscientiousness—if only because they’re often selected partly for achieving high GPAs at elite universities where you have to work pretty hard to get good grades.
Also, computer programming requires a very high level of conscientiousness. The six main facets of conscientiousness are ‘Competence, Order, Dutifulness, Achievement Striving, Self-Discipline, and Deliberation’. Serious coding requires basically all of these—whereas incompetent, disorganized, slapdash, unmotivated, undisciplined, and unthoughtful programmers simply won’t write good code. This also fits with Simon Baron-Cohen’s observation that coders tend to be much stronger on ‘Systemizing’ than on ‘Empathizing’—and systemizing seems closely related to detail-oriented conscientiousness.
Severin—thanks for an interesting post, and a useful distinction between community-building and movement-building.
I would just offer a couple of cautions about the Google Project Aristotle, which has many of the same methodological, statistical, and theoretical problems of other social psychology and organizational psychology research that I’m familiar with, that try to assess the ‘key ingredients’ of successful groups.
First, when Google finds (as summarized by you) that ‘It is not the fanciness of degrees or individual intelligence or agentyness or any other property of the individual team members, but five factors:....‘, it’s crucial to bear in mind that Google is one of the hardest companies in the world to work for. They’re incredibly selective about the general intelligence, conscientiousness, and social skills of their employees. Basically, their workers are all pretty ‘close to ceiling’ in terms of the cognitive and personality traits that predict work success. So, lacking much variance in those traits (e.g. a severe ‘restriction of range’ with respect to individual intelligence), of course they’ll find that differences in individual intelligence across groups don’t predict much (although it’s not clear to me they actually assessed this quantitatively in their study.)
Second, there are strong ideological and theoretical biases in organizational behavior research to show that group organization, norms, management, and corporate culture have big effects on group efficiency, and to downplay the role of individual differences (e.g. intelligence, Big Five personality traits, empathizing vs. systemizing styles, etc). This bias has been a problem for many decades. For example, intelligence researchers tend to find that the average level of individual intelligence within a group has a huge effect on group effectiveness, and that group-level dynamics (such as ‘psychological safety’) aren’t as important. Likewise, personality psychologists often find that the average levels of Conscientiousness, Agreeableness, Emotional Stability, and Openness within a group strongly predict group-level dynamics.
It’s also worth noting that some of the things that sound like group-level dynamics are actually aggregates of individual traits—e.g. ‘dependability’ is basically a sum of individual levels of the Big Five trait Conscientiousness, and ‘structure and clarity’ (‘An individual’s understanding of job expectations, the process for fulfilling these expectations, and the consequences of one’s performance’) sounds heavily dependent individual levels of general intelligence.
So, overall, I would just suggest some caution in how we interpret studies down by organizational psychologists inside large corporations—especially if the corporation is extremely selective about who they hire. (Counterpoint: many EAs might be about as smart as Google employees, so the restriction-of-range issue might apply as strongly within EA as within Google.)
Thank you so much for your comments and your expertise on this subject.
Some mild-moderate disagreements I have with your claims/framing in your comment above (which should be understood locally as disagreements with this comment, not an endorsement of the original post or Project Aristotle):
I have a few disagreements here. The first is as you say, both EA-in-practice and Google are quite selective on some traits like intelligence. So to the extent that Google’s data is only useful in a range-restricted setting, they should be somewhat applicable to EA as well.
FWIW it is not at all ex ante obvious to me that (eg) variation in intelligence is less predictive of work output than group dynamics, for employees above Google’s bar (certainly I’ve heard many people argue the opposite).
Secondly I’m anecdotally pretty skeptical that Google is selective on social skills of their employees, particularly among the engineers. In my time there(2018-2019), I didn’t notice people being particularly astute or sociable, compared to a less restricted sample like random Uber drivers. I expect there’s some truth to classical stereotypes of engineers/software developers.
I’m also somewhat skeptical that they select heavily on conscientiousness in the Big Five sense of hardworkingness or orderliness. In Silicon Valley, there are a lot of memes and jokes about how nobody gets any work done in Big Tech in general and Google specifically. Obviously exaggerated, but certainly there’s some truth to it. (Also they couldn’t be that selective on hard work since they hired me lol).
I think what I say above is noticeably more true for American employees than immigrant ones, at least anecdotally. Also, there are good theoretical reasons for this. (I expect second-gen people to be somewhere in between).
I don’t know if you intended your comment this way, but of course your critique for ideological or incentive biases of organizational behavior research should also apply to intelligence and personality researchers (we have straightforward theoretical reasons to think intelligence researchers will overstate the impact of intelligence on work performance, personality researchers will overstate the impact of personality traits, etc). So a bias argument unfortunately isn’t enough for outsiders to judge the truth here without digging into the details. :/
Linch—these are mostly fair points.
Insofar as both Google and EA are quite range-restricted in terms of people having very high intelligence, conscientiousness, & openness, organizational behavior lessons from the former might transfer to the latter, if they’re empirically solid.
I would quibble with your claim that Google doesn’t select much on ‘conscientiousness in the Big Five sense of hardworkingness or orderliness’. I think there’s a tendency of people who are moderately high on conscientiousness to compare themselves to the most driven, workaholic, ambitious people they know, and to feel like they fall short of what they could be doing.
However, everyone who’s taught students at a large state university (like me), or managed a business, or hired contractors to renovate a house, will be better-calibrated to the full range of conscientiousness out there in the population. Compared to that distribution, I would bet that almost everybody selected to work at Google will be in the top 10% of conscientiousness—if only because they’re often selected partly for achieving high GPAs at elite universities where you have to work pretty hard to get good grades.
Also, computer programming requires a very high level of conscientiousness. The six main facets of conscientiousness are ‘Competence, Order, Dutifulness, Achievement Striving, Self-Discipline, and Deliberation’. Serious coding requires basically all of these—whereas incompetent, disorganized, slapdash, unmotivated, undisciplined, and unthoughtful programmers simply won’t write good code. This also fits with Simon Baron-Cohen’s observation that coders tend to be much stronger on ‘Systemizing’ than on ‘Empathizing’—and systemizing seems closely related to detail-oriented conscientiousness.
Yep, all of those are valid points. Thanks!