Thanks for writing this! I think there’s something here, but some grumbles about Project Aristotle:
First, a naïve interpretation of the results is something like “psychological safety causes you to make more money” but that’s not what they were actually measuring. From this page:
Executives were most concerned with results (e.g., sales numbers or product launches), but team members said that team culture was the most important measure of team effectiveness. Fittingly, the team lead’s concept of effectiveness spanned both the big picture and the individuals’ concerns saying that ownership, vision, and goals were the most important measures.
So they are defining employee satisfaction and culture as “success”, which is fine, but makes the results close to tautological.[1]
Secondly, as far as I can tell they have never released any quantitative results. My (perhaps cynical) interpretation is that this is because the results were not impressive.
Thirdly, when people have attempted to do quantitative analyses they have mixed results. E.g. this meta-analysis of healthcare psychological safety studies finds that only 9⁄62 published papers found a significant effect of psychological safety; given publication bias, this is close to a null result.
Lastly, a near universal problem with regression analyses of this kind is halo effects. If you have an idea that was shot down by your manager but it later turned out to have been a great idea, then your manager “didn’t give you space to explore your ideas.” But if your idea ex-post turned out to be dumb, then she was giving you “helpful feedback”. The apparent explanatory power of “psychological safety” comes from descriptions that are superficially about process actually being descriptions of outcomes. My understanding of the expert consensus is that regression analyses of the type Google is trying to do here are doomed to failure because of confounding effects like this, which makes one wonder about the epistemic environment in which the project was undertaken.
Anyway, reversed stupidity is not intelligence, I do think psychological safety is important and maybe is useful for EA. But I would be hesitant to rely much on Project Aristotle.
They list sales results as one of the four things they measure, but do not provide the weights. My cynical guess is that they found that sales do not correlate particularly well with psychological safety, but I don’t think we can say based on publicly available information.
Thanks for writing this! I think there’s something here, but some grumbles about Project Aristotle:
First, a naïve interpretation of the results is something like “psychological safety causes you to make more money” but that’s not what they were actually measuring. From this page:
So they are defining employee satisfaction and culture as “success”, which is fine, but makes the results close to tautological.[1]
Secondly, as far as I can tell they have never released any quantitative results. My (perhaps cynical) interpretation is that this is because the results were not impressive.
Thirdly, when people have attempted to do quantitative analyses they have mixed results. E.g. this meta-analysis of healthcare psychological safety studies finds that only 9⁄62 published papers found a significant effect of psychological safety; given publication bias, this is close to a null result.
Lastly, a near universal problem with regression analyses of this kind is halo effects. If you have an idea that was shot down by your manager but it later turned out to have been a great idea, then your manager “didn’t give you space to explore your ideas.” But if your idea ex-post turned out to be dumb, then she was giving you “helpful feedback”. The apparent explanatory power of “psychological safety” comes from descriptions that are superficially about process actually being descriptions of outcomes. My understanding of the expert consensus is that regression analyses of the type Google is trying to do here are doomed to failure because of confounding effects like this, which makes one wonder about the epistemic environment in which the project was undertaken.
Anyway, reversed stupidity is not intelligence, I do think psychological safety is important and maybe is useful for EA. But I would be hesitant to rely much on Project Aristotle.
They list sales results as one of the four things they measure, but do not provide the weights. My cynical guess is that they found that sales do not correlate particularly well with psychological safety, but I don’t think we can say based on publicly available information.
Oh dear. Well, there goes that bit of evidence out the window.