I think the language of a person who “achieves orders of magnitude more” suggests that their output (research, book, etc) is orders of magnitude better, instead of just being more popular. Sometimes more popular is better, but often in EA that’s not what we’re focused on.
I also believe you’re talking about hiring individuals in this piece(?), but most of your examples are about successful teams, which have different qualities to successful individuals.
I thought your examples of Math Olympiad scores correlating with Nobel Prize wins was a useful exception to this trend, because those are about an individual and aren’t just about popularity.
FWIW I think I see the distinction between popularity and other qualities as less clear as you seem to do. For instance, I would expect that book sales and startup returns are also affected by how “good” in whatever other sense the book or startup product is. Conversely, I would guess that realistically Nobel Prizes and other scientific awards are also about popularity and not just about the quality of the scientific work by other standards. I’m happy to concede that, in some sense, book sales seem more affected by popularity than Nobel Prizes, but it seems a somewhat important insight to me that neither is “just about popularity” nor “just about achievement/talent/quality/whatever”.
It’s also not that clear to me whether there is an obviously more adequate standard of overall “goodness” here: how much joy the book brings readers? What literary critics would say about the book? I think the ultimate lesson here is that the choice of metric is really important, and depends a lot on what you want to know or decide, which is why “Carefully choose the underlying population and the metric for performance” is one of our key points of advice. I can see that saying something vague and general like “some people achieve more” and then giving examples of specific metrics pushes against this insight by suggesting that these are the metrics we should generally most care about. FWIW I still feel OK about our wording here since I feel like in an opening paragraph we need to balance nuance/detail and conciseness / getting the reader interested.
As an aside, my vague impression is that it’s somewhat controversial to what extent successful teams have different qualities to successful individuals. In some sense this is of course true since there are team properties that don’t even make sense for individuals. However, my memory is that for a while there was some more specific work in psychology that was allegedly identifying properties that predicted team success better than the individual abilities of its members, which then largely didn’t replicate.
However, my memory is that for a while there was some more specific work in psychology that was allegedly identifying properties that predicted team success better than the individual abilities of its members, which then largely didn’t replicate.
Woolley et al (2010) was an influential paper arguing that individual intelligence doesn’t predict collective intelligence well. Here’s one paper criticising them. I’m sure there are plenty of other relevant papers (I seem to recall one paper providing positive evidence that individual intelligence predicted group performance fairly well, but can’t find it now).
I think the language of a person who “achieves orders of magnitude more” suggests that their output (research, book, etc) is orders of magnitude better, instead of just being more popular. Sometimes more popular is better, but often in EA that’s not what we’re focused on.
I also believe you’re talking about hiring individuals in this piece(?), but most of your examples are about successful teams, which have different qualities to successful individuals.
I thought your examples of Math Olympiad scores correlating with Nobel Prize wins was a useful exception to this trend, because those are about an individual and aren’t just about popularity.
Thanks for clarifying!
FWIW I think I see the distinction between popularity and other qualities as less clear as you seem to do. For instance, I would expect that book sales and startup returns are also affected by how “good” in whatever other sense the book or startup product is. Conversely, I would guess that realistically Nobel Prizes and other scientific awards are also about popularity and not just about the quality of the scientific work by other standards. I’m happy to concede that, in some sense, book sales seem more affected by popularity than Nobel Prizes, but it seems a somewhat important insight to me that neither is “just about popularity” nor “just about achievement/talent/quality/whatever”.
It’s also not that clear to me whether there is an obviously more adequate standard of overall “goodness” here: how much joy the book brings readers? What literary critics would say about the book? I think the ultimate lesson here is that the choice of metric is really important, and depends a lot on what you want to know or decide, which is why “Carefully choose the underlying population and the metric for performance” is one of our key points of advice. I can see that saying something vague and general like “some people achieve more” and then giving examples of specific metrics pushes against this insight by suggesting that these are the metrics we should generally most care about. FWIW I still feel OK about our wording here since I feel like in an opening paragraph we need to balance nuance/detail and conciseness / getting the reader interested.
As an aside, my vague impression is that it’s somewhat controversial to what extent successful teams have different qualities to successful individuals. In some sense this is of course true since there are team properties that don’t even make sense for individuals. However, my memory is that for a while there was some more specific work in psychology that was allegedly identifying properties that predicted team success better than the individual abilities of its members, which then largely didn’t replicate.
Woolley et al (2010) was an influential paper arguing that individual intelligence doesn’t predict collective intelligence well. Here’s one paper criticising them. I’m sure there are plenty of other relevant papers (I seem to recall one paper providing positive evidence that individual intelligence predicted group performance fairly well, but can’t find it now).
Great, thank you! I do believe work by Woolley was what I had in mind.