It feels a bit silly to post this as a reply to you, because you’re probably either aware of all of this or, otherwise, could convince me that I’m mistaken. But your comment prompted me to think of these points. So I think it’s more useful for other readers who come across this thread.
I kind of tend to be a bit disappointed with correlations in reality, maybe because of this effect (or #2 and #3 below).
You probably have excellent intuitions when it comes to what correlations really mean, but I suspect that that takes a lot of statistical training. When I think that something is correlated, the scatter plot I have in my mind is around r = 0.95. Then I need to remind myself that that’s not how reality usually works, so it’s probably just r = 0.7 or even r = 0.5, at which point it’s starting to get hard to even see the correlation in the scatter plot. Just eyeballing it from an online correlation simulator it looks like you get samples that are below average on one dimension even among just the top 5% of samples along the other dimension. I’m guessing that a lot of people intuitively overestimate how correlated r = 0.7 really is and update too strongly on it.
This seems to be compounded by the halo effect. Sometimes I meet really impressively smart people, and then I’m so shocked when they turn out to be quite irrational about a lot of stuff, at least according to me. ^.^ My level of intuitive shockedness in turn strikes me as a bit irrational too, so I suspect the above overestimation of correlations and some halo effect are at work there.
There’s probably a lot of value in combining multiple virtues just as there is great value in combining multiple skills. I’ve read an amazing article somewhere (maybe by Ozzie or Kat?) that argued that if you master five skills to a normal level of mastery, you may plausibly be the one best-qualified person in the world for jobs that combine these skills.
I wish I knew how to quantify this right, but my intuition is that the 11 qualities in this article would have to be very, very highly correlated or else there’ll be almost no one who excels at all of them. Maybe there’ll be a few more people who are at least decent at all of them. But some of them also function as multiplies of someone’s power, so if they are very powerful because of the qualities they do have but then lack self-reflection, kindness, respect or are addicted to contrarianism or signaling, that can be catastrophic. So the badness of failures of the correlation reasoning can make it worthwhile to not rely on it exclusively. (To use an unnecessarily violent metaphor: A kind, self-reflected, respectful person with an (for our community) normal IQ may be more like a well-aimed shotgun. A high-IQ person who lacks just one virtue may be like a high-powered unaimed sniper rifle with a loose trigger.)
The researchers have probably thought of this and you know all of this stuff better than me anyway, but, not having had the time to investigate it, I idly wonder how many of these correlation experiments really measure some fairly uninteresting common cause. Like, I looove solving Raven’s matrices! They give me the most amazing flow experience, and when I can’t solve them in the allotted time, I can’t help but keep trying for hours afterwards. A friend of mine hated them. Sure enough she got a lower score in the same test, but I don’t think I can really derive anything from that that I didn’t already know. So these studies probably need to correct for countless things like need for cognition and such.
Focusing on one metric because all the others are correlated with it anyway is also quite exploitable. If someone is low on altruism but really high on IQ, they may be particularly drawn to exploiting mechanisms for personal gain that focus too much on IQ. So such mechanisms would then lead to a distribution of outcomes where there is one bump for the desired outcome and one bump for all the people willing and able to exploit the mechanism. That may be fine if the desired people add much more value than the exploitative people can take away, but in communities and among employees it’s usually the opposite.
It feels a bit silly to post this as a reply to you, because you’re probably either aware of all of this or, otherwise, could convince me that I’m mistaken. But your comment prompted me to think of these points. So I think it’s more useful for other readers who come across this thread.
I kind of tend to be a bit disappointed with correlations in reality, maybe because of this effect (or #2 and #3 below).
You probably have excellent intuitions when it comes to what correlations really mean, but I suspect that that takes a lot of statistical training. When I think that something is correlated, the scatter plot I have in my mind is around r = 0.95. Then I need to remind myself that that’s not how reality usually works, so it’s probably just r = 0.7 or even r = 0.5, at which point it’s starting to get hard to even see the correlation in the scatter plot. Just eyeballing it from an online correlation simulator it looks like you get samples that are below average on one dimension even among just the top 5% of samples along the other dimension. I’m guessing that a lot of people intuitively overestimate how correlated r = 0.7 really is and update too strongly on it.
This seems to be compounded by the halo effect. Sometimes I meet really impressively smart people, and then I’m so shocked when they turn out to be quite irrational about a lot of stuff, at least according to me. ^.^ My level of intuitive shockedness in turn strikes me as a bit irrational too, so I suspect the above overestimation of correlations and some halo effect are at work there.
There’s probably a lot of value in combining multiple virtues just as there is great value in combining multiple skills. I’ve read an amazing article somewhere (maybe by Ozzie or Kat?) that argued that if you master five skills to a normal level of mastery, you may plausibly be the one best-qualified person in the world for jobs that combine these skills.
I wish I knew how to quantify this right, but my intuition is that the 11 qualities in this article would have to be very, very highly correlated or else there’ll be almost no one who excels at all of them. Maybe there’ll be a few more people who are at least decent at all of them. But some of them also function as multiplies of someone’s power, so if they are very powerful because of the qualities they do have but then lack self-reflection, kindness, respect or are addicted to contrarianism or signaling, that can be catastrophic. So the badness of failures of the correlation reasoning can make it worthwhile to not rely on it exclusively. (To use an unnecessarily violent metaphor: A kind, self-reflected, respectful person with an (for our community) normal IQ may be more like a well-aimed shotgun. A high-IQ person who lacks just one virtue may be like a high-powered unaimed sniper rifle with a loose trigger.)
The researchers have probably thought of this and you know all of this stuff better than me anyway, but, not having had the time to investigate it, I idly wonder how many of these correlation experiments really measure some fairly uninteresting common cause. Like, I looove solving Raven’s matrices! They give me the most amazing flow experience, and when I can’t solve them in the allotted time, I can’t help but keep trying for hours afterwards. A friend of mine hated them. Sure enough she got a lower score in the same test, but I don’t think I can really derive anything from that that I didn’t already know. So these studies probably need to correct for countless things like need for cognition and such.
Focusing on one metric because all the others are correlated with it anyway is also quite exploitable. If someone is low on altruism but really high on IQ, they may be particularly drawn to exploiting mechanisms for personal gain that focus too much on IQ. So such mechanisms would then lead to a distribution of outcomes where there is one bump for the desired outcome and one bump for all the people willing and able to exploit the mechanism. That may be fine if the desired people add much more value than the exploitative people can take away, but in communities and among employees it’s usually the opposite.