[Update: This comment of mine was wrong, but I still think the claim in the post is contradicted by the source cited; see below.]
It looks like integrity and conscientiousness tests were also the 3rd and 4th highest rated things for the āValidityā column itself, out of a large list? And they appear to have ranked above interviews and some CV-like things (e.g., reference checks and job experience (years)), yet your post recommends interviews and CVs.
Iām pretty confused by this apparent misreading of the source. I think readers should treat that as a reason for being more skeptical of the rest of the post. My guess is itād be worth you āmoving a bit slowerā (to check sources more carefully etc.) and stating things less confidently in future posts.
They are the third and fourth row in the table, but the rows arenāt ordered by the validity column. When you order by the validity column, integrity tests are 8th and conscientiousness tests are 12th unless Iāve miscounted.
I admit that I cherry-picked this article, basically only looked at the validity numbers in the table, and donāt know anything else from that literature. This post has a wider view for those interested: https://āāforum.effectivealtruism.org/āāposts/āāj9JnmEZtgKcpPj8kz/āāhiring-how-to-do-it-better On the other hand, the validity table was excerpted on an 80,000 Hours page for years, which gives me some confidence by proxy. Also, the points of my post rely only lightly on this article.
Added 2022-10-19: It would be nice if you removed the bolding on the text that you found to be wrong. Otherwise the impatient reader is apt to miss the bracketed text above.
Oh crap, my bad, shouldāve looked closer! (And thatās ironic given my comment, although at least Iād say the epistemic standards for comments can/āshould be lower than for posts.) Sorry about that.
Though still I think ānot predictiveā seems like a strong misrepresentation of that source. I think correlations of .41 and .31 would typically be regarded as moderate/āweak and definitely not āapproximately zeroā. (And then also the apparently high incremental validity is interesting, though not as immediately easy to figure out the implications of.)
I agree that āthe points of [your] post rely only lightly on this articleā. But I still think this is a sufficiently strong & (in my view, seemingly) obvious* misrepresentation of the source that it would make sense to see this issue as a reason for readers to be skeptical of any other parts of the post that they havenāt closely checked.
(I also find it surprising that in your two replies in this thread you didnāt note that the table indeed seems inconsistent with ānot predictiveā.)
*I.e., it doesnāt seem like it requires a very close/ācareful reading of the source to determine that itās saying something very different to ānot predictiveā. (Though I may be wrongāI only jumped to the table, and I guess itās possible other parts of the paper are very misleadingly written.)
Okay, you convince me. Iāve rewritten that item.
The reason why I wasnāt noting that the table is inconsistent with ānot predictiveā is that I was unconsciously equating ānot predictiveā with ānot sufficiently predictive to be worth the candidateās timeā. Only your insisting made me think about it more carefully. Given that unconscious semantics, itās not a strong misrepresentation of the source either. But of course itās sloppy and you were right to point it out.
I hope this somewhat restores your expectation of my epistemic integrity. Also, I think there is not just evidence against, but also evidence for epistemic integrity in my article. That should factor into how āskepticalā readers ought to be. Examples: The last paragraph of the introduction. The fact that I edit the article based on comments once Iām convinced that a comment is correct. The fact that I call out edits and donāt just rewrite history. The fact that itās well-structured overall (not necessarily at the paragraph level), which makes it easy to respond to claims. The fact that I include and address possible objections to my points.
Addendum: I would explain the high incremental validity by the fact that a GMA test barely measures conscientiousness and integrity. In fact, footnote āc,dā mentions that āthe correlation between integrity and ability is zeroā. But conscientiousness and integrity are important for job performance (depending on the job). I would expect much lower incremental validity over structured interviews or work samples. Because these, when done well, tell a lot about conscientiousness and integrity by themselves.
[Update: This comment of mine was wrong, but I still think the claim in the post is contradicted by the source cited; see below.]
It looks like integrity and conscientiousness tests were also the 3rd and 4th highest rated things for the āValidityā column itself, out of a large list? And they appear to have ranked above interviews and some CV-like things (e.g., reference checks and job experience (years)), yet your post recommends interviews and CVs.
Iām pretty confused by this apparent misreading of the source. I think readers should treat that as a reason for being more skeptical of the rest of the post. My guess is itād be worth you āmoving a bit slowerā (to check sources more carefully etc.) and stating things less confidently in future posts.
They are the third and fourth row in the table, but the rows arenāt ordered by the validity column. When you order by the validity column, integrity tests are 8th and conscientiousness tests are 12th unless Iāve miscounted.
I admit that I cherry-picked this article, basically only looked at the validity numbers in the table, and donāt know anything else from that literature. This post has a wider view for those interested: https://āāforum.effectivealtruism.org/āāposts/āāj9JnmEZtgKcpPj8kz/āāhiring-how-to-do-it-better On the other hand, the validity table was excerpted on an 80,000 Hours page for years, which gives me some confidence by proxy. Also, the points of my post rely only lightly on this article.
Added 2022-10-19: It would be nice if you removed the bolding on the text that you found to be wrong. Otherwise the impatient reader is apt to miss the bracketed text above.
Oh crap, my bad, shouldāve looked closer! (And thatās ironic given my comment, although at least Iād say the epistemic standards for comments can/āshould be lower than for posts.) Sorry about that.
Though still I think ānot predictiveā seems like a strong misrepresentation of that source. I think correlations of .41 and .31 would typically be regarded as moderate/āweak and definitely not āapproximately zeroā. (And then also the apparently high incremental validity is interesting, though not as immediately easy to figure out the implications of.)
I agree that āthe points of [your] post rely only lightly on this articleā. But I still think this is a sufficiently strong & (in my view, seemingly) obvious* misrepresentation of the source that it would make sense to see this issue as a reason for readers to be skeptical of any other parts of the post that they havenāt closely checked.
(I also find it surprising that in your two replies in this thread you didnāt note that the table indeed seems inconsistent with ānot predictiveā.)
*I.e., it doesnāt seem like it requires a very close/ācareful reading of the source to determine that itās saying something very different to ānot predictiveā. (Though I may be wrongāI only jumped to the table, and I guess itās possible other parts of the paper are very misleadingly written.)
Particularly when the most predictive things in that table were .51 and .54.
Okay, you convince me. Iāve rewritten that item.
The reason why I wasnāt noting that the table is inconsistent with ānot predictiveā is that I was unconsciously equating ānot predictiveā with ānot sufficiently predictive to be worth the candidateās timeā. Only your insisting made me think about it more carefully. Given that unconscious semantics, itās not a strong misrepresentation of the source either. But of course itās sloppy and you were right to point it out.
I hope this somewhat restores your expectation of my epistemic integrity. Also, I think there is not just evidence against, but also evidence for epistemic integrity in my article. That should factor into how āskepticalā readers ought to be. Examples: The last paragraph of the introduction. The fact that I edit the article based on comments once Iām convinced that a comment is correct. The fact that I call out edits and donāt just rewrite history. The fact that itās well-structured overall (not necessarily at the paragraph level), which makes it easy to respond to claims. The fact that I include and address possible objections to my points.
Addendum: I would explain the high incremental validity by the fact that a GMA test barely measures conscientiousness and integrity. In fact, footnote āc,dā mentions that āthe correlation between integrity and ability is zeroā. But conscientiousness and integrity are important for job performance (depending on the job). I would expect much lower incremental validity over structured interviews or work samples. Because these, when done well, tell a lot about conscientiousness and integrity by themselves.