Thanks very much for sharing this, and in particular the fascinating charts. I was pretty surprised at how large a fraction of successful applicants were, on these axis, strictly dominated by other rejected applicants, and how large the overlap was in the box and whiskers plots. Sometimes colleges argue they can’t just look at SAT because they have more applicants with perfect SATs than they have spaces, but that doesn’t explain why you would almost all your successful applicants (from this school) would have sub-perfect SATs.
I have have thoughts that could perhaps change the conclusion:
It’s well known that colleges care a lot about extracurriculars. If these really are a good sign of flexibility, work ethics, initiative and so on, perhaps we should care about them also. If so, colleges might be correctly adjusting, the low correlations we observe in the charts are just because we can’t directly observe those facts, and high quality people are more concentrated in top schools than this data would suggest.
Additionally, SCOTUS is due to hear Students For Fair Admissions vs Harvard later this year, and Metaculus currently gives them a 75% chance to successfully get racial discrimination in university admissions found unlawful. If so the correlation with SAT/GPA might improve a lot after this year, so the phenomena you’re highlighting might be a relatively short-lived one.
Sometimes colleges argue they can’t just look at SAT because they have more applicants with perfect SATs than they have spaces, but that doesn’t explain why you would almost all your successful applicants (from this school) would have sub-perfect SATs.
If the SATs are normed to a lower level, you might expect that doing perfectly on them might not be super predictive of outlier ability, compared to just doing extremely well.
Analogously, imagine a test for Alzheimer’s with 100 questions. Even if most people without Alzheimer’s only get <90/100 questions right on them, we might not expect there’s that much extra predictive validity between 99 and 100 right. (see also).
Thanks very much for sharing this, and in particular the fascinating charts. I was pretty surprised at how large a fraction of successful applicants were, on these axis, strictly dominated by other rejected applicants, and how large the overlap was in the box and whiskers plots. Sometimes colleges argue they can’t just look at SAT because they have more applicants with perfect SATs than they have spaces, but that doesn’t explain why you would almost all your successful applicants (from this school) would have sub-perfect SATs.
I have have thoughts that could perhaps change the conclusion:
It’s well known that colleges care a lot about extracurriculars. If these really are a good sign of flexibility, work ethics, initiative and so on, perhaps we should care about them also. If so, colleges might be correctly adjusting, the low correlations we observe in the charts are just because we can’t directly observe those facts, and high quality people are more concentrated in top schools than this data would suggest.
Additionally, SCOTUS is due to hear Students For Fair Admissions vs Harvard later this year, and Metaculus currently gives them a 75% chance to successfully get racial discrimination in university admissions found unlawful. If so the correlation with SAT/GPA might improve a lot after this year, so the phenomena you’re highlighting might be a relatively short-lived one.
If the SATs are normed to a lower level, you might expect that doing perfectly on them might not be super predictive of outlier ability, compared to just doing extremely well.
Analogously, imagine a test for Alzheimer’s with 100 questions. Even if most people without Alzheimer’s only get <90/100 questions right on them, we might not expect there’s that much extra predictive validity between 99 and 100 right. (see also).