This still feels wrong to me: if they’re so smart, where are the nobel laureates? The famous physicists?
I think expecting nobel laureates is a bit much, especially given the demographics (these people are relatively young). But if you’re looking for people who are publicly-legible intellectual powerhouses, I think you can find a reasonable number:
Many historical MIRI researchers (random examples: Scott Garrabrant, Abram Demski, Jessica Taylor)
Paul Christiano (also formerly did research at MIRI)
(Many more not listed, including non-central examples like Robin Hanson, Vitalik Buterin, Shane Legg, and Yoshua Bengio[2].)
And, like, idk, man. 130 is pretty smart but not “famous for their public intellectual output” level smart. There are a bunch of STEM PhDs, a bunch of software engineers, some successful entrepreneurs, and about the number of “really very smart” people you’d expect in a community of this size.
Who sure is working on AI x-risk and collaborating with much more central rats/EAs, but only came into it relatively recently, which is both evidence in favor of one of the core claims of the post but also evidence against what I read as the broader vibes.
This list includes many people that, despite being well-known as very smart people in the EA/rationalist community, are not really “publicly-legible intellectual powerhouses.”
Taylor, Garrabrant and Demski publish articles on topics related to AI primarily, but as far as I could tell, none of them have NeurIPS or ICML papers. In comparison, the number of unique authors with papers featured on NeurIPS is more than 10,000 each year. Taylor and Demski have fewer than 4,000 Twitter followers, and Garrabrant does not seem to use Twitter. The three researchers have a few hundred academic citations each.
I find it hard to justify describing them as “publicly-legible intellectual powerhouses” when they seem, at a glance, not much more professionally impressive than great graduate students at top math or CS graduate programs (though such students are very impressive).
A better example of an EA-adjacent publicly-legible intellectual powerhouse in the ML area is Dan Hendrycks, who had a NeurIPS paper published in college and has at least 60 times more citations than those MIRI or former MIRI researchers you mentioned.
My claim is something closer to “experts in the field will correctly recognize them as obviously much smarter than +2 SD”, rather than “they have impressive credentials” (which is missing the critically important part where the person is actually much smarter than +2 SD).
I don’t think reputation has anything to do with titotal’s original claim and wasn’t trying to make any arguments in that direction.
Also… putting that aside, that is one bullet point from my list, and everyone else except Qiaochu has a wikipedia entry, which is not a criteria I was tracking when I wrote the list but think decisively refutes the claim that the list includes many people who are not publicly-legible intellectual powerhouses. (And, sure, I could list Dan Hendryks. I could probably come up with another twenty such names, even though I think they’d be worse at supporting the point I was trying to make.)
I strongly endorse the overall vibe/message of titotal’s post here, but I’d add, as a philosopher, that EA philosophers are also a fairly professionally impressive bunch.
Peter Singer is a leading academic ethicist by any standards. The GPI in Oxford’s broadly EA-aligned work is regularly published in leading journals. I think it is fair to say Derek Parfit was broadly aligned with EA, and a key influence on the actually EA philosophers, and many philosophers would tell you he was a genuinely great philosopher. Many of the most controversial EA ideas like longtermism have roots in his work. Longtermism is less like a view believed only by a few marginalised scientists, and more like say, a controversial new interpretation of quantum mechanics that most physicists reject, but some young people at top departments like and which you can publish work defending in leading journals.
And, like, idk, man. 130 is pretty smart but not “famous for their public intellectual output” level smart.
Yeah “2 sds just isn’t that big a deal” seems like an obvious hypothesis here (“People might over-estimate how smart they are” is, of course, another likely hypothesis).
Also of course OP was being overly generous by assuming that it’s a normal distribution centered around 128. If you take a bunch of random samples of a normal distribution, and only look at subsamples with median 2 sds out, in approximately ~0 subsamples will you find it equally likely to see + 0 sds and +4 sds.
If you take a bunch of random samples of a normal distribution, and only look at subsamples with median 2 sds out, in approximately ~0 subsamples will you find it equally likely to see + 0 sds and +4 sds.
Wait, are you claiming +0 SD is significantly more likely than +4 SD in a subsample with median +2 SD, or are you claiming that +4 SD is more likely than +0 SD? And what makes you think so?
The former. I think it should be fairly intuitive if you think about the shape of the distribution you’re drawing from. Here’s the code, courtesy of Claude 3.5. [edit: deleted the quote block with the code because of aesthetics, link should still work].
I think expecting nobel laureates is a bit much, especially given the demographics (these people are relatively young). But if you’re looking for people who are publicly-legible intellectual powerhouses, I think you can find a reasonable number:
Wei Dai
Hal Finney (RIP)
Scott Aaronson
Qiaochu Yuan[1]
Many historical MIRI researchers (random examples: Scott Garrabrant, Abram Demski, Jessica Taylor)
Paul Christiano (also formerly did research at MIRI)
(Many more not listed, including non-central examples like Robin Hanson, Vitalik Buterin, Shane Legg, and Yoshua Bengio[2].)
And, like, idk, man. 130 is pretty smart but not “famous for their public intellectual output” level smart. There are a bunch of STEM PhDs, a bunch of software engineers, some successful entrepreneurs, and about the number of “really very smart” people you’d expect in a community of this size.
He might disclaim any current affiliation, but for this purpose I think he obviously counts.
Who sure is working on AI x-risk and collaborating with much more central rats/EAs, but only came into it relatively recently, which is both evidence in favor of one of the core claims of the post but also evidence against what I read as the broader vibes.
This list includes many people that, despite being well-known as very smart people in the EA/rationalist community, are not really “publicly-legible intellectual powerhouses.”
Taylor, Garrabrant and Demski publish articles on topics related to AI primarily, but as far as I could tell, none of them have NeurIPS or ICML papers. In comparison, the number of unique authors with papers featured on NeurIPS is more than 10,000 each year. Taylor and Demski have fewer than 4,000 Twitter followers, and Garrabrant does not seem to use Twitter. The three researchers have a few hundred academic citations each.
I find it hard to justify describing them as “publicly-legible intellectual powerhouses” when they seem, at a glance, not much more professionally impressive than great graduate students at top math or CS graduate programs (though such students are very impressive).
A better example of an EA-adjacent publicly-legible intellectual powerhouse in the ML area is Dan Hendrycks, who had a NeurIPS paper published in college and has at least 60 times more citations than those MIRI or former MIRI researchers you mentioned.
My claim is something closer to “experts in the field will correctly recognize them as obviously much smarter than +2 SD”, rather than “they have impressive credentials” (which is missing the critically important part where the person is actually much smarter than +2 SD).
I don’t think reputation has anything to do with titotal’s original claim and wasn’t trying to make any arguments in that direction.
Also… putting that aside, that is one bullet point from my list, and everyone else except Qiaochu has a wikipedia entry, which is not a criteria I was tracking when I wrote the list but think decisively refutes the claim that the list includes many people who are not publicly-legible intellectual powerhouses. (And, sure, I could list Dan Hendryks. I could probably come up with another twenty such names, even though I think they’d be worse at supporting the point I was trying to make.)
I strongly endorse the overall vibe/message of titotal’s post here, but I’d add, as a philosopher, that EA philosophers are also a fairly professionally impressive bunch.
Peter Singer is a leading academic ethicist by any standards. The GPI in Oxford’s broadly EA-aligned work is regularly published in leading journals. I think it is fair to say Derek Parfit was broadly aligned with EA, and a key influence on the actually EA philosophers, and many philosophers would tell you he was a genuinely great philosopher. Many of the most controversial EA ideas like longtermism have roots in his work. Longtermism is less like a view believed only by a few marginalised scientists, and more like say, a controversial new interpretation of quantum mechanics that most physicists reject, but some young people at top departments like and which you can publish work defending in leading journals.
Yeah “2 sds just isn’t that big a deal” seems like an obvious hypothesis here (“People might over-estimate how smart they are” is, of course, another likely hypothesis).
Also of course OP was being overly generous by assuming that it’s a normal distribution centered around 128. If you take a bunch of random samples of a normal distribution, and only look at subsamples with median 2 sds out, in approximately ~0 subsamples will you find it equally likely to see + 0 sds and +4 sds.
Wait, are you claiming +0 SD is significantly more likely than +4 SD in a subsample with median +2 SD, or are you claiming that +4 SD is more likely than +0 SD? And what makes you think so?
The former. I think it should be fairly intuitive if you think about the shape of the distribution you’re drawing from. Here’s the code, courtesy of Claude 3.5. [edit: deleted the quote block with the code because of aesthetics, link should still work].