Identifying highly capable individuals is indeed hard, but I don’t think this is any more of a problem in AI safety research than in other fields.
Quite. I think that my model of Eli was setting the highest standard possible—not merely a good researcher, but a great one, the sort of person who can bring whole new paradigms/subfields into existence (Kahneman & Tversky, Von Neumann, Shannon, Einstein, etc), and then noting that because the tails come apart (aka regressional goodharting), optimising for the normal metrics used in standard hiring practices won’t get you these researchers (I realise that probably wasn’t true for Von Neumann, but I think it was true for all the others).
Quite. I think that my model of Eli was setting the highest standard possible—not merely a good researcher, but a great one, the sort of person who can bring whole new paradigms/subfields into existence (Kahneman & Tversky, Von Neumann, Shannon, Einstein, etc), and then noting that because the tails come apart (aka regressional goodharting), optimising for the normal metrics used in standard hiring practices won’t get you these researchers (I realise that probably wasn’t true for Von Neumann, but I think it was true for all the others).