Catherine here, I work for Open Phil on the technical AI program area. I’m not going to comment fully on our entire case for the Open Phil AI Fellows program, but I want to just address some things that seem wrong to me here:
“early-career AI safety researchers”
The OpenPhil AI PhD Fellows are mostly not early-career “AI safety” researchers. (see the fellowship description here)
The pool of AI safety-oriented PhD students across the world is a stronger cohort in total than any of these particular groups (because it includes them), and not much weaker on average.
I don’t think this would be true, even if the “it includes them” claim were true. I think you need much more evidence to justify a claim that “a larger set containing X is not much weaker on average than the set X itself”.
there are more students from top schools moving into AI safety than econ, philosophy, and GCBRs
? I think you’re claiming there are more grad-school-bound undergrads-from-top-schools, total, aspiring to be “AI safety researchers” than to be economists? This seems definitely false to me. Am I misunderstanding?
I think you need much more evidence to justify a claim that “a larger set containing X is not much weaker on average than the set X itself”.
If OpenPhil’s fellow are not expected to do research on AI safety then apparently the justification for funding is quite different, so let’s put them to one side.
The CS DPhil scholars at Oxford seem similar to EA CS PhDs at Toronto, ANU, and other rank 10-30 schools.
The RSP students are also seem similar, with broader interests but less credentials.
Paul’s grantees seem more aligned though less qualified and supervised, though there are only three.
Overall, rank 10-30 AI safety PhD students seems comparable to these three latter groups, and clearly not much weaker.
? I think you’re claiming there are more grad-school-bound undergrads-from-top-schools, total, aspiring to be “AI safety researchers” than to be economists? This seems definitely false to me. Am I misunderstanding?
Edited to clarify that this means researchers on longtermist econ issues.
But I am interested to know if this argument is wrong in any other respect!
Catherine here, I work for Open Phil on the technical AI program area. I’m not going to comment fully on our entire case for the Open Phil AI Fellows program, but I want to just address some things that seem wrong to me here:
The OpenPhil AI PhD Fellows are mostly not early-career “AI safety” researchers. (see the fellowship description here)
I don’t think this would be true, even if the “it includes them” claim were true. I think you need much more evidence to justify a claim that “a larger set containing X is not much weaker on average than the set X itself”.
? I think you’re claiming there are more grad-school-bound undergrads-from-top-schools, total, aspiring to be “AI safety researchers” than to be economists? This seems definitely false to me. Am I misunderstanding?
If OpenPhil’s fellow are not expected to do research on AI safety then apparently the justification for funding is quite different, so let’s put them to one side.
The CS DPhil scholars at Oxford seem similar to EA CS PhDs at Toronto, ANU, and other rank 10-30 schools.
The RSP students are also seem similar, with broader interests but less credentials.
Paul’s grantees seem more aligned though less qualified and supervised, though there are only three.
Overall, rank 10-30 AI safety PhD students seems comparable to these three latter groups, and clearly not much weaker.
Edited to clarify that this means researchers on longtermist econ issues.
But I am interested to know if this argument is wrong in any other respect!