Consequently, it’s possible to be skeptical of the motivations anyone in AI safety, expert or novice, on the grounds that “isn’t it convenient the best way to save the world is to do cool AI stuff?”
Fair point overall, and I’ll edit in a link to this comment in the post. It would be interesting to see data on what percentage of people working AI safety due to EA motivations would likely be working in AI regardless of impact. I’d predict that it’s significant but not a large majority (say, 80% CI of 25-65%).
A few reactions to specific points/claims:
It’s possible that they know the same amount about AI X-risk mitigation, and would perhaps have similar success rate working on some alignment research (which to a great deal involves GPT-3 prompt hacking with near-0 maths).
My understanding is that most alignment research involves either maths or skills similar to ML research/engineering; there is some ~GPT-3 prompt hacking (e.g. this post?) but it seems like <10% of the field?
Imagine that two groups wanted to organise an AI camp or event: a group of AI novice undergrads who have been engaged in EA vs a group of AI profs with no EA connections. Who is more likely to get funding?
I’m not sure about specifically organizing an event, but I’d guess that experienced AI profs with no EA connections but who seemed genuinely interested in reducing AI x-risk would be able to get substantial funding/support for their research.
EA-funded AI safety is actually a pretty sweet deal for an AI novice who gets to do something that’s cool at very little cost.
The field has probably gotten easier to break into over time but I’d guess most people attempting to enter still experience substantial costs, such as rejections and mental health struggles.
Fair point overall, and I’ll edit in a link to this comment in the post. It would be interesting to see data on what percentage of people working AI safety due to EA motivations would likely be working in AI regardless of impact. I’d predict that it’s significant but not a large majority (say, 80% CI of 25-65%).
A few reactions to specific points/claims:
My understanding is that most alignment research involves either maths or skills similar to ML research/engineering; there is some ~GPT-3 prompt hacking (e.g. this post?) but it seems like <10% of the field?
I’m not sure about specifically organizing an event, but I’d guess that experienced AI profs with no EA connections but who seemed genuinely interested in reducing AI x-risk would be able to get substantial funding/support for their research.
The field has probably gotten easier to break into over time but I’d guess most people attempting to enter still experience substantial costs, such as rejections and mental health struggles.