People seem surprised and bewildered when AI folks defect away from AI safety towards capabilities. People trust that as AI companies grow, those gaining power and money from shares will not be adversely influenced by that power and money.
fwiw I donât actually know many examples of this, and the ones I hear cited often seem uncompelling to me. E.g.:
Mechanizeâs founders donât seem like EAs who got corrupted by AI money but rather EAs with unusual moral and empirical views which result in them thinking that the best course of action is the exact opposite of what most EAs think
Thanks! I only know a handful of people in this category, but for what itâs worth, it again feels like people who were predisposed to thinking that working on pretraining would be okay rather than them being âcorrupted.â
E.g., I recently talked to someone who told me that their main takeaway from a safety fellowship was realizing that they didnât fit in because they actually werenât worried about existential risk in the same way that the other attendees were.
Hmm, I think if smart EA/âRat types get âcorruptedâ in general, theyâll present as thoughtful people with reasons that are hard to dismiss quickly when questioned by EAs. I get the vague sense that your evidence bar for âcorruptionâ is going to be too high to be useful in most worlds where thereâs a lot of corruption.
(thatâs not to say that EAs/âRats/âetc. who join labs/âstart wildly profitable companies speeding up AI progress have been âcorruptedââI just think if they were, it would present pretty similarly to how it has done and itâs hard to get lots of easy to share evidence)
fwiw I donât actually know many examples of this, and the ones I hear cited often seem uncompelling to me. E.g.:
Greg Brockman doesnât seem like a true believer in OpenAIâs nonprofit mission who got corrupted but rather someone who went into it wanting to make a profit
Mechanizeâs founders donât seem like EAs who got corrupted by AI money but rather EAs with unusual moral and empirical views which result in them thinking that the best course of action is the exact opposite of what most EAs think
(Counterexamples appreciated, though!)
I think he would include a lot of people who work at Anthropic, for example, on pre-training, some of whom went through MATS or something.
Thanks! I only know a handful of people in this category, but for what itâs worth, it again feels like people who were predisposed to thinking that working on pretraining would be okay rather than them being âcorrupted.â
E.g., I recently talked to someone who told me that their main takeaway from a safety fellowship was realizing that they didnât fit in because they actually werenât worried about existential risk in the same way that the other attendees were.
Hmm, I think if smart EA/âRat types get âcorruptedâ in general, theyâll present as thoughtful people with reasons that are hard to dismiss quickly when questioned by EAs. I get the vague sense that your evidence bar for âcorruptionâ is going to be too high to be useful in most worlds where thereâs a lot of corruption.
(thatâs not to say that EAs/âRats/âetc. who join labs/âstart wildly profitable companies speeding up AI progress have been âcorruptedââI just think if they were, it would present pretty similarly to how it has done and itâs hard to get lots of easy to share evidence)