[Question] How much EA analysis of AI safety as a cause area exists?

AI safety has be­come a big deal in EA, and so I’m cu­ri­ous about how much “due dili­gence” on it has been done by the EA com­mu­nity as a whole. Ob­vi­ously there have been many in-per­son dis­cus­sions, but it’s very difficult to eval­u­ate whether these con­tain new or high-qual­ity con­tent. Prob­a­bly a bet­ter met­ric is how much work has been done which:

1. Is pub­li­cly available;

2. En­gages in de­tail with core ar­gu­ments for why AI might be dan­ger­ous (type A), OR tries to eval­u­ate the cred­i­bil­ity of the ar­gu­ments with­out di­rectly en­gag­ing with them (type B);

3. Was mo­ti­vated or in­sti­gated by EA.

I’m wary of fo­cus­ing too much on credit as­sign­ment, but it seems im­por­tant to be able to an­swer a ques­tion like “if EA hadn’t ever formed, to what ex­tent would it have been harder for an im­par­tial ob­server in 2019 to eval­u­ate whether work­ing on AI safety is im­por­tant?” The clear­est ev­i­dence would be if there were much rele­vant work pro­duced by peo­ple who were em­ployed at EA orgs, funded by EA grants, or con­vinced to work on AI safety through their in­volve­ment with EA. Some such work comes to mind, and I’ve listed it be­low; what am I miss­ing?

Type A work which meets my crite­ria above:

Type A work which only par­tially meets crite­rion 3 (or which I’m un­cer­tain about):

Type B work which meets my crite­ria above:

Things which don’t meet those crite­ria:

  • This 80,000 hours re­port (which men­tions the ar­gu­ments, but doesn’t thor­oughly eval­u­ate them)

  • Superintelligence

  • The AI Foom debate

Edited to add: Wei Dai asked why I didn’t count Nick Bostrom as “part of EA”, and I wrote quite a long an­swer which ex­plains the mo­ti­va­tions be­hind this ques­tion much bet­ter than my origi­nal post. So I’ve copied most of it be­low:

The three ques­tions I am ul­ti­mately try­ing to an­swer are: a) how valuable is it to build up the EA move­ment? b) how much should I up­date when I learn that a given be­lief is a con­sen­sus in EA? and c) how much ev­i­dence do the opinions of other peo­ple provide in favour of AI safety be­ing im­por­tant?

To an­swer the first ques­tion, as­sum­ing that anal­y­sis of AI safety as a cause area is valuable, I should fo­cus on con­tri­bu­tions by peo­ple who, as I said, were “mo­ti­vated or in­sti­gated by EA”. Here Nick doesn’t count (ex­cept in­so­far as EA made his book come out sooner or bet­ter).

To an­swer the sec­ond ques­tion, it helps to know whether the fo­cus on AI safety in EA came about be­cause many peo­ple did com­pre­hen­sive due dili­gence and shared their find­ings, or whether there wasn’t much in­ves­ti­ga­tion and the ubiquity of the be­lief was driven via an in­for­ma­tion cas­cade. For this pur­pose, I should count work by peo­ple to the ex­tent that they or peo­ple like them are likely to crit­i­cally in­ves­ti­gate other be­liefs that are or will be­come wide­spread in EA. Be­ing mo­ti­vated to in­ves­ti­gate AI safety by EA is the best ev­i­dence, but for the pur­pose of an­swer­ing this ques­tion I prob­a­bly should have used “mo­ti­vated by EA or mo­ti­vated by very similar things to what EAs are mo­ti­vated by”, and should par­tially count Nick.

To an­swer the third ques­tion, it helps to know whether the peo­ple who have be­come con­vinced that AI safety is im­por­tant are a rel­a­tively ho­moge­nous group who might all have highly cor­re­lated bi­ases and hid­den mo­ti­va­tions, or whether a wide range of peo­ple have be­come con­vinced. For this pur­pose, I should count work by peo­ple to the ex­tent that they are dis­similar to the tran­shu­man­ists and ra­tio­nal­ists who came up with the origi­nal safety ar­gu­ments, and also to the ex­tent that they red­erived the ar­gu­ments for them­selves rather than be­ing in­fluenced by the ex­ist­ing ar­gu­ments. Here EAs who started off not be­ing in­clined to­wards tran­shu­man­ism or ra­tio­nal­ism at all count the most, and Nick counts very lit­tle.