What is the substantial role EA played in the founding of Deepmind? I find this claim surprising since Deepmind was founded in 2010.
Or is Deepmind not a top three capability company according to you?
Shane Legg started DeepMind together with Demis after he heard about about Superintelligence concerns by Eliezer and Kurzweil, and was active for many months on LessWrong reading Eliezer’s arguments and commenting on the site.
...which predates EA? EAs don’t generally take credit/responsibility for intellectual influences pre-2011, even if those intellectuals then become active in the EA community. If we’re not taking credit for Peter Singer’s work pre-2011, I don’t think we should take responsibility for Eliezer or Kurzweil’s work pre-2011.
Sure, you can slice things here however you want. I do think it’s important to be at least consistent with both positives and negatives here. I’ve definitely seen people claim credit for GiveWell as part of EA, which of course was also founded pre-2011. I don’t think think there is a clear answer on how to handle this, and it seems most important to just be consistent.
IMO it does also feel really weird to say the sentence “we’ve had a substantial influence on 2 out of the 3 top AI capability companies”, when like, the same people also had a substantial influence on the third.
Anthropic: Founded due to xrisk concern, most of the staff are EA, most of the initial funding was EA, EA encouraged people to work there etc
OpenAI: Founded on the back of superintelligence concerns, the FLI Open Letter (the last one!)and a behind closed doors conference organised by EAs. Much of the staff was EAs, and OpenAI was publicised and legitimised by EAs, including capabilities jobs on the job board etc. OpenPhil money got it going (maybe counterfactual would have done anyway)
That’s certainly what OpenPhil says about it. I don’t know enough to comment, beyond saying it clearly still legitimised the company and in general as said the EA energy and ideology definitely counterfactual I would say led to their foundation
I’d say without ideas around AGI doom coming out of EA/Longtermist spheres and being popularised by people in that sphere ( Stuart Russell, Nick Bostrom, FLI etc), openAI probably doesn’t happen. So I’d say makes their foundation over 50% more likely. In this sense ‘led to’ seems pretty adequate
With the AI XRisk stuff it’s hard to say what counts as ‘EA’ vs ‘proto-EA’; you are right though, it’s mostly the pre-EA bostrom-miri- rationalist cluster. I’ve lump them in with EA, maybe you don’t want to
I also think to strengthen it ‘entirely founding Anthropic’ and as well as founding the other companies, continually work to legitimise them and their influence; only very recently did substantial amounts of EA start working against these companies
Have played a substantial role in founding all three of the top leading AI capability companies (cry emoji).
What is the substantial role EA played in the founding of Deepmind? I find this claim surprising since Deepmind was founded in 2010. Or is Deepmind not a top three capability company according to you?
Shane Legg started DeepMind together with Demis after he heard about about Superintelligence concerns by Eliezer and Kurzweil, and was active for many months on LessWrong reading Eliezer’s arguments and commenting on the site.
See this tweet thread: https://twitter.com/ShaneLegg/status/1598047654159978496
And this old LessWrong account from Shane: https://www.lesswrong.com/users/shane_legg
...which predates EA? EAs don’t generally take credit/responsibility for intellectual influences pre-2011, even if those intellectuals then become active in the EA community. If we’re not taking credit for Peter Singer’s work pre-2011, I don’t think we should take responsibility for Eliezer or Kurzweil’s work pre-2011.
Sure, you can slice things here however you want. I do think it’s important to be at least consistent with both positives and negatives here. I’ve definitely seen people claim credit for GiveWell as part of EA, which of course was also founded pre-2011. I don’t think think there is a clear answer on how to handle this, and it seems most important to just be consistent.
IMO it does also feel really weird to say the sentence “we’ve had a substantial influence on 2 out of the 3 top AI capability companies”, when like, the same people also had a substantial influence on the third.
Yeah I think this is a legitimate type of entry to include on the list, though I’d like one for each and an actual description.
Anthropic: Founded due to xrisk concern, most of the staff are EA, most of the initial funding was EA, EA encouraged people to work there etc OpenAI: Founded on the back of superintelligence concerns, the FLI Open Letter (the last one!)and a behind closed doors conference organised by EAs. Much of the staff was EAs, and OpenAI was publicised and legitimised by EAs, including capabilities jobs on the job board etc. OpenPhil money got it going (maybe counterfactual would have done anyway)
I sense that the OpenPhil money was quite a small and easily replaced part of the raise. Is that not your sense?
That’s certainly what OpenPhil says about it. I don’t know enough to comment, beyond saying it clearly still legitimised the company and in general as said the EA energy and ideology definitely counterfactual I would say led to their foundation
Made their foundation maybe 5% more likely, perhaps. Led to it seems way too strong, right?
I’d say without ideas around AGI doom coming out of EA/Longtermist spheres and being popularised by people in that sphere ( Stuart Russell, Nick Bostrom, FLI etc), openAI probably doesn’t happen. So I’d say makes their foundation over 50% more likely. In this sense ‘led to’ seems pretty adequate
Isn’t Bostrom publishing that stuff before EA? Though would we take responsibility if it were positive?
With the AI XRisk stuff it’s hard to say what counts as ‘EA’ vs ‘proto-EA’; you are right though, it’s mostly the pre-EA bostrom-miri- rationalist cluster. I’ve lump them in with EA, maybe you don’t want to
I guess if we would lump if positive we should lump if negative
Has anyone written a more detailed account of this?
I also think to strengthen it ‘entirely founding Anthropic’ and as well as founding the other companies, continually work to legitimise them and their influence; only very recently did substantial amounts of EA start working against these companies