I haven’t seen any coverage of the double structure or Anthropic exit which suggests that Amodei helped think up or write the double structure. Certainly, the language they use around the Anthropic public benefit corporation indicates they all think, at least post-exit, that the OA double structure was a terrible idea (eg. see the end of this article).
You don’t know that. They seem to have often had near majorities, rather than being a token 1 or 2 board members.
By most standards, Karnofsky and Sutskever are ‘doomers’, and Zillis is likely a ‘doomer’ too as that is the whole premise of Neuralink and she was a Musk representative (which is why she was pushed out after Musk turned on OA publicly and began active hostilities like breaking contracts with OA). Hoffman’s views are hard to characterize, but he doesn’t seem to clearly come down as an anti-doomer or to be an Altman loyalist. (Which would be a good enough reason for Altman to push him out; and for a charismatic leader, neutralizing a co-founder is always useful, for the same reason no one would sell life insurance to an Old Bolshevik in Stalinist Russia.)
If I look at the best timeline of the board composition I’ve seen thus far, at a number of times post-2018, it looks like there was a ‘near majority’ or even outright majority. For example, 2020-12-31 has either a tie or an outright majority for either side depending on how you assume Sutskever & Hoffman (Sutskever?/Zilis/Karnofsky/D’Angelo/McCauley vs Hoffman? vs Altman/Brockman), and with the 2021-12-31 list the Altman faction needs to pick up every possible vote to match the existing 5 ‘EA’ faction (Zilis/Karnofsky/D’Angelo/McCauley/Toner vs Hurd?/Sutskever?/Hoffman? vs Brockman/Altman) although this has to be wrong because the board maxes out at 7 according to the bylaws so it’s unclear how exactly the plausible majorities evolved over time.
I am reasonably confident Helen replaced Holden as a board member, so I don’t think your 2021-12-31 list is accurate. Maybe there was a very short period where they were both on the board, but I heard the intention was for Helen to replace Holden.
The main thing that I doubt is that Sam knew at the time that he was gifting the board to doomers. Ilya was a loyalist and non-doomer when appointed. Elon was I guess some mix of doomer and loyalist at the start. Given how AIS worries generally increased in SV circles over time, more likely than not some of D’Angelo, Hoffman, and Hurd moved toward the “doomer” pole over time.
Ilya has always been a doomer AFAICT, he was just loyal to Altman personally, who recruited him to OA. (I can tell you that when I spent a few hours chatting with him in… 2017 or something? a very long time ago, anyway—I don’t remember him dismissing the dangers or being pollyannaish.) ‘Superalignment’ didn’t come out of nowhere or surprise anyone about Ilya being in charge. Elon was… not loyal to Altman but appeared content to largely leave oversight of OA to Altman until he had one of his characteristic mood changes, got frustrated and tried to take over. In any case, he surely counts as a doomer by the time Zilis is being added to the board as his proxy. D’Angelo likewise seems to have consistently, in his few public quotes, been concerned about the danger.
A lot of people have indeed moved towards the ‘doomer’ pole but much of that has been timelines: AI doom in 2060 looks and feels a lot different from AI doom in 2027.
Hmm, OK. Back when I met Ilya, about 2018, he was radiating excitement that his next idea would create AGI, and didn’t seem sensitive to safety worries. I also thought it was “common knowledge” that his interest in safety increased substantially between 2018-22, and that’s why I was unsurprised to see him in charge of superalignment.
Re Elon-Zillis, all I’m saying is that it looked to Sam like the seat would belong to someone loyal to him at the time the seat was created.
You may well be right about D’Angelo and the others.
Hm, maybe it was common knowledge in some areas? I just always took him for being concerned. There’s not really any contradiction between being excited about your short-term work and worried about long-term risks. Fooling yourself about your current idea is an important skill for a researcher. (You ever hear the joke about Geoff Hinton? He suddenly solves how the brain works, at long last, and euphorically tells his daughter; she replies: “Oh Dad—not again!”)
Just judging from his Twitter feed, I got the weak impression D’Angelo is somewhat enthusiastic about AI and didn’t catch any concerns about existential safety.
I haven’t seen any coverage of the double structure or Anthropic exit which suggests that Amodei helped think up or write the double structure. Certainly, the language they use around the Anthropic public benefit corporation indicates they all think, at least post-exit, that the OA double structure was a terrible idea (eg. see the end of this article).
You don’t know that. They seem to have often had near majorities, rather than being a token 1 or 2 board members.
By most standards, Karnofsky and Sutskever are ‘doomers’, and Zillis is likely a ‘doomer’ too as that is the whole premise of Neuralink and she was a Musk representative (which is why she was pushed out after Musk turned on OA publicly and began active hostilities like breaking contracts with OA). Hoffman’s views are hard to characterize, but he doesn’t seem to clearly come down as an anti-doomer or to be an Altman loyalist. (Which would be a good enough reason for Altman to push him out; and for a charismatic leader, neutralizing a co-founder is always useful, for the same reason no one would sell life insurance to an Old Bolshevik in Stalinist Russia.)
If I look at the best timeline of the board composition I’ve seen thus far, at a number of times post-2018, it looks like there was a ‘near majority’ or even outright majority. For example, 2020-12-31 has either a tie or an outright majority for either side depending on how you assume Sutskever & Hoffman (Sutskever?/Zilis/Karnofsky/D’Angelo/McCauley vs Hoffman? vs Altman/Brockman), and with the 2021-12-31 list the Altman faction needs to pick up every possible vote to match the existing 5 ‘EA’ faction (Zilis/Karnofsky/D’Angelo/McCauley/Toner vs Hurd?/Sutskever?/Hoffman? vs Brockman/Altman) although this has to be wrong because the board maxes out at 7 according to the bylaws so it’s unclear how exactly the plausible majorities evolved over time.
I am reasonably confident Helen replaced Holden as a board member, so I don’t think your 2021-12-31 list is accurate. Maybe there was a very short period where they were both on the board, but I heard the intention was for Helen to replace Holden.
The main thing that I doubt is that Sam knew at the time that he was gifting the board to doomers. Ilya was a loyalist and non-doomer when appointed. Elon was I guess some mix of doomer and loyalist at the start. Given how AIS worries generally increased in SV circles over time, more likely than not some of D’Angelo, Hoffman, and Hurd moved toward the “doomer” pole over time.
Ilya has always been a doomer AFAICT, he was just loyal to Altman personally, who recruited him to OA. (I can tell you that when I spent a few hours chatting with him in… 2017 or something? a very long time ago, anyway—I don’t remember him dismissing the dangers or being pollyannaish.) ‘Superalignment’ didn’t come out of nowhere or surprise anyone about Ilya being in charge. Elon was… not loyal to Altman but appeared content to largely leave oversight of OA to Altman until he had one of his characteristic mood changes, got frustrated and tried to take over. In any case, he surely counts as a doomer by the time Zilis is being added to the board as his proxy. D’Angelo likewise seems to have consistently, in his few public quotes, been concerned about the danger.
A lot of people have indeed moved towards the ‘doomer’ pole but much of that has been timelines: AI doom in 2060 looks and feels a lot different from AI doom in 2027.
Hmm, OK. Back when I met Ilya, about 2018, he was radiating excitement that his next idea would create AGI, and didn’t seem sensitive to safety worries. I also thought it was “common knowledge” that his interest in safety increased substantially between 2018-22, and that’s why I was unsurprised to see him in charge of superalignment.
Re Elon-Zillis, all I’m saying is that it looked to Sam like the seat would belong to someone loyal to him at the time the seat was created.
You may well be right about D’Angelo and the others.
Hm, maybe it was common knowledge in some areas? I just always took him for being concerned. There’s not really any contradiction between being excited about your short-term work and worried about long-term risks. Fooling yourself about your current idea is an important skill for a researcher. (You ever hear the joke about Geoff Hinton? He suddenly solves how the brain works, at long last, and euphorically tells his daughter; she replies: “Oh Dad—not again!”)
Just judging from his Twitter feed, I got the weak impression D’Angelo is somewhat enthusiastic about AI and didn’t catch any concerns about existential safety.