One minor quibble with this post’s language, rather than any of its actual claims: The title includes the phrase “safety by default”, and the terms “optimism” and “optimist” are repeatedly applied to these researchers or their views. The title is reasonable in a sense, as these interviews were partially/mostly about whether AI would be “safe by default”, or why we might believe that it would be, or why these researchers believe that that’s likely. And the use of “optimism”/”optimist” are reasonable in a sense, as these researchers were discussing why they’re relatively optimistic, compared to something like e.g. the “typical MIRI view”.
But it seems potentially misleading to use those phrases here without emphasising (or at least mentioning) that at least some of these researchers think there’s a greater than 1% chance of extinction or other existential catastrophe as a result of AI. E.g., the statement “Rohin reported an unusually large (90%) chance that AI systems will be safe without additional intervention” implies a 10% credence that that won’t be the case (and Paul and Adam seem to share very roughly similar views, based on Rohin’s summaries). Relevant quote from The Precipice:
In 1939, Enrico Fermi told Szilard the chain reaction was but a ‘remote possibility’ [...]
Fermi was asked to clarify the ‘remote possibility’ and ventured ‘ten percent’. Isidor Rabi, who was also present, replied, ‘Ten percent is not a remote possibility if it means that we may die of it. If I have pneumonia and the doctor tells me that there is a remote possibility that I might die, and it’s ten percent, I get excited about it’
And in this case, the stakes are far greater (meaning no offence to Isidor Rabi).
My guess would be that a decent portion of people who (a) were more used to something like the FHI/80k/Oxford views, and less used to the MIRI/Bay Area views, and (b) read this without having read the interviews in great detail, might think that these researchers believe something like “The chance things go wrong is too small to be worth anyone else worrying about.” Which doesn’t seem accurate, at least for Rohin, Paul, and Adam.
To be clear: I don’t think you’re intending to convey that message. And I definitely wouldn’t want to try shut down any statements about AI that don’t sound like “this is a huge deal, everyone get in here now!” I’m just a bit concerned about posts accidentally conveying an overly optimistic/sanguine message when that wasn’t actually their intent, and when it wasn’t supported by the arguments/evidence provided.
(Something informing this comment is my past experience reading a bunch of cognitive science work on how misinformation spreads and can be sticky. Some discussion here, and a particularly relevant paper here.)
One minor quibble with this post’s language, rather than any of its actual claims: The title includes the phrase “safety by default”, and the terms “optimism” and “optimist” are repeatedly applied to these researchers or their views. The title is reasonable in a sense, as these interviews were partially/mostly about whether AI would be “safe by default”, or why we might believe that it would be, or why these researchers believe that that’s likely. And the use of “optimism”/”optimist” are reasonable in a sense, as these researchers were discussing why they’re relatively optimistic, compared to something like e.g. the “typical MIRI view”.
But it seems potentially misleading to use those phrases here without emphasising (or at least mentioning) that at least some of these researchers think there’s a greater than 1% chance of extinction or other existential catastrophe as a result of AI. E.g., the statement “Rohin reported an unusually large (90%) chance that AI systems will be safe without additional intervention” implies a 10% credence that that won’t be the case (and Paul and Adam seem to share very roughly similar views, based on Rohin’s summaries). Relevant quote from The Precipice:
And in this case, the stakes are far greater (meaning no offence to Isidor Rabi).
My guess would be that a decent portion of people who (a) were more used to something like the FHI/80k/Oxford views, and less used to the MIRI/Bay Area views, and (b) read this without having read the interviews in great detail, might think that these researchers believe something like “The chance things go wrong is too small to be worth anyone else worrying about.” Which doesn’t seem accurate, at least for Rohin, Paul, and Adam.
To be clear: I don’t think you’re intending to convey that message. And I definitely wouldn’t want to try shut down any statements about AI that don’t sound like “this is a huge deal, everyone get in here now!” I’m just a bit concerned about posts accidentally conveying an overly optimistic/sanguine message when that wasn’t actually their intent, and when it wasn’t supported by the arguments/evidence provided.
(Something informing this comment is my past experience reading a bunch of cognitive science work on how misinformation spreads and can be sticky. Some discussion here, and a particularly relevant paper here.)