The recent push for productization is making everyone realize that alignment is a capability. A gaslighting chatbot is a bad chatbot compared to a harmless helpful one. As you can see currently, the world is phasing out AI deployment, fixing the bugs, then iterating.
While that’s one way to look at it, another way is to notice the arms race dynamics and how every major tech company is now throwing LLMs into the public head over heels even when they stil have some severe flaws. Another observation is that e.g. OpenAI’s safety efforts are not very popular among end users, given that in their eyes these safety measures make the systems less capable/interesting/useful. People tend to get irritated when their prompt is answered with “As a language model trained by OpenAI, I am not able to <X>”, rather than feeling relief over being saved from a dangerous output.
As for your final paragraph, it is easy to say “<outcome X> is just one ouf of infinite possibilities”, but you’re equating trajectories with outcomes. The existence of infinite possibilities doesn’t really help when there’s a systematic reason that causes many or most of them to have human extinction as an outcome. Whether this is actually the case or not is of course an open and hotly debated question, but just claiming “it’s just a single point on the x axis so the probability mass must be 0″ is surely not how you get closer to an actual answer.
why the overemphasis/obsession on doom scenario?
Because it is extremely important that we do what we can to avoid such a scenario. I’m glad that e.g. airlines still invest a lot in improving flight safety and preventing accidents even though flying is already the safest way of traveling. Humanity is basically at this very moment boarding a giant AI-rplane that is about to take off for the very first time, and I’m rather happy there’s a number of people out there looking at the possible worst case and doing their best to figure out how we can get this plane safely off the ground rather than saying “why are people so obsessed with the doom scenario? A plane crash is just one out of infinite possibilities, we’re gonna be fine!”.
While that’s one way to look at it, another way is to notice the arms race dynamics and how every major tech company is now throwing LLMs into the public head over heels even when they stil have some severe flaws. Another observation is that e.g. OpenAI’s safety efforts are not very popular among end users, given that in their eyes these safety measures make the systems less capable/interesting/useful. People tend to get irritated when their prompt is answered with “As a language model trained by OpenAI, I am not able to <X>”, rather than feeling relief over being saved from a dangerous output.
As for your final paragraph, it is easy to say “<outcome X> is just one ouf of infinite possibilities”, but you’re equating trajectories with outcomes. The existence of infinite possibilities doesn’t really help when there’s a systematic reason that causes many or most of them to have human extinction as an outcome. Whether this is actually the case or not is of course an open and hotly debated question, but just claiming “it’s just a single point on the x axis so the probability mass must be 0″ is surely not how you get closer to an actual answer.
Because it is extremely important that we do what we can to avoid such a scenario. I’m glad that e.g. airlines still invest a lot in improving flight safety and preventing accidents even though flying is already the safest way of traveling. Humanity is basically at this very moment boarding a giant AI-rplane that is about to take off for the very first time, and I’m rather happy there’s a number of people out there looking at the possible worst case and doing their best to figure out how we can get this plane safely off the ground rather than saying “why are people so obsessed with the doom scenario? A plane crash is just one out of infinite possibilities, we’re gonna be fine!”.