One thing I am cautiously optimistic about (at least as regards long term outcomes) is that I think ‘a few high-profile sub-existential-catastrophe events’ are fairly likely. In particular I think that we will soon have AIs capable of impersonating real humans, both online and on the phone via speech synthesis.
These will be superhuman, or maybe just at the level of an expert human, in terms of things like ‘writing a provocative tweet’ or ‘selling you insurance’ or ‘handling call center tasks’. Or, once the technology is out in the open, ‘scamming your grandma’, ‘convincing someone to pay a bitcoin ransom’, and so on. At that point such AIs seem likely to still be short of being able to generalize to the point of escaping confinement, or being trained to the point where emergent motives would cause them to try to do so. But they would likely be ubiquitous enough that they would attract broad public notice and, quite likely, cause considerable fear. We might not have enough attention directed towards AI safety yet, but I think public consciousness will increase dramatically before all the pieces that would make hard takeoff possible are in place.
One thing I am cautiously optimistic about (at least as regards long term outcomes) is that I think ‘a few high-profile sub-existential-catastrophe events’ are fairly likely. In particular I think that we will soon have AIs capable of impersonating real humans, both online and on the phone via speech synthesis.
These will be superhuman, or maybe just at the level of an expert human, in terms of things like ‘writing a provocative tweet’ or ‘selling you insurance’ or ‘handling call center tasks’. Or, once the technology is out in the open, ‘scamming your grandma’, ‘convincing someone to pay a bitcoin ransom’, and so on. At that point such AIs seem likely to still be short of being able to generalize to the point of escaping confinement, or being trained to the point where emergent motives would cause them to try to do so. But they would likely be ubiquitous enough that they would attract broad public notice and, quite likely, cause considerable fear. We might not have enough attention directed towards AI safety yet, but I think public consciousness will increase dramatically before all the pieces that would make hard takeoff possible are in place.