AI systems that match or exceed human intelligence could very plausibly arrive within the next decade, and raise some significant challenges as they do. Alongside the well-known issues surrounding AI safety, there are many other potential problems that we don’t yet seem prepared for, but that could affect society on a huge scale. Forethought has published lots of research on this recently, and we wanted to cover some of the key points for people who might be interested in exploring some of these challenges with their career.
A few of the issues that seem particularly important:
How a sharp acceleration in technological progress could leave us unprepared for new technologies, including potentially dangerous ones like engineered pathogens.
How growing reliance on AI for information could reshape how individuals and institutions form beliefs, and the risks this might bring.
The prospect of power and value lock-in, where AI provides the means for institutions and groups to entrench power dynamics and values.
Questions around AI sentience and welfare, and why now may be a particularly important window for thinking about them.
Ideas for how people can engage with these problems through their career, along with relevant organizations in the (broad) space.
Here’s an excerpt from our section on the risks of accelerated technological progress:
We can think of technological development as akin to pulling balls from an urn. As we develop each new piece of technology, it’s like we’re reaching into the urn hoping to pull out balls that will help humanity. Many of these are straightforwardly beneficial, like vaccines and other medical technologies. But some balls can be highly dangerous, like nuclear technology. The trouble is, we have a limited ability to know whether technology will harm us or help us until we’ve developed it.
The scientific speedup that advanced AI may bring is like tipping this urn upside down, spilling thousands of balls onto the ground at once. If this happens, we’ll have little time to evaluate and prepare for each new technology before the next arrives, leaving us able to do little more than hope that one of these destructive technologies isn’t revealed.
You can read the full article here, and explore the many other cause areas we’ve covered here.
AI systems that match or exceed human intelligence could very plausibly arrive within the next decade, and raise some significant challenges as they do. Alongside the well-known issues surrounding AI safety, there are many other potential problems that we don’t yet seem prepared for, but that could affect society on a huge scale. Forethought has published lots of research on this recently, and we wanted to cover some of the key points for people who might be interested in exploring some of these challenges with their career.
A few of the issues that seem particularly important:
How a sharp acceleration in technological progress could leave us unprepared for new technologies, including potentially dangerous ones like engineered pathogens.
How growing reliance on AI for information could reshape how individuals and institutions form beliefs, and the risks this might bring.
The prospect of power and value lock-in, where AI provides the means for institutions and groups to entrench power dynamics and values.
Questions around AI sentience and welfare, and why now may be a particularly important window for thinking about them.
Ideas for how people can engage with these problems through their career, along with relevant organizations in the (broad) space.
Here’s an excerpt from our section on the risks of accelerated technological progress:
We can think of technological development as akin to pulling balls from an urn. As we develop each new piece of technology, it’s like we’re reaching into the urn hoping to pull out balls that will help humanity. Many of these are straightforwardly beneficial, like vaccines and other medical technologies. But some balls can be highly dangerous, like nuclear technology. The trouble is, we have a limited ability to know whether technology will harm us or help us until we’ve developed it.
The scientific speedup that advanced AI may bring is like tipping this urn upside down, spilling thousands of balls onto the ground at once. If this happens, we’ll have little time to evaluate and prepare for each new technology before the next arrives, leaving us able to do little more than hope that one of these destructive technologies isn’t revealed.
You can read the full article here, and explore the many other cause areas we’ve covered here.