We have published a preprint of a paper describing the concept of differential technology development and exploring when and how its implementation may be viable. This manuscript has been in the works at FHI for a while, so we would love your feedback!
Authors: Jonas B. Sandbrink, Hamish Hobbs, Jacob L. Swett, Allan Dafoe, Anders Sandberg
Further details
Nick Bostrom articulated the concept of Differential Technological Development in Superintelligence. This concept, and the concept of differential progress more broadly, has already been fairly widely discussed and written about, but within a limited community. This includes here on the EA Forum. The concept is typically used to argue that it would be beneficial to accelerate risk-reducing technological progress and retard risk-increasing technological progress, to manage potentially catastrophic or existential technological risks such as those from advanced AI systems and biotechnologies.
With this paper, we attempt to thoroughly explore the concept, make it accessible to the broader research and policy community, and explore the contexts in which its implementation is likely to be viable. To our knowledge, this is the first academic article seeking to comprehensively articulate a principle of differential technology development.
In the paper we argue that:
Responsible innovation efforts to date have largely focussed on shaping individual technologies.
However, as demonstrated by the preferential advancement of low-emission technologies, certain technologies reduce risks from other technologies or constitute low-risk substitutes.
Governments and other relevant actors may leverage risk-reducing interactions across technology portfolios to mitigate risks beyond climate change.
We adapt Nick Bostrom’s original concept to propose a responsible innovation principle of “differential technology development”, which calls relevant actors to leverage risk-reducing interactions between technologies by affecting their relative timing.
Thus, it may be beneficial to delay risk-increasing technologies and preferentially advance risk-reducing defensive, safety, or substitute technologies.
Implementing differential technology development requires the ability to anticipate or identify impacts and intervene in the relative timing of technologies.
We find that both are sometimes viable and that differential technology development may still be usefully applied even late in the diffusion of some harmful technologies.
A principle of differential technology development may inform government research funding priorities and technology regulation, as well as philanthropic research and development funders and corporate social responsibility measures.
Differential technology development may be particularly promising to mitigate potential catastrophic risks from emerging technologies like synthetic biology and artificial intelligence.
While writing the paper, we considered alternative terms to describe the concept of differential technology development. Our leading alternatives were “responsible innovation sequencing” or “differential progress”. However, in the end we decided that differential technological development was the best fit for integrating this article into the responsible innovation literature. That said, we do see merit to either defining technology broadly to include “not only gadgets but also methods, techniques and institution design principles” or to extending the concept to include forms of differential development beyond technology development.
Amongst other things, paper outlines four relevant categories of technologies:
Risk increasing technologies cause negative societal impacts by causing insidious harm or through their potential to cause a catastrophe.
Safety technologies reduce or prevent negative societal impacts by modifying risk-increasing technologies.
Defensive technologies decrease risks from risk-increasing technologies without modifying these technologies.
Substitute technologies achieve similar benefits to a risk-increasing technology while featuring less risk.
We hope that this paper can provide a useful framing for discussions and future research about differential technology development, as well as generating ideas for how it may be best implemented in practice.
We are grateful to Michael Aird, Markus Anderljung, Jan Ole Ernst, Ben Garfinkel, Sihao Huang, Matthijs Maas, Cassidy Nelson, and James Wagstaff for useful discussions and comments on the manuscript. Furthermore, we are also grateful for feedback from participants of work-in-progress meetings of the Future of Humanity Institute and Centre for the Governance of AI. We thank Shrestha Rath for help with formatting and organising references. Jonas B. Sandbrink’s doctoral research is funded by Open Philanthropy. Hamish Hobbs’ contribution to the paper largely occurred while funded as a Research Scholar at the Future of Humanity Institute.
Differential technology development: preprint on the concept
Summary
We have published a preprint of a paper describing the concept of differential technology development and exploring when and how its implementation may be viable. This manuscript has been in the works at FHI for a while, so we would love your feedback!
Authors: Jonas B. Sandbrink, Hamish Hobbs, Jacob L. Swett, Allan Dafoe, Anders Sandberg
Further details
Nick Bostrom articulated the concept of Differential Technological Development in Superintelligence. This concept, and the concept of differential progress more broadly, has already been fairly widely discussed and written about, but within a limited community. This includes here on the EA Forum. The concept is typically used to argue that it would be beneficial to accelerate risk-reducing technological progress and retard risk-increasing technological progress, to manage potentially catastrophic or existential technological risks such as those from advanced AI systems and biotechnologies.
With this paper, we attempt to thoroughly explore the concept, make it accessible to the broader research and policy community, and explore the contexts in which its implementation is likely to be viable. To our knowledge, this is the first academic article seeking to comprehensively articulate a principle of differential technology development.
In the paper we argue that:
Responsible innovation efforts to date have largely focussed on shaping individual technologies.
However, as demonstrated by the preferential advancement of low-emission technologies, certain technologies reduce risks from other technologies or constitute low-risk substitutes.
Governments and other relevant actors may leverage risk-reducing interactions across technology portfolios to mitigate risks beyond climate change.
We adapt Nick Bostrom’s original concept to propose a responsible innovation principle of “differential technology development”, which calls relevant actors to leverage risk-reducing interactions between technologies by affecting their relative timing.
Thus, it may be beneficial to delay risk-increasing technologies and preferentially advance risk-reducing defensive, safety, or substitute technologies.
Implementing differential technology development requires the ability to anticipate or identify impacts and intervene in the relative timing of technologies.
We find that both are sometimes viable and that differential technology development may still be usefully applied even late in the diffusion of some harmful technologies.
A principle of differential technology development may inform government research funding priorities and technology regulation, as well as philanthropic research and development funders and corporate social responsibility measures.
Differential technology development may be particularly promising to mitigate potential catastrophic risks from emerging technologies like synthetic biology and artificial intelligence.
While writing the paper, we considered alternative terms to describe the concept of differential technology development. Our leading alternatives were “responsible innovation sequencing” or “differential progress”. However, in the end we decided that differential technological development was the best fit for integrating this article into the responsible innovation literature. That said, we do see merit to either defining technology broadly to include “not only gadgets but also methods, techniques and institution design principles” or to extending the concept to include forms of differential development beyond technology development.
Amongst other things, paper outlines four relevant categories of technologies:
Risk increasing technologies cause negative societal impacts by causing insidious harm or through their potential to cause a catastrophe.
Safety technologies reduce or prevent negative societal impacts by modifying risk-increasing technologies.
Defensive technologies decrease risks from risk-increasing technologies without modifying these technologies.
Substitute technologies achieve similar benefits to a risk-increasing technology while featuring less risk.
We hope that this paper can provide a useful framing for discussions and future research about differential technology development, as well as generating ideas for how it may be best implemented in practice.
Preprint link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4213670
Acknowledgements
We are grateful to Michael Aird, Markus Anderljung, Jan Ole Ernst, Ben Garfinkel, Sihao Huang, Matthijs Maas, Cassidy Nelson, and James Wagstaff for useful discussions and comments on the manuscript. Furthermore, we are also grateful for feedback from participants of work-in-progress meetings of the Future of Humanity Institute and Centre for the Governance of AI. We thank Shrestha Rath for help with formatting and organising references. Jonas B. Sandbrink’s doctoral research is funded by Open Philanthropy. Hamish Hobbs’ contribution to the paper largely occurred while funded as a Research Scholar at the Future of Humanity Institute.