Executive summary: The report develops a framework to weigh the historical benefits of science against potential future harms from misuse of advanced biotechnology. It concludes that faster science seems very unlikely to be bad overall based on lower risk estimates, the inevitability of such technologies emerging, and science’s potential to reduce long-term risks.
Key points:
The report tallies benefits of science against future costs like civilization-ending events enabled by technological advances. It calibrates risks using forecasts from the Existential Risk Persuasion tournament.
For non-catastrophic harms, historic benefits outweigh forecasted risks. However, for civilization-ending events, desirability of accelerating science depends on which risk estimates are used and value placed on the long-term future.
The author’s view is that faster science is very unlikely to be net negative due to lower existential risk estimates being more credible, advanced biotech likely emerging regardless, and science’s potential to reduce risks by enabling countermeasures.
Though unlikely, risks warrant concern and justify pursuing both scientific acceleration and technology risk reduction in parallel rather than worrying about their interaction.
Under the author’s preferred assumptions, the model estimates investing in science has a social impact 220x that of direct cash transfers, realized by raising incomes slightly for many generations.
While science acceleration seems desirable, the report argues existential risks themselves warrant separate concern and scrutiny.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: The report develops a framework to weigh the historical benefits of science against potential future harms from misuse of advanced biotechnology. It concludes that faster science seems very unlikely to be bad overall based on lower risk estimates, the inevitability of such technologies emerging, and science’s potential to reduce long-term risks.
Key points:
The report tallies benefits of science against future costs like civilization-ending events enabled by technological advances. It calibrates risks using forecasts from the Existential Risk Persuasion tournament.
For non-catastrophic harms, historic benefits outweigh forecasted risks. However, for civilization-ending events, desirability of accelerating science depends on which risk estimates are used and value placed on the long-term future.
The author’s view is that faster science is very unlikely to be net negative due to lower existential risk estimates being more credible, advanced biotech likely emerging regardless, and science’s potential to reduce risks by enabling countermeasures.
Though unlikely, risks warrant concern and justify pursuing both scientific acceleration and technology risk reduction in parallel rather than worrying about their interaction.
Under the author’s preferred assumptions, the model estimates investing in science has a social impact 220x that of direct cash transfers, realized by raising incomes slightly for many generations.
While science acceleration seems desirable, the report argues existential risks themselves warrant separate concern and scrutiny.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.