The Unjournal: reporting some progress
Our group (curating articles and evaluations) is now live on Sciety HERE.
The first evaluated research project (paper) has now been posted HERE .
First evaluation: Denkenberger et al
Our first evaluation is for “Long Term Cost-Effectiveness of Resilient Foods for Global Catastrophes Compared to Artificial General Intelligence Safety”, by
David Denkenberger, Anders Sandberg, Ross Tieman, and Joshua M. Pearce, published in the International Journal of Disaster Risk Reduction.
These three reports and ratings (see a sample below) come from three experts with (what we believe to be) complementary backgrounds (note, these evaluators agreed to be identified rather than remain anonymous):
Alex Bates: An award-winning cost-effectiveness analyst with some background in considering long-term and existential risks
Anca Hanea: A senior researcher and applied probabilist based at the Centre of Excellence for Biosecurity Risk Analysis (CEBRA) at the University of Melbourne. She has done prominent research into eliciting and aggregating (expert) judgments, working with the RepliCATS project.
These evaluations were, overall, fairly involved. They engaged with specific details of the paper as well as overall themes, directions, and implications. While they were largely positive about the paper, they did not seem to pull punches. Some examples of their feedback and evaluation below (direct quotes).
Extract of evaluation content
I’d be surprised if I ever again read a paper with such potential importance to global priorities.
My view is that it would be premature to reallocate funding from AI Risk reduction to resilient food on the basis of this paper alone. I think the paper would have benefitted from more attention being paid to the underlying theory of cost-effectiveness motivating the investigation. Decisions made in places seem to have multiplied uncertainty which could have been resolved with a more consistent approach to analysis.
The most serious conceptual issue which I think needs to be resolved before this can happen is to demonstrate that ‘do nothing’ would be less cost-effective than investing $86m in resilient foods, given that the ‘do nothing’ approach would potentially include strong market dynamics leaning towards resilient foods.”.
the authors’ cost-effectiveness model, which attempts to decrease uncertainty about the potential uncertainty-reducing and harm/likelihood-reducing ‘power’ of resilient food R&D and compare it to R&D on AGI safety, is an important contribution”
It would have been useful to see a brief discussion of some of these acknowledged epistemic uncertainties (e.g., the impact of resilient foods on public health, immunology, and disease resistance) to emphasize that some epistemic uncertainty could be reduced by exactly the kind of resilient food R&D they are advocating for.
The structure of the models is not discussed. How did [they] decide that this is a robust structure (no sensitivity to structure performed as far as I understood)”
It is unclear if the compiled data sets are compatible. I think the quantification of the model should be documented better or in a more compact way.”
The authors also responded in detail. Some excerpts:
The evaluations provided well thought out and constructively critical analysis of the work, pointing out several assumptions which could impact findings of the paper while also recognizing the value of the work in spite of some of these assumptions. “
Thank you for highlighting this point [relating to discount rate], this is an important consideration that would make valuable future work.
They also respond specifically to many points, e.g.,
In reality we anticipate that there are a myriad of ways in which nuclear risk and AGI would interact with one another. …
I do not attempt to summarize the overall evaluations and responses here. If you are interested in helping do this, or communicating the synthesis, please dm me.
Followup and AMA
We have roughly ten further papers in the evaluation pipeline, with an emphasis on potentially-high impact work published in the NBER working paper series. We expect two more to be released soon.
AMA: I will try to take this milestone as an opportunity to answer any questions you may have about The Unjournal. I’ll try to respond within 48 hours.
(Again, see https://bit.ly/eaunjournal for our Gitbook project space, which should answer some questions.)
His research “focuses on how scientists and policymakers collaborate to address global catastrophic risks such as climate change, pandemics, and other emerging threats … strategies and tools that we can use to make better decisions under deep uncertainty”. He was previously a Visiting Researcher at the Future of Humanity Institute (FHI).