christian.r

Karma: 1,018

Christian Ruhl, Founders Pledge

I am a Senior Researcher at Founders Pledge, where I work on global catastrophic risks. Previously, I was the program manager for Perry World House’s research program on The Future of the Global Order: Power, Technology, and Governance. I’m interested in biosecurity, nuclear weapons, the international security implications of AI, probabilistic forecasting and its applications, history and philosophy of science, and global governance. Please feel free to reach out to me with questions or just to connect!

christian.r 4 Mar 2022 19:01 UTC
11 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
A Project Candor for Global Catastrophic Risks
Biorisk and Recovery from Catastrophe, Values and Reflective Processes, Effective Altruism
This is a proposal to fund a large-scale public communications project on global catastrophic risks (GCRs), modeled on the Eisenhower administration’s Project Candor. Project Candor was a Cold War public relations campaign to “inform the public of the realities of the ‘Age of Peril’” (see Unclassified 1953 Memo from Eisenhower Library). Policymakers were concerned that the public did not yet understand that the threats from nuclear weapons and the Soviet Union had inaugurated a new era in human history: the Age of Peril. Today, at the precipice, the Age of Peril continues with possible risks from engineered pandemics, thermonuclear exchange, great power war, and more. Voting behavior and public discourse, however, do not seem attuned to these risks. A new privately-funded Project Candor would communicate to the public the nature of the threats, their probabilities, and what we can do about them. This proposal is related to “a fund for movies and documentaries” and “new publications on the most pressing issues,” but differs in that it would be a unified and coordinated campaign across multiple media.

christian.r 4 Mar 2022 19:18 UTC
8 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Creative Arms Control
Biorisk and Recovery from Catastrophe
This is a proposal to fund research efforts on “creative arms control,” or non-treaty-based international governance mechanisms. Traditional arms control—formal treaty-based international agreements—has fallen out of favor among some states, to the extent that some prominent policymakers have asked whether we’ve reached “The End of Arms Control.”^[1] Treaties are difficult to negotiate and may be poorly suited to some fast-moving issues like autonomous weapons, synthetic biology, and cyber operations; by the time traditional arms control is negotiated, the technology may have outpaced the regulations. Partly for this reason, states and private actors alike have increasingly turned to informal “norms” processes (e.g. on cyber), codes of conduct and technical agreements, or confidence-building measures (CBMs). How well does such “creative arms control” work? Is it a suitable instrument for regulating emerging technologies this century? How hard is it to turn a norms process into a verification-based treaty regime? Research on these questions is still thin, and greater funding could therefore be very valuable for future regulation on GCR/X-risk-related technologies.
1. ^
  For a while, it seemed like the 2021 extension of New START had invalidated Ambassador Brooks’s points in that 2020 article. I think Russia’s 2022 invasion of Ukraine is going to make agreement on formal arms control very difficult again.

christian.r 4 Mar 2022 22:47 UTC
3 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Experimental Wargames for Great Power War and Biological Warfare
Biorisk and Recovery from Catastrophe, Epistemic Institutions
This is a proposal to fund a series of “experimental wargames,” on great power war and biological warfare. Wargames have long been a standard tool of think tanks, the military, and the academic IR world since the early Cold War. Until recently, however, these games were largely used to uncover unknown unknowns and help with scenario planning. Most such games continue to be unscientific exercises. Recent work on “experimental wargames” (see, e.g. this paper on drones and escalation), however, has leveraged wargaming methods with randomly-assigned groups and varying scenarios to see how decision-makers will react in hypothetical crisis situations. A series of well-designed experimental wargames on crisis decision-making in a great power confrontation or during a biological attack could help identify weaknesses, quantify risks, and uncover cognitive biases at work in high-pressure decision-making. Additionally, they would have the added benefit of raising awareness about global catastrophic risks.

christian.r 8 Mar 2022 16:37 UTC
24 points
0 ∶ 0
on: Comparing top forecasters and domain experts
Thank you for writing this overview! I think it’s very useful. A few notes on the famous “30%” claim:
- Part of the problem with fully understanding the performance of IC analysts is that much of the information about the tournaments and the ICPM is classified.
- What originally happened is that someone leaked info about ACE to David Ignatius, who then published it in his column. (The IC never denied the claim.^[1]) The document you cite is part of a case study by MITRE that’s been approved for public release.
One under-appreciated takeaway that you hint at is that prediction markets (rather than non-market aggregation platforms) are poorly suited to classified environments. Here’s a quote from a white paper I co-wrote last year:^[2]
“Prediction markets are especially challenging to implement in classified environments because classified markets will necessarily have large limitations on participation, requiring the use of algorithmic correctives to solve liquidity problems. Good liquidity, like that of a well-functioning stock market, is difficult to achieve in prediction markets like the ICPM, requiring prediction markets to have corrective tools like setting liquidity parameters and using automated market makers, which attempt to simulate efficient market behavior in electronic prediction markets.”
More broadly, I would like to push back a little against the idea that your point 3(a) ( whether supers outperform IC analysts) is really much evidence for or against 3 (whether supers outperform domain experts).
First, the IARPA tournaments asked a wide range of questions, but intelligence analysts tend to be specialized. If you’re looking at the ICPM, are you really looking at the performance of domain experts? Or are you looking at e.g. an expert on politics in the Horn of Africa trying to forecast the price of the Ruble? On the one hand, since participants self-selected which questions they answered, we might expect domain experts to stick to their domain. On the other, analysts might have seen it as a “game,” a “break,” or “professional development”—in short, an opportunity to try their hand something outside their expertise. The point is that we simply don’t know whether the ICPM really reflects “expert” opinion.
Second, I am inclined to believe that comparisons between IC analysts and supers may tell us more about the secrecy heuristic than about forecaster performance. From the same white paper:
“Experimental research on using secrecy as a heuristic for informational quality demonstrates that people tend to weigh secret information more heavily than publicly available information, viewing secret information as higher quality than public information.^[3] Secrecy does matter, especially in situations where information asymmetry exists, but a pervasive secrecy bias may negatively affect the accuracy of a classified crowd in some cases.”
I personally see much of the promise of forecasting platforms not as a tool for beating experts, but as a tool for identifying them more reliably (more reliably than by the usual signals, like a PhD).
1. ^
  Tetlock discusses this a bit in Chapter 4 of Superforecasting.
2. ^
  Keeping Score: A New Approach to Geopolitical Forecasting, https://global.upenn.edu/sites/default/files/perry-world-house/Keeping%20Score%20Forecasting%20White%20Paper.pdf.
3. ^
  Travers et al., “The Secrecy Heuristic,” https://www.jstor.org/stable/43785861.
What links here?
- Gavin's comment on Comparing top forecasters and domain experts by Gavin (8 Mar 2022 18:35 UTC; 3 points)

christian.r 8 Mar 2022 20:10 UTC
5 points
0 ∶ 0
in reply to: Gavin’s comment on: Comparing top forecasters and domain experts
Thanks Gavin! That makes sense on how you view this and (3).

christian.r 9 May 2022 12:12 UTC
4 points
0 ∶ 0
on: Space governance—problem profile
Hi Fin!

This is great. Thank you for writing it up and posting it! I gave it a strong upvote.

(TLDR for what follows: I think this is very neglected, but I’m highly uncertain about tractability of formal treaty-based regulation)

As you know, I did some space policy-related work at a think tank about a year ago, and one of the things that surprised us most is how neglected the issue is — there are only a handful of organizations seriously working on it, and very few of them are the kinds of well-connected and -respected think tanks that actually influence policy (CSIS is one). This is especially surprising because — as Jackson Wagner writes below — so much of space governance runs through U.S. policy. Anyway, I think that’s another point in favor of working on this!

As I think I mentioned when we talked about space stuff a little while ago, I’m a bit skeptical about tractability of “traditional” (ie formal, treaty-based) arms control. You note some of the challenges in the 80K version of the write up. Getting the major powers to agree to anything right now, let alone something as sensitive as space tech, seems unlikely. Moreover, the difficulties of verification and ease of cheating are high, as they are with all dual-use technology. Someone can come up with a nice “debris clean up” system that just happens to also be a co-orbital ASAT, for example.

But I think there are other mechanisms for creating “rules of the orbit” — that’s the word Simonetta di Pippo, the director of UNOOSA used at a workshop I helped organize last year. (https://global.upenn.edu/sites/default/files/perry-world-house/Dipippo_SpaceWorkshop.pdf)

Cyber is an example where a lot of actors have apparently decided that treaty-based arms control isn’t going to cut it (in part for political reasons, in part because the tech moves so fast), but there are still serious attempts at creating norms and regulation (https://carnegieendowment.org/2020/02/26/cyberspace-and-geopolitics-assessing-global-cybersecurity-norm-processes-at-crossroads-pub-81110). That includes standard setting and industry-driven processes, which feel especially appropriate in space, where private actors play such an important role. We have a report on autonomous weapons and AI-enabled warfare coming out soon at Founders Pledge, and I think that’s another space where people put too much emphasis on treaty-based regulation and neglect norms and confidence building measures for issues where great powers can agree on risk reduction.

Again, I think this is a great write up, and love that you are drawing attention to these issues. Thank you!

christian.r 19 May 2022 10:35 UTC
74 points
0 ∶ 0
on: Are you really in a race? The Cautionary Tales of Szilárd and Ellsberg
Hi Haydn,
This is awesome! Thank you for writing and posting it. I especially liked the description of the atmosphere at RAND, and big +1 on the secrecy heuristic being a possibly big problem.^[1] Some people think it helps explain intelligence analysts’ underperformance in the forecasting tournaments, and I think there might be something to that explanation.
We have a report on autonomous weapons systems and military AI applications coming out soon (hopefully later today) that gets into the issue of capability (mis)perception in arms races too, and your points on competition with China are well taken.
What I felt was missing from the post was the counterfactual: what if the atomic scientists’ and defense intellectuals’ worst fears about their adversaries had been correct? It’s not hard to imagine. The USSR did seem poised to dominate in rocket capabilities at the time of Sputnik.
I think there’s some hindsight bias going on here. In the face of high uncertainty about an adversary’s intentions and capabilities, it’s not obvious to me that skepticism is the right response. Rather, we should weigh possible outcomes. In the Manhattan Project case, one of those possible outcomes was that a murderous totalitarian regime would be the first to develop nuclear weapons, become a permanent regional hegemon, or worse, a global superpower. I think the atomic scientists’ and U.S. leadership’s decision then was the right one, given their uncertainties at the time.
I think it would be especially interesting to see whether misperception is actually more common historically. But I think there are examples of “racing” where assessments were accurate or even under-confident (as you mention, thermonuclear weapons).
Thanks again for writing this! I think you raise a really important question — when is AI competition “suboptimal”?^[2]
1. ^
  https://www.jstor.org/stable/43785861
2. ^
  In Charles Glaser’s sense (https://www.belfercenter.org/sites/default/files/files/publication/glaser.pdf)
What links here?
- Why policymakers should beware claims of new “arms races” (Bulletin of the Atomic Scientists) by christian.r (14 Jul 2022 13:38 UTC; 55 points)

christian.r 19 May 2022 13:10 UTC
13 points
0 ∶ 0
in reply to: HaydnBelfield’s comment on: Are you really in a race? The Cautionary Tales of Szilárd and Ellsberg
Thank you for the reply! I definitely didn’t mean to mischaracterize your opinions on that case :)

Agreed, a project like that would be great. Another point in favor of your argument that this is a dynamic to watch out for on AI competition is if verifying claims of superiority is harder for software (along the lines of Missy Cummings’s “The AI That Wasn’t There” https://tnsr.org/roundtable/policy-roundtable-artificial-intelligence-and-international-security/#essay2). That seems especially vulnerable to misperceptions

christian.r 19 May 2022 14:07 UTC
3 points
0 ∶ 0
in reply to: HaydnBelfield’s comment on: Risks from Autonomous Weapon Systems and Military AI
Hi Haydn,

That’s a great point. I think you’re right — I should have dug a bit deeper on how the private sector fits into this.

I think cyber is an example where the private sector has really helped to lead — like Microsoft’s involvement at the UN debates, the Paris Call, the Cybersecurity Tech Accord, and others — and maybe that’s an example of how industry stakeholders can be engaged.

I also think that TEVV-related norms and confidence building measures would probably involve leading companies.

I still broadly thinking that states are the lever to target at this stage in the problem, given that they would be (or are) driving demand. I am also always a little unsure about using cluster munitions as an example of success — both because I think autonomous weapons are just a different beast in terms of military utility, and of course because of the breaches (including recently).

Thank you again for pointing out that hole in the report!

christian.r 20 May 2022 0:43 UTC
11 points
0 ∶ 0
in reply to: Kevin Lacker’s comment on: Risks from Autonomous Weapon Systems and Military AI
Hi Kevin,

Thank you for your comment and thanks for reading :)

The key question for us is not “what is autonomy?” — that’s bogged down the UN debates for years — but rather “what are the systemic risks of certain military AI applications, including a spectrum of autonomous capabilities?” I think many systems around today are better thought of as closer to “automated” than truly “autonomous,” as I mention in the report, but again, I think that binary distinctions like that are less salient than many people think. What we care about is the multi-dimensional problem of more and more autonomy in more and more systems, and how that can destabilize the international system.

I agree with your point that it’s a tricky definitional problem. In point 3 under the section on the “Killer Robot Ban” in the report, one of the key issues there is “The line between autonomous and automated systems is blurry.” I think you’re pointing to a key problem with how people often think about this issue.

I’m sorry I won’t be able to give a satisfying answer about “ethical norms” as it’s a bit outside the purview of the report, which focuses more on strategic stability and GCRs. (I will say that I think the idea of “human in the loop” is not the solution it’s often made out to be, given some of the issues with speed and cognitive biases discussed in the report). There are some people doing good work on related questions in international humanitarian law though that will give a much more interesting answer.

Thanks again!

christian.r 20 May 2022 16:53 UTC
6 points
0 ∶ 0
in reply to: Gentzel’s comment on: Risks from Autonomous Weapon Systems and Military AI
Thanks for engaging so closely with the report! I really appreciate this comment.

Agreed on the weapon speed vs. decision speed distinction — the physical limits to the speed of war are real. I do think, however, that flash wars can make non-flash wars more likely (eg cyber flash war unintentionally intrudes on NC3 system components, that gets misinterpreted as preparation for a first strike, etc.). I should have probably spelled that out more clearly in the report.

I think we actually agree on the broader point — it is possible to leverage autonomous systems and AI to make the world safer, to lengthen decision-making windows, to make early warning and decision-support systems more reliable.

But I don’t think that’s a given. It depends on good choices. The key questions for us are therefore: How do we shape the future adoption of these systems to make sure that’s the world we’re in? How can we trust that our adversaries are doing the same thing? How can we make sure that our confidence in some of these systems is well-calibrated to their capabilities? That’s partly why a ban probably isn’t the right framing.

I also think this exchange illustrates why we need more research on the strategic stability questions.

Thanks again for the comment!

christian.r 26 May 2022 13:12 UTC
6 points
0 ∶ 0
in reply to: Leo’s comment on: Building a Better Doomsday Clock
Thank you! I also really struggle with the clock metaphor. It seems to have just gotten locked in as the Bulletin took off in the early Cold War. The time bomb is a great suggestion — it communicates the idea much better

christian.r 7 Aug 2022 12:43 UTC
5 points
0 ∶ 0
on: Islands, nuclear winter, and trade disruption as a human existential risk factor
Really enjoyed reading this and learned a lot. Thank you for writing it! I’m especially intrigued by the proposal for regional alliances in table 6 — including the added bit about expansionist regional powers in the co-benefits column of the linked supplemental version of the table.

I was curious about one part of the paper on volcanic eruptions. You wrote that eg “Indonesia harbours many of the world’s large volcanoes from which an ASRS could originate (eg, Toba and Tambora eruptions).” Just eyeballing maps of the biggest known volcanoes, the overlap with some island refuges seems concerning. Do we know what ash deposit models say about how much these places would get covered in ash for various kinds of volcanic eruptions in the region and what this would mean for infrastructure and agriculture?

Thanks!

christian.r 8 Aug 2022 11:21 UTC
1 point
0 ∶ 0
in reply to: Matt Boyd’s comment on: Islands, nuclear winter, and trade disruption as a human existential risk factor
Thanks, Matt! That makes sense

christian.r 15 Aug 2022 18:22 UTC
10 points
0 ∶ 0
on: The Parable of the Boy Who Cried 5% Chance of Wolf
Thanks for writing this! I think it’s great. Reminds me of another wild animal metaphor about high-stakes decision-making under uncertainty—Reagan’s 1984 “Bear in the Woods” campaign ad:
There is a bear in the woods. For some people, the bear is easy to see. Others don’t see it at all. Some people say the bear is tame. Others say it’s vicious and dangerous. Since no one can really be sure who’s right, isn’t it smart to be as strong as the bear—if there is a bear?
I think that kind of reasoning is helpful when communicating about GCRs and X-risks.

christian.r 16 Sep 2022 16:08 UTC
18 points
2 ∶ 0
on: The Pugwash Conferences and the Anti-Ballistic Missile Treaty as a case study of Track II diplomacy
Hi Rani, it’s great to see the report out. It’s good to have this clear deep dive on the canonical case. I especially like that it points to some attributes of track II dialogues that we should pay special attention to when evaluating them as potential interventions. Great work!

christian.r 24 Jan 2023 18:22 UTC
7 points
0 ∶ 0
in reply to: Will Aldred’s comment on: Call me, maybe? Hotlines and Global Catastrophic Risk [Founders Pledge]
Thanks, will! Really appreciate this comment :)
Credit for the title actually goes to my colleagues on the FP research team (I believe it was Tom or Johannes who first came up with it).

christian.r 24 Jan 2023 22:38 UTC
7 points
2 ∶ 0
in reply to: ParetoPrinciple’s comment on: Call me, maybe? Hotlines and Global Catastrophic Risk [Founders Pledge]
Hi ParetoPrinciple! I appreciate your engaging with the document :)

I quote Schelling throughout (and think he actually makes some of the points you hint at more clearly in The Strategy of Conflict — eg the quote in footnote 61 here). You’re definitely right that no hotlines discussion would be complete without this!

christian.r 14 Feb 2023 21:07 UTC
2 points
0 ∶ 0
in reply to: Conor Barnes’s comment on: Philanthropy to the Right of Boom [Founders Pledge]
Thanks, Conor!

christian.r 15 Feb 2023 15:12 UTC
1 point
0 ∶ 0
in reply to: Stephen Clare’s comment on: Philanthropy to the Right of Boom [Founders Pledge]
Thanks for the kind comment, Stephen! You’re right I phrased that wrong—it is about tractability, not probability. I agree with you that the tractability of escalation control is probably the biggest issue here, but I also think we should expect low-hanging fruit given the relative neglectedness. There are a couple of concrete projects that I am/would be be excited about:
- Escalation management with North Korea -- How can we keep nuclear war limited with North Korea? As I understand it, the problem of deterrence after DPRK first use has gotten relatively little attention, despite its relevance to U.S.-China conflict. A project to study this and help the U.S. and China find guardrails, CBMs, etc. to work on this seems especially valuable. The broader class of interventions here is something like “keeping ‘smaller’ nuclear wars geographically and politically limited”;
- Intrawar crisis communications projects—especially understanding China’s attitude to crisis communications. How can states better communicate the desire to terminate a war and the terms of peace once nuclear war has broken out?
- Three-body escalation control—Like you, I’m concerned about a future where China has a much larger arsenal (see China Military Power Report), and the fact that U.S. planners have barely begun to think about the “nuclear three body problem” with China and Russia. A large-scale project that seeks to better understand these dynamics and what, if any, of the game theory from the Cold War applies to three-way escalation control, how different theories of limited war might interact, etc. would be valuable.