This is a really good summary. I think the main area I have remaining uncertainty about these types of arguments is around if tech decisively favors finders or not. I believe there’s been a lot of analysis that implies that modern nuclear submarines are essentially not feasible to track and destroy, and there are others that argue AI-enabled modeling and simulation could be applied as a “fog of war” machine to simulate an opponent’s sensor system and optimize concealment of nuclear forces.
Nevertheless, without more detail these sorts of counterarguments seem highly contingent to me: they depend on the state of technology that actually gets deployed, and there may be decisive counter-counter arguments based on other military capabilities.
It would be interesting to see more EAs “grok” these sorts of arguments and to think through corresponding nuclear modernization and arms control strategies that would be implied by trying to assure a desirable long-term future. I’ve tended to think more redundant and accurate arsenals of low-yield weapons like neutron bombs would tend to strengthen deterrence while eliminating most of the risk of nuclear winter and long-term radiation, but it’s easy to imagine there might be better directions yet to ratchet competition and negotiation since that kind of proposal could also be very destabilizing!
Gentzel
On flash-war risks, I think a key variable is what the actual forcing function is on decision speed and the key outcome you care about is the decision quality.
Fights where escalation is more constrained by decision making speed than weapon speed are where we should expect flash war dynamics. These could include: short-range conflicts, cyber wars, the use of directed energy weapons, influence operations/propaganda battles, etc.
For nuclear conflict, unless some country gets extremely good at stealth, strategic deception, and synchronized mass incapacitation/counterforce, there will still be warning and a delay before impact. The only reasons to respond faster than dictated by the speed of the adversary weapons and delays in your own capacity to act would be if doing so could further reduce attrition, or enable better retaliation… but I don’t see much offensive prospect for that. If the other side is doing a limited strike, then you want to delay escalation/just increase survivability, if the other side is shooting for an incapacitating strike, then their commitment will be absolutely massive and their pre-mobilization high, so retaliation would be your main option left at that point anyway. Either way, you might get bombers off the ground and cue up missile defenses, but for second strike I don’t see that much advantage to speeds faster than those imposed by the attacker, especially given the risk of acting on false alarm. This logic seems to be clearly present in all the near miss cases: there is the incentive to wait for more information from more sensors.
Improving automation in sensing quality, information fusion, and attention rationing would all seem useful for finding false alarms faster. In general it would be interesting to see more attention put into AI-enabled de-escalation, signaling, and false alarm reduction.
I think most of the examples of nuclear risk near misses favor the addition of certain types of autonomy, namely those that increase sensing redundancy and thus contribute to to improving decision quality and expanding the length of the response window. To be concrete:For the Stanislav example: if the lights never start flashing in the first place because of the lack of radar return (e.g. if the Soviets had more space-based sensors), then there’d be no time window for Stanislav to make a disastrous mistake. The more diverse and high quality sensors you have, and the better feature detection you have, the more accurate a picture you will have and the harder it will be for the other side to trick you.
If during the Cuban missile crisis, the submarine which Arkhipov was on knew that the U.S. was merely dropping signaling charges (not attacking), then there would have been no debate about nuclear attack: the Soviets would have just known they’d been found.
In the training tape false alarm scenario: U.S. ICBMs can wait to respond because weapon arrival is not instant, satellite sensors all refute the false alarm: catastrophe averted. If you get really redundant sensor systems that can autonomously refute false alarms, you don’t get such a threatening alert in the first place, just a warning that something is broken in your overall warning infrastructure: this is exactly what you want.
Full automation of NC3 is basically a decision to attack, and something you’d only want to activate at the end of a decision window where you are confident that you are being attacked.
The cluster munitions divestment example seems plausibly somewhat more successful in the West, but not elsewhere (e.g. the companies that remain on the “Hall of Shame” list). I’d expect something similar here if the pressure against LAWs were narrow (e.g. against particular types with low strategic value). Decreased demand does seem more relevant than decreased investment though.
If LAWs are stigmatized entirely, and countries like the U.S. don’t see a way to tech their way out to sustain advantage, then you might not get the same degree of influence in the first place since demand remains.
I find it interesting that the U.S. wouldn’t sign the Convention on Cluster Munitions, but also doesn’t seem to be buying or selling any more. One implication might be that the stigma disincentivizes change/tech progress: since more discriminant cluster munitions would be stigmatized as well. I presume this reduces the number of such weapons, but increases the risk of collateral damage per weapon by slowing the removal of older, more indiscriminate/failure prone weapons from arsenals.
https://www.washingtonpost.com/news/checkpoint/wp/2016/09/02/why-the-last-u-s-company-making-cluster-bombs-wont-produce-them-anymore/
While in principle, you could drive down the civilian harm with new smaller bomblets that reliably deactivate themselves if they don’t find a military target, as far as I can tell, to the degree that the U.S. is replacing cluster bombs, it is just doing so with big indiscriminate bombs (BLU 136/BLU134) that will just shower a large target area with fragments.
Given the time it takes to form relationships with nodes in decision making networks, and the difficulty of reducing uncertainty from the outside, at some point it makes sense to either aim people at such jobs or to make friends with people in them. That lobbying and working in government aren’t unique tactics or roles in society doesn’t matter if they are neglected by those who are capable of pursuing similar goals: different organizations compete for influence in different directions. Early investment to enable direct interaction with decision making networks can be how you get the “when” right, figure out “who” to target, and sometimes even figure out the “what to improve” by seeing what is going wrong in the first place.
If an outside organization only does outside research and competitors invest more in making internal connections, the competitors gain advantage and influence with time. Even if one gains a more objective perspective by looking in from the outside and avoiding political fights, a lot of the most valuable information for decision making is going to be internal. This failure mode leads to forms of naivety that are persistent: external actors can see things that clearly look like mistakes, by actors with biases that are obvious to outsiders, and then conclude more confidently than is justified that their own views are correct.
I am not sure we should focus more into this area, I just want to make sure that in general, people who go into policy or advocacy don’t propagate bad ideas, or discredit EA with important people who would otherwise be aligned with our goals in the future.
I do think that knowing the history of transformative technologies (and policies that effected how they were deployed) will have a lot of instrumental value for EAs trying to make good decisions about things like gene editing and AI.
You seem to be missing the part where most people are disagreeing with the post in significant ways.
F-15s and MRAPs still have to be operated by multiple people, which requires incentive alignment between many parties. Some autonomous weapons in the future may be able to self-sustain and repair, (or be a part of a self-sustaining autonomous ecosystem) which would mean that they can be used while being aligned with fewer people’s interests.
A man-at arms wouldn’t be able to take out a whole town by himself if more than a few peasants coordinate with pitchforks, but depending on how LAWS are developed, a very small group of people could dominate the world.
I actually agree with a lot of your arguments, but I don’t agree overall. AI weapons will be good and bad in many ways, and if the are good or bad overall depends on who has control, how well they are made, and the dynamics of how different countries race and adapt.
Getting Nuclear Policy Right Is Hard
That is the point.
The reason it is appropriate to call this ethical reaction time, rather than just reaction time is because the focus of planning and optimization is around ethics and future goals. To react quickly with respect to an opportunity that is hard to notice, you have to be looking for it.
Technical reaction time is a better name in some ways, but it implies too narrow of a focus, while just reaction time implies too wide of a focus. There probably is a better name though.
I just added some examples to make it a bit more concrete.
I think you may be misunderstanding what I mean by ethical reaction time, but I may change the post to reduce the confusion. I think adding examples would make the post a lot more valuable.
Basically, what I mean by ethical reaction time is just being able to make ethical choices as circumstances unfold and not be caught unable to act, or acting in a bad way.
Here’s a few examples, some hypothetical, some actual:
Policy:
One can imagine the Reach Every Mother and Child Act might have passed last year if a few congressmen were more responsive in adjusting it to get past partisan objections (left wing opposing funding religious groups, right wing opposing funding contraception). That likely would have saved a few thousand lives, and possibly millions according to USAID. (http://effective-altruism.com/ea/pk/how_to_support_and_improve_the_reach_every_mother/) My model of the political constraints on the Reach Act may be wrong here though.
Any regulation of technology that starts occurring in response to the technology rapidly coming into existence: Uber executed its plans faster than it could be banned, which was probably good. We don’t necessarily want the same to be true for certain tech risks in AI and biology, which makes it important to figure out quickly the correct way to regulate things.
Anything on the U.S. Federal Register that is going poorly: if you don’t notice a new rule come up relevant to your area of interest (easy to imagine this with animal welfare and new tech) and respond within the comment period, your concerns aren’t going to inform the regulation (if you put in relevant research, and get ignored, you can sue the federal agency and win: this happened when the FDA first failed to ban trans fat). This is also a sort of situation that may actually require you to do research under a lot of time pressure.
Donor coordination:
If your organization is not prepared to accept/talk to donors, there are often times you will lose a lot that you’d otherwise get. This I think is one of the reasons David Goldberg with Founder’s Pledge would carry a backpack with him containing everything needed for someone to legally commit some % of their venture cash out value to effective charities.
Start-ups:
If initially you need partnerships/funding and others are competing for those partnerships/funding then OODA loop applies (but less for funding since there can be more sources).
Salary negotiations:
It makes a lot of sense to know how you are going to negotiate ahead of time, or to be very quick in thought. (http://haseebq.com/my-ten-rules-for-negotiating-a-job-offer/) Saying the wrong thing could cost you thousands of dollars, and if you are donating that to AMF, that’s likely costing lives.
Grants/research opportunities:
Grant opportunities = competitive = OODA loop. Less true when there is a deadline that is far away however.
In 2015, there were several EA organizations that had the opportunity to get free research from grad students who were interested in Effective Altruism from the School of Public Policy at the University of Maryland. In order to get free research, an organization would have had to submit a general/rough research proposal (<1 page), in which they could enumerate some of the means by which a study or literature review would be undertaken to maintain rigor/ guidelines for the advisor monitoring grad students. No one was able to react within the month after solicitation, so other non-EA organizations got free research instead for the grad student projects class. It does seem reasonable that EA orgs may have had better priorities, and that there is reason to be skeptical of the grad students, but it would have been a good way to get a bunch of students going into policy more bought into EA even if they didn’t produce work at a level we’d accept. This is also partly my fault, since I could have informed groups earlier, though not by a lot.
Handling the throughput vs. responsiveness trade off:
If you set up systems so that you can be reactive without draining as much of your attention, you can get more things done generally. Dropping responsiveness and responding to emails once per day or less may make sense if you are a coder/researcher, but it doesn’t make sense if you are an information node between organizations that need to coordinate. Adopting simple algorithms like writing down everything in my working memory before taking a call has made me both a lot more willing to take calls, and sped up my ability to get right back to work after interruption.
Time sensitive opportunities:
If factory farming and or malaria are going to be gone at some point in the next 20 years due to development/economics, then there won’t be the same opportunity to reduce suffering/save lives in the future that there is now. That being said, donations don’t require prompt reaction to opportunity the way policy opportunities with respect to these do.
Ethical Reaction Time: What it is and why it matters
This is the referenced program:
http://www.bia.gov/WhoWeAre/BIA/OIS/HumanServices/HousingImprovementProgram/
Sorry for taking so long to respond. This is the comment:
https://drive.google.com/file/d/0BzKyV3bimCTiWDhreW91Tm9RT1E/view?usp=sharing
Transgenic mosquitoes, update Effective Altruism Policy Analytics
Great summary of why I hate when people walk across the road instead of running. Or when people space themselves out instead of clustering so that no cars can get by.
How to support and improve the Reach Every Mother and Child Act
This is my current heuristic, though if we learn unexpected things from feedback I could imagine updating in a different direction:
If positive feedback (successful comment) --> Try to restart project
If really good negative feedback --> Make a better lessons learned post and propose a different type of project
If ambiguous negative feedback --> Recommend people avoid experimenting with this type of policy action and focus on other policy interventions.
FWIW, Los Alamos claims they replicated Hiroshima and the Berkeley Hills Fire Smoke Plumes with their fire models to within 1 km of plume height. It’s pretty far into the presentation though, and most of their sessions are not public, so I can hardly blame anyone for not encountering this.