The COILS Framework for Decision Analysis: A Shortened Intro+Pitch
Edit 5/10/2022: If you would like to see example applications of this framework or other clarifications/objections, see my previous, long post about this.
Intro/Context: Where Are All the Advanced Pro-Con Analytical Frameworks?
Many people have heard of pro-con analysis: when you are trying to make a decision or conduct policy analysis, it can help to list out the plausible advantages and disadvantages of the options.
However, doing this analysis can be difficult, especially if the arguments involve uncertainty or if you already feel biased for or against some of the options.
Thus, I’ve long been confused why there is not also a widely-known or “standard” framework—even among EAs—for doing deeper pro-con analysis. I am especially confused by the non-popularity of such a framework among EAs despite how they already use frameworks/heuristics such as the “importance, tractability, neglectedness” (ITN) framework for broad cause area prioritization (despite its flaws).
The COILS Framework: A Simplified Description
I’ve long been developing and advocating a framework that I’ve temporarily labeled the COILS framework (Counterfactuality, Implementation, Linkage, Significance). This framework holds that any given advantage/disadvantage for a decision is built on four categories of claims/assumptions:
Advantage/disadvantage: “This plan would cause X, which is good/bad.”
Counterfactuality: X would not occur (to a similar extent) without the plan.
Implementation: The plan details and ultimate implementation will involve doing Y.
Linkage: X will occur following the plan implementation (Y).
Significance: X is morally good/bad.
To give an alternate, more-testable statement of this framework: Any rebuttal that one offers against any claimed advantage/disadvantage exclusively targets one or more of these four categories of claims; you cannot make a rebuttal outside of these four categories of claims. The four concept categories are therefore collectively exhaustive.
At least, I believe the above is probably accurate, although I definitely recognize it is possible I’ve made an oversight for reasons described in this footnote. If you do disagree with such a claim, please let me know why!
Additionally, I also think that the four categories are at least mostly mutually exclusive (i.e., the definitions don’t overlap), although I find it is often more efficient in application to not nitpick over the exact delineation between concept pairs like “implementation and linkage” and “linkage and significance.” However, I am less confident about this claim than the claim that they are collectively exhaustive.
Arguments in favor of using/popularizing this framework
I don’t think that using this framework is always optimal, especially for very simple decisions. However, there are two semi-overlapping categories of arguments as to why I think this framework probably ought to be used/popularized among EAs:
It seems helpful for checking some of your key assumptions, especially when you’re already biased in favor of believing some argument;
It standardizes/labelizes certain concepts (which seems helpful for reasons such as easier concept communication among groups and faster concept categorization for one’s own thinking).
Breaking down complex questions/concepts into smaller pieces is generally helpful if the smaller categories are still collectively exhaustive and do not involve significant duplication (i.e., are mostly mutually exclusive).
The EA community has already deemed the ITN framework to be useful (despite its flaws), and the COILS framework shares similarities with the ITN framework.
The COILS framework is adapted from a framework popular in competitive policy debate (the “stock issues”), and there are reasons to expect that the policy debate community would have some valid justifications for gravitating around such a framework.
One Potential Objection
One of the biggest arguments in my mind against this framework is the heuristic-level objection: “If this is a good idea that could have been adopted for a while, why hasn’t it been adopted more widely?”
I have a variety of thoughts in response, but to be honest I don’t have a specific, short answer that I feel very confident about, so I’ll just briefly list a few thoughts in this footnote.
Regardless, I would love to hear other people’s feedback/objections!
Ultimately, I think that it could help EA discourse and subsequent decision-making if more people popularized/used this framework, similar to how EAs tend to think the ITN framework has been helpful, but I would love to hear other people’s feedback, including objections. If you would like to read a less-neat and slightly outdated but longer explanation of this framework, including example applications and clearer definitions, you can see this post.
This is liable to change, especially if I can find better labels for “implementation” (which is actually a category that involves questions related to feasibility and plan details—e.g., the presence/absence of grandfather clauses—in addition to “implementation”) and “linkage” (which is basically trying to say “X outcome occurs in the world with Y implementation” but without claiming “Y causes X,” which is a derived statement from the combination of counterfactuality and “linkage”).
E.g., “this regulation would save 100 lives by reducing certain pollutants.”
A caveat/clarification: this does not include redirection responses that do not challenge individual pros/cons, such as “it’s true that your plan would cause X and that is as good as you say it is, but your plan also causes W which is bad enough to outweigh the claimed benefits of X.”
I’ve rarely gotten feedback on the framework from people other than policy debaters, and even then I have rarely ever received criticism—and the criticism I did receive I largely addressed years ago. Thus, most of my refining of the framework has been driven by internal criticism, which I acknowledge might just be missing something.
Yes, technically I could have analyzed “use this framework” according to the framework itself, but for length reasons and given that I don’t expect the audience to already be familiar with using the framework, I chose not to. Also, “use this framework” is a somewhat ambiguous proposal, which can make formal application of this framework more difficult (although it definitely does not prevent informal/heuristic-level application of this framework, much like how people often heuristically use the ITN framework.)
I.e., there is the potential for double-counting between the two categories, and even to some extent within the “heuristic-level justifications” category.
To be clear, stock issues theory is a bit of a mess given how there are different interpretations—partially stemming from the variation in leagues/regions—and debaters will occasionally sacrifice formality and precision for simplicity in rounds, but the overall idea is fairly widespread, especially among my former league.
I won’t go much into detail on many of the potential arguments against this framework for length purposes and given that I have previously written a short section on that subject. Most of the more-impactful objections to the framework, though, I think would stem from claiming that the framework is inaccurate in some way.
1. People could have used/formalized the ITN framework for a long time, but I think time has suggested that it is useful in the EA community.
2. Aside from the stock issues in policy debate, this framework probably already does exist but with different names and other partial variations.
3. This could require a critical mass of adoption in a community in order to be worth learning.