Developing Counterfactual Trust in Moral Trade

Introduction

This post is a continuation of my earlier “Modeling Moral Trade in Antibiotic Resistance and Alternative Proteins” and “Generating More Surplus in Moral Trades.” Here I’ll discuss ways to develop counterfactual trust in moral trade.

Moral trade requires two trust-based components: factual trust and counterfactual trust. Factual trust simply means that all actors believe that the others will do what they agree to do. In most cases, this is pretty simple, and may be established through traditional means.

Counterfactual trust means that both actors believe that the others would have acted in a certain way if not for the trade. Take, for example, the moral trade I described in my last post: Open Philanthropy’s animal welfare (AW) and global health and development (GH&D) teams agree to fund a campaign to get JBS to use slower growing chickens, improving animal welfare and slowing antibiotic resistance. GH&D needs to trust AW’s assertion that, without trade, they would have funded a campaign against Purdue, and AW needs to trust GH&D’s claim that they would have funded GiveWell’s top recommendation.

Basic Tools

This sort of trust is much more difficult to build. The cornerstone is standard in any relationship: personal trust of the individuals involved in the exchange. While it’s difficult for any individual to perfectly determine even their own counterfactual action, especially over long time periods, honestly sharing best guesses and personal beliefs will help the group determine the true counterfactual.

A historical track record can also be useful for determining someone’s counterfactual action. If they’ve been donating to a specific charity or set of charities for a long time, even without trust, a likely counterfactual can be established. A track record of systematic decision making is also useful, even if the outputs of the system are different over time.

The third component is a clear, systematic, and public process for determining cost-effectiveness. If different agents can accept that others have different philosophical components to their worldview, but similar epistemics, they may be able to agree about the cost-effectiveness of different interventions based on different philosophical assumptions. Even if the epistemics are different, trust that the epistemic views are honestly held and consistently followed may well be sufficient for trade to occur.

Partly “locking in” this process is valuable. However, it’s important to leave room for adjustment in case there’s a flaw in the process. There may be inconsistency in your preferences that leaves you vulnerable to dutch booking. Parameters may also be updated in ways you don’t expect and cause the model to not fully reflect your understanding of the interventions.

There are perverse incentives in the creation of the cost-effectiveness analysis. Each worldview is incentivized to rate the alternative intervention as low as possible in order to decrease the amount they have to pay to get the intervention funded. Honest analyses will help trading partners find terms that are agreeable to all of them, since traders will not want to give extra surplus to other worldviews and may instead allocate all funds to their top intervention. This practice may lead to short-term losses, but long-term gains, once agents with the other worldview become more reasonable with their CEAs.

Weirder Ideas

An external evaluator could provide a service to determine a counterfactual allocation. This technique is used in California’s program to verify offsets for their cap and trade system. Evaluators project a counterfactual level of emissions, then measure true emissions with the project, and finally find the difference between the two to determine the counterfactual impact. While there are notable differences between these two scenarios, the technical guidance from the California Air Resources Board for evaluators may prove useful for similar actors in the context of moral trade. GiveWell also evaluates whether matching donors are giving counterfactually.

Prediction markets could assist evaluators or be used on their own to determine the expected counterfactual. Unfortunately, the size of these markets and relative lack of reasoning transparency from grantmakers might be insufficient for good prediction. However, for a large moral trade with open information this suggestion could work.

Over time, a proper market could emerge in which actors offer different funding percentages at which to trade. The highest offers of one worldview would match with the highest offers from another worldview, thereby maximizing the moral surplus generated. Having known actors with trusted counterfactuals would be essential for making this market work, but public information on potential traders’ profiles would help encourage that.

Effect of Relative Cost-Effectiveness Estimates and Expected Surplus

Relative cost-effectiveness estimates also affect the level and importance of counterfactual trust.

Estimated Cost-Effectiveness of Alternate Intervention	Effect on Counterfactual Trust
Low (0-40%)	Low cost-effectiveness makes it very easy to establish counterfactual trust.
Medium (41-80%)	Medium cost-effectiveness makes it relatively easy to establish counterfactual trust.
High (81-100%)	High cost-effectiveness makes it hard to establish counterfactual trust, since it’s unclear if the intervention would have been funded anyway.

In the chart I assume that both worldviews have equal estimates, but in reality, people with different worldviews will likely have different ones. The magnitude of these differences will be an important factor in determining counterfactual trust. For example, if people with worldview A estimate the relative cost-effectiveness of the alternative intervention according to worldview A to be 40%, but people with worldview B estimate the value according to worldview A to be 85%, people with worldview B may require greater evidence that the intervention wouldn’t have been counterfactually funded and that people with worldview A are giving their honest estimate.

Possible levels of surplus also have an effect on negotiation.

Possible Level of Surplus	Effect on Negotiation
Low (0-10%)	Parties must agree very closely about relative cost-effectiveness estimates to find surplus-generating terms.
Medium (11-25%)	Parties must agree somewhat closely about relative cost-effectiveness estimates to find surplus-generating terms.
High (26+%)	Parties do not need to agree very closely about cost-effectiveness estimates to find surplus-generating terms.

As before, I assumed that both worldviews have similar estimates of possible levels of surplus. In reality, this may not be the case, and differing estimates may affect negotiation differently. At any level of estimated surplus, parties may refuse to trade if they don’t view estimates as reasonable, since they would like a larger fraction of the surplus.

The percentages in this section are somewhat arbitrary best guesses. More analysis of real moral trades and grantmaker expertise could help determine more accurate numbers. They will also vary based on the typical variation of cost-effectiveness estimates in a given cause area (e.g. despite low levels of surplus, it may be relatively easy to make gains from trade in the global health and animal welfare spaces as compared to longtermist work, since there’s much better data in those areas, and cost-effectiveness estimates from different parties should be much closer.)

Here are some scenarios (listed percentages are the estimated cost-effectiveness of the alternate intervention according to each party’s philosophical beliefs):

6%/96%
- Low level of possibility for surplus means that both parties have to agree more closely about cost-effectiveness estimates
- High cost-effectiveness according to one view makes it hard to establish counterfactual trust, since the worldview assigning 6% relative cost-effectiveness to the alternate intervention may think that the other worldview might have funded it anyway, since 96% is so close to the bar
51%/51%
- Low level of possibility for surplus means that both parties have to agree more closely about cost-effectiveness estimates
- Because neither view places the alternate intervention close to the bar, both parties likely agree that the other would not have counterfactually funded it
40%/96%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- High cost-effectiveness according to one view makes it hard to establish counterfactual trust, since the worldview assigning 40% relative cost-effectiveness to the alternate intervention may think that the other worldview might have funded it anyway, since 96% is so close to the bar
96%/96%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- High cost-effectiveness according to both views makes it hard to establish counterfactual trust, since they both think the other worldview might have funded it anyway, since 96% is close to the bar
80%/80%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- Cost-effectiveness isn’t so high according to either worldview that each party doesn’t trust that the other would not have counterfactually funded it.

The key takeaway from the first part of the section is this: the lower the relative cost-effectiveness estimate, the easier it is to establish counterfactual trust, but the closer the parties have to agree on cost-effectiveness estimates of the alternative intervention according to each worldview in order to find a surplus-generating trade. More even cost-effectiveness estimates between worldviews also makes it easier to trade.

Because of this relationship, trades that involve more distinct worldviews may be more promising. Here are more scenarios, this time with three parties (listed percentages are the estimated cost-effectiveness of the alternate intervention according to each party’s philosophical beliefs):

6%/6%/96%
- Moderate possibility of surplus means that the parties must agree someone closely about the cost-effectiveness estimates
- High cost-effectiveness according to one view makes it hard to establish counterfactual trust, since the worldviews assigning 6% relative cost-effectiveness to the alternate intervention may think that the other worldview might have funded it anyway, since 96% is so close to the bar
51%/51%/51%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- Because no view places the alternate intervention close to the bar, all parties likely agree that the other would not have counterfactually funded it
40%/40%/96%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- High cost-effectiveness according to one view makes it hard to establish counterfactual trust, since the worldviews assigning 40% relative cost-effectiveness to the alternate intervention may think that the other worldview might have funded it anyway, since 96% is so close to the bar
96%/96%/96%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- High cost-effectiveness according to both views makes it hard to establish counterfactual trust, since they all think the other worldviews might have funded it anyway, since 96% is close to the bar
80%/80%/80%
- High level of possibility for surplus means that the parties don’t have to agree closely about the cost-effectiveness estimates
- Cost-effectiveness isn’t so high according to any worldview that each party doesn’t trust that the other would not have counterfactually funded it.

Ideal Conditions and How to Create Them

The ideal moral trade is intrapersonal; that is, a trade that one makes with themself. It’s usually easy to trust oneself, and they don’t have to determine specific funding percentages at which the trade will occur, since all the money is coming from the same pot.

Another good condition is for the agents to be part of the same organization, such as Open Philanthropy or EA Funds. Each organization has different teams with different goals, but their high levels of communication, visibility into processes, and common backgrounds and empirical strategies may facilitate trade. Things do get trickier though, since a specific value within a range of surplus-generating funding values must be selected.

If you intend to engage in moral trade in the future, here are some steps to take today that can help you prepare:

Begin investing in relationships with others that you might trade with in the future
Build a track record, including…
- Conversations with other potential trading partners
- Public writing
- Donations or volunteering based on the values and beliefs you shared
Practice reasoning transparency in your track record – share your values, beliefs, and how you arrived at them

Because these factors are so important in establishing counterfactual trust, there may be incentives toward dishonesty that will enable trades with better terms in the future. For example, if I’m an animal-focused person, I may want to rate alternative proteins work as relatively ineffective compared to chicken welfare campaigns, so that I get better terms when I trade with the climate-focused people to fund more alt proteins work. However, I don’t expect this effect to be significant. Developing a track record with dishonest beliefs or donations one considers worse than optimal is very costly, and the marginal benefit from a possible future moral trade with slightly better terms seems unlikely to compensate for these costs. This question merits more investigation – I haven’t run the numbers, and it may become a more important question in developed markets.

Thanks to Quentin Mot for feedback on this post.