Hereâs a puzzle Iâve thought about a few times recently:
The impact of an activity (I) is due to two factors, X and Y. Those factors combine multiplicatively to produce impact. Examples include:
The funding of an organization and the people working at the org
A manager of a team who acts as a lever on the work of their reports
The EA Forum acts as a lever on top of the efforts of the authors
A product manager joins a team of engineers
Letâs assume in all of these scenarios that you are only one of the players in the situation, and you can only control your own actions.
From a counterfactual analysis, if you can increase your contribution by 10%, then you increase the impact by 10%, end of story.
From a Shapley Value perspective, itâs a bit more complicated, but we can start with a prior that you split your impact evenly with the other players.
Both these perspectives have a lot going for them! The counterfactual analysis has important correspondences to reality. If you do 10% better at your job the world gets 0.1I better. Shapley Values prevent the scenario where the multiplicative impact causes the involved agents to collectively contribute too much.
I notice myself feeling relatively more philosophically comfortable running with the Shapely Value analysis in the scenario where I feel aligned with the other players in the game. And potentially the Shapley Value approach downsides go down if I actually run the math (Fake edit: I ran a really hacky guess as to how Iâd calculate this using this calculator and it wasnât that helpful).
But I donât feel 100% bought-in to the Shapley Value approach, and think thereâs a value in paying attention to the counterfactuals. My unprincipled compromise approach would be to take some weighted geometric mean and call it a day.
Iâm annoyed at vague âvalueâ questions. If you ask a specific question the puzzle dissolves. What should you do to make the world go better? Maximize world-EV, or equivalently maximize your counterfactual value (not in the maximally-naive way â take into account how âyour actionsâ affect âothersâ actionsâ). How should we distribute a fixed amount of credit or a prize between contributors? Something more Shapley-flavored, although this isnât really the question that Shapley answers (and that question is almost never relevant, in my possibly controversial opinion).
Happy to talk about well-specified questions. Annoyed at questions like âshould I use counterfactuals hereâ that donât answer the obvious reply, âuse them FOR WHAT?â
I donât feel 100% bought-in to the Shapley Value approach, and think thereâs a value in paying attention to the counterfactuals. My unprincipled compromise approach would be to take some weighted geometric mean and call it a day.
FOR WHAT?
Letâs assume in all of these scenarios that you are only one of the players in the situation, and you can only control your own actions.
If this is your specification (implicitly /â further specification: youâre an altruist trying to maximize total value, deciding how to trade off between increasing X and doing good in other ways) then there is a correct answer â maximize counterfactual value (this is equivalent to maximizing total value, or argmaxing total value over your possible actions), not your personal Shapley value or anything else. (Just like in all other scenarios. Multiplicative-ness is irrelevant. Maximizing counterfactual value is always the answer to questions about what action to take.)
I think youâre right to be more uncomfortable with the counterfactual analysis in cases where youâre aligned with the other players in the game. Cribbing from a comment Iâve made on this topic before on the forum:
I think that counterfactual analysis is the right approach to take on the first point if/âwhen you have full information about whatâs going on. But in practice you essentially never have proper information on what everyone elseâs counterfactuals would look like according to different actions you could take.
If everyone thinks in terms of something like âapproximate shares of moral creditâ, then this can help in coordinating to avoid situations where a lot of people work on a project because it seems worth it on marginal impact, but it would have been better if theyâd all done something different. Doing this properly might mean impact markets (where the âmarketâ part works as a mechanism for distributing cognition, so that each market participant is responsible for thinking through their own alternative options, and feeding that information into the system via their willingness to do work for different amounts of pay), but I think that you can get some rough approximation to the benefits of impact markets without actual markets by having people do the things they would have done with marketsâand in this context, that means paying attention to the share of credit different parties would get.
Shapley values are one way to divide up that credit. They have some theoretical appeal, but itâs basically as âwhat would a fair division of credit be, which divides the surplus compared to outside optionsâ. And theyâre extremely complex to calculate so in practice Iâd recommend against even trying. Instead just think of it as an approximate bargaining solution between the parties, and use some other approximation to bargaining solutionsâI think Austinâs practice of looking to the business world for guidance is a reasonable approach here.
(If thereâs nobody whom youâre plausibly coordinating with then I think trying to do a rough counterfactual analysis is reasonable, but that doesnât feel true of any of your examples.)
So, as a self-professed mechanism geek, I feel like the Shapley Value stuff should be my cup of tea, but I must confess Iâve never wrapped my head around it. Iâve read Nunoâs post and played with the calculator, but still have little intuitive sense of how these things work even with toy examples, and definitely no idea on how they can be applied in real-world settings.
I think delineating impact assignment for shared projects is important, though I generally look to the business world for inspiration on the most battle-tested versions of impact assignment (equity, commissions, advertising fees, etc). Startup/âtech company equity & compensation, for example, at least provides a clear answer to âhow much does the employer value your workâ. The answer is suboptimal in many ways (eg my guess is startups by default assign too much equity to the founders), but at least it provides a simple starting point; better to make up numbers and all that.
Rationally and ideally, you should just maximize the expected value of your actions, taking into account your potential influence on others and their costs, including opportunity costs. This just follows assuming expected utility theory axioms. It doesnât matter that there are other agents; you can just capture them as part of your outcomes under consideration.
When youâre assigning credit across other actors whose impacts arenât roughly independent, including for estimating their cost-effectiveness for funding, Shapley values (or something similar) can be useful. You want assigned credit to sum to 100% to avoid double counting or undercounting. (Credit for some actors can even be negative, though.)
But, if you were going to calculate Shapley values, which means estimating a bunch of subgroup counterfactuals that didnât or wouldnât happen, anyway, you may be able to just directly estimate how to best allocate resources instead. You could skip credit assignments (EDIT especially ex ante credit assignments, or when future work will be similar to past work in effects).
I think you can get closer to dissolving this problem by considering why youâre assigning credit. Often, weâre assigning some kind of finite financial rewards.
Imagine that a group of n people have all jointly created $1 of value in the world, and that if any one of them did not participate, there would only be $0 units of value. Clearly, we canât give $1 to all of them, because then we would be paying $n to reward an event that only created $0 of value, which is inefficient. If, however, only the first guy (i=1) is an âagentâ that responds to incentives, while the others (1<=i<=n) are âenvironmentâ whose behaviour is unresponsive to incentives, then it is fine to give the first guy a reward of $1.
This is how you can ground the idea that agents who cooperate should share their praise (something like a Shapley Value approach), whereas rival agents who donât buy into your reward scheme should be left out of the shapley calculation.
Iâll give general takes in another comment, but I just wanted to call out how I think that at least for some of your examples the assumptions are unrealistic (and this can make the puzzle sound worse than it is).
Take the case of âThe funding of an organization and the people working at the orgâ. In this case the must factors combine in a sub-multiplicative way rather than a multiplicative way. For itâs clear that if you double the funding and double the people working at the org you should approximately double the output (rather than quadruple it). I think that Cobb-Douglas production functions are often a useful modelling tool here.
In the case of managers or the Forum I suspect that itâs also not quite multiplicativeâbut a bit closer to it. In any case I do think that after accounting for this thereâs still a puzzle about how to evaluate it.
Hereâs a puzzle Iâve thought about a few times recently:
The impact of an activity (I) is due to two factors, X and Y. Those factors combine multiplicatively to produce impact. Examples include:
The funding of an organization and the people working at the org
A manager of a team who acts as a lever on the work of their reports
The EA Forum acts as a lever on top of the efforts of the authors
A product manager joins a team of engineers
Letâs assume in all of these scenarios that you are only one of the players in the situation, and you can only control your own actions.
From a counterfactual analysis, if you can increase your contribution by 10%, then you increase the impact by 10%, end of story.
From a Shapley Value perspective, itâs a bit more complicated, but we can start with a prior that you split your impact evenly with the other players.
Both these perspectives have a lot going for them! The counterfactual analysis has important correspondences to reality. If you do 10% better at your job the world gets 0.1I better. Shapley Values prevent the scenario where the multiplicative impact causes the involved agents to collectively contribute too much.
I notice myself feeling relatively more philosophically comfortable running with the Shapely Value analysis in the scenario where I feel aligned with the other players in the game. And potentially the Shapley Value approach downsides go down if I actually run the math (Fake edit: I ran a really hacky guess as to how Iâd calculate this using this calculator and it wasnât that helpful).
But I donât feel 100% bought-in to the Shapley Value approach, and think thereâs a value in paying attention to the counterfactuals. My unprincipled compromise approach would be to take some weighted geometric mean and call it a day.
Interested in comments.
Iâm annoyed at vague âvalueâ questions. If you ask a specific question the puzzle dissolves. What should you do to make the world go better? Maximize world-EV, or equivalently maximize your counterfactual value (not in the maximally-naive way â take into account how âyour actionsâ affect âothersâ actionsâ). How should we distribute a fixed amount of credit or a prize between contributors? Something more Shapley-flavored, although this isnât really the question that Shapley answers (and that question is almost never relevant, in my possibly controversial opinion).
Happy to talk about well-specified questions. Annoyed at questions like âshould I use counterfactuals hereâ that donât answer the obvious reply, âuse them FOR WHAT?â
FOR WHAT?
If this is your specification (implicitly /â further specification: youâre an altruist trying to maximize total value, deciding how to trade off between increasing X and doing good in other ways) then there is a correct answer â maximize counterfactual value (this is equivalent to maximizing total value, or argmaxing total value over your possible actions), not your personal Shapley value or anything else. (Just like in all other scenarios. Multiplicative-ness is irrelevant. Maximizing counterfactual value is always the answer to questions about what action to take.)
I think youâre right to be more uncomfortable with the counterfactual analysis in cases where youâre aligned with the other players in the game. Cribbing from a comment Iâve made on this topic before on the forum:
I think that counterfactual analysis is the right approach to take on the first point if/âwhen you have full information about whatâs going on. But in practice you essentially never have proper information on what everyone elseâs counterfactuals would look like according to different actions you could take.
If everyone thinks in terms of something like âapproximate shares of moral creditâ, then this can help in coordinating to avoid situations where a lot of people work on a project because it seems worth it on marginal impact, but it would have been better if theyâd all done something different. Doing this properly might mean impact markets (where the âmarketâ part works as a mechanism for distributing cognition, so that each market participant is responsible for thinking through their own alternative options, and feeding that information into the system via their willingness to do work for different amounts of pay), but I think that you can get some rough approximation to the benefits of impact markets without actual markets by having people do the things they would have done with marketsâand in this context, that means paying attention to the share of credit different parties would get.
Shapley values are one way to divide up that credit. They have some theoretical appeal, but itâs basically as âwhat would a fair division of credit be, which divides the surplus compared to outside optionsâ. And theyâre extremely complex to calculate so in practice Iâd recommend against even trying. Instead just think of it as an approximate bargaining solution between the parties, and use some other approximation to bargaining solutionsâI think Austinâs practice of looking to the business world for guidance is a reasonable approach here.
(If thereâs nobody whom youâre plausibly coordinating with then I think trying to do a rough counterfactual analysis is reasonable, but that doesnât feel true of any of your examples.)
So, as a self-professed mechanism geek, I feel like the Shapley Value stuff should be my cup of tea, but I must confess Iâve never wrapped my head around it. Iâve read Nunoâs post and played with the calculator, but still have little intuitive sense of how these things work even with toy examples, and definitely no idea on how they can be applied in real-world settings.
I think delineating impact assignment for shared projects is important, though I generally look to the business world for inspiration on the most battle-tested versions of impact assignment (equity, commissions, advertising fees, etc). Startup/âtech company equity & compensation, for example, at least provides a clear answer to âhow much does the employer value your workâ. The answer is suboptimal in many ways (eg my guess is startups by default assign too much equity to the founders), but at least it provides a simple starting point; better to make up numbers and all that.
Rationally and ideally, you should just maximize the expected value of your actions, taking into account your potential influence on others and their costs, including opportunity costs. This just follows assuming expected utility theory axioms. It doesnât matter that there are other agents; you can just capture them as part of your outcomes under consideration.
When youâre assigning credit across other actors whose impacts arenât roughly independent, including for estimating their cost-effectiveness for funding, Shapley values (or something similar) can be useful. You want assigned credit to sum to 100% to avoid double counting or undercounting. (Credit for some actors can even be negative, though.)
But, if you were going to calculate Shapley values, which means estimating a bunch of subgroup counterfactuals that didnât or wouldnât happen, anyway, you may be able to just directly estimate how to best allocate resources instead. You could skip credit assignments (EDIT especially ex ante credit assignments, or when future work will be similar to past work in effects).
(I endorse this.)
FYI thanks for all the helpful comments here â I promptly got covid and havenât had a chance to respond đ
I think you can get closer to dissolving this problem by considering why youâre assigning credit. Often, weâre assigning some kind of finite financial rewards.
Imagine that a group of n people have all jointly created $1 of value in the world, and that if any one of them did not participate, there would only be $0 units of value. Clearly, we canât give $1 to all of them, because then we would be paying $n to reward an event that only created $0 of value, which is inefficient. If, however, only the first guy (i=1) is an âagentâ that responds to incentives, while the others (1<=i<=n) are âenvironmentâ whose behaviour is unresponsive to incentives, then it is fine to give the first guy a reward of $1.
This is how you can ground the idea that agents who cooperate should share their praise (something like a Shapley Value approach), whereas rival agents who donât buy into your reward scheme should be left out of the shapley calculation.
Iâll give general takes in another comment, but I just wanted to call out how I think that at least for some of your examples the assumptions are unrealistic (and this can make the puzzle sound worse than it is).
Take the case of âThe funding of an organization and the people working at the orgâ. In this case the must factors combine in a sub-multiplicative way rather than a multiplicative way. For itâs clear that if you double the funding and double the people working at the org you should approximately double the output (rather than quadruple it). I think that Cobb-Douglas production functions are often a useful modelling tool here.
In the case of managers or the Forum I suspect that itâs also not quite multiplicativeâbut a bit closer to it. In any case I do think that after accounting for this thereâs still a puzzle about how to evaluate it.