Fair point re: uncertainty. The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s, so both the Shapley value and counterfactuals have that problem (?). Maybe one can exclude agents acording to how close their goals are to yours, e.g., totally exclude a paperclip maximizer from both counterfactual and Shapley value calculations, and apply order indifference to allies only (?). This is something I haven’t though about; thanks for pointing it out.
Fair point re: epistemic status. Changed my epistemic status.
“The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s”—true, but the NGO’s counterfactual impact is reduced when I feel it’s fairer for the NGO to be able to claim the full amount (though of course you’d never know the government’s true motivations in real life)
Fair point re: uncertainty. The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s, so both the Shapley value and counterfactuals have that problem (?). Maybe one can exclude agents acording to how close their goals are to yours, e.g., totally exclude a paperclip maximizer from both counterfactual and Shapley value calculations, and apply order indifference to allies only (?). This is something I haven’t though about; thanks for pointing it out.
Fair point re: epistemic status. Changed my epistemic status.
“The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s”—true, but the NGO’s counterfactual impact is reduced when I feel it’s fairer for the NGO to be able to claim the full amount (though of course you’d never know the government’s true motivations in real life)