Slightly conflicted agree vote: your model here offloads so much to judgment calls that fall on people who are vulnerable to perverse incentives (like, alignment/capabilities as a binary distinction is a bad frame, but it seems like anyone who’d be unusually well suited to thinking clearly about it’s alternatives make more money and have less stressful lives if their beliefs fall some ways vs others).
Other than that, I’m aware that no one’s really happy about the way they tradeoff “you could copenhagen ethics your way out of literally any action in the limit” against “saying that the counterfactual a-hole would do it worse if I didn’t is not a good argument”. It seems like a law of opposite advice situation, maybe? As in some people in the blase / unilateral / powerhungry camp could stand to be nudged one way and some people in the scrupulous camp could stand to be nudged another.
It also matters that the “oppose carbon capture or nuclear energy because it might make people feel better without solving the ‘real problem’.” environmentalists have very low standards even when you condition on them being environmentalists. That doesn’t mean they can’t be memetically adaptive and then influential, but it might be tactically important (i.e. you have a messaging problem instead of a more virtuous actually-trying-to-think-clearly problem)
Slightly conflicted agree vote: your model here offloads so much to judgment calls that fall on people who are vulnerable to perverse incentives (like, alignment/capabilities as a binary distinction is a bad frame, but it seems like anyone who’d be unusually well suited to thinking clearly about it’s alternatives make more money and have less stressful lives if their beliefs fall some ways vs others).
Other than that, I’m aware that no one’s really happy about the way they tradeoff “you could copenhagen ethics your way out of literally any action in the limit” against “saying that the counterfactual a-hole would do it worse if I didn’t is not a good argument”. It seems like a law of opposite advice situation, maybe? As in some people in the blase / unilateral / powerhungry camp could stand to be nudged one way and some people in the scrupulous camp could stand to be nudged another.
It also matters that the “oppose carbon capture or nuclear energy because it might make people feel better without solving the ‘real problem’.” environmentalists have very low standards even when you condition on them being environmentalists. That doesn’t mean they can’t be memetically adaptive and then influential, but it might be tactically important (i.e. you have a messaging problem instead of a more virtuous actually-trying-to-think-clearly problem)