Ubuntu—yes, regarding the underestimated ‘allure of gossip, public shaming, witch hunts, etc’, I think the moral psychology at work in these things runs so deep that even the most rationalist & clever EAs can be prone to them—and then we can sometimes deceive ourselves about what’s really going on.
However, the moral psychology around public shaming evolved for some good adaptive reasons, to help deter bad actors, solve coordination problems, enforce social norms, virtue-signal our values, internalize self-control heuristics, etc. So I don’t think we should dismiss them entirely. (My 2019 book ‘Virtue Signaling’ addresses some of these issues.)
Indeed, leveraging the power of these ‘darker’ facets of moral psychology (e.g. public shaming) has arguably been crucial in many effective moral crusades throughout history, e.g. against torture, slavery, sexism, racism, nuclear brinksmanship, chemical weapons, etc. They may still prove useful in fighting against AI X-risk...
I think it would be problematic if a society heaped full adoration on risk-takers when their risks worked out, but doled out negative social consequences (which I’ll call “shame” to track your comment) only based on ex ante expected-value analysis when things went awry. That would overincentivize risktaking.
To maintain proper incentives, one could argue that society should map the amount of public shame/adoration to the expected value of the decision(s) made in cases like this, whether the risk works out or not. However, it would be both difficult and burdensome to figure out all the decisions someone made, assign an EV to each, and then sum to determine how much public shame or adoration the person should get.
By assigning shame or adoration primarily based on the observed outcome, society administers the shame/adoration incentives in a way that at least makes the EV of public shame/adoration at least somewhat related to the EV of the decision(s) made. Unfortunately, that approach means that people whose risks don’t pan out often end up with shame that may not be morally justified.
Ubuntu—yes, regarding the underestimated ‘allure of gossip, public shaming, witch hunts, etc’, I think the moral psychology at work in these things runs so deep that even the most rationalist & clever EAs can be prone to them—and then we can sometimes deceive ourselves about what’s really going on.
However, the moral psychology around public shaming evolved for some good adaptive reasons, to help deter bad actors, solve coordination problems, enforce social norms, virtue-signal our values, internalize self-control heuristics, etc. So I don’t think we should dismiss them entirely. (My 2019 book ‘Virtue Signaling’ addresses some of these issues.)
Indeed, leveraging the power of these ‘darker’ facets of moral psychology (e.g. public shaming) has arguably been crucial in many effective moral crusades throughout history, e.g. against torture, slavery, sexism, racism, nuclear brinksmanship, chemical weapons, etc. They may still prove useful in fighting against AI X-risk...
I think it would be problematic if a society heaped full adoration on risk-takers when their risks worked out, but doled out negative social consequences (which I’ll call “shame” to track your comment) only based on ex ante expected-value analysis when things went awry. That would overincentivize risktaking.
To maintain proper incentives, one could argue that society should map the amount of public shame/adoration to the expected value of the decision(s) made in cases like this, whether the risk works out or not. However, it would be both difficult and burdensome to figure out all the decisions someone made, assign an EV to each, and then sum to determine how much public shame or adoration the person should get.
By assigning shame or adoration primarily based on the observed outcome, society administers the shame/adoration incentives in a way that at least makes the EV of public shame/adoration at least somewhat related to the EV of the decision(s) made. Unfortunately, that approach means that people whose risks don’t pan out often end up with shame that may not be morally justified.
Yeah that’s a fair point to raise. I guess I’m just lamenting that these facets aren’t refined enough to catch less false positives by this point.