I think your first paragraph provides a potential answer to your second :-)
There’s an implicit “Sam fell prey to motivated reasoning, but I wouldn’t do that” in your comment, which itself seems like motivated reasoning :-)
(At least, it seems like motivated reasoning in the absence of a strong story for Sam being different from the rest of us. That’s why I’m so interested in what people like nbouscal have to say.)
So you think there’s too much danger of cutting yourself and everyone else via motivated reasoning, ala Dan Luu’s “Normalization of Deviance” and the principles have little room for errors in implementing them, is that right?
most human beings perceive themselves as good and decent people, such that they can understand many of their rule violations as entirely rational and ethically acceptable responses to problematic situations. They understand themselves to be doing nothing wrong, and will be outraged and often fiercely defend themselves when confronted with evidence to the contrary.
Specifically, I was saying that wrong results would come up if you failed in one of the steps of reasoning, and there’s no self-correction mechanism for bad reasoning like Sam Bankman-Fried was doing.
I think your first paragraph provides a potential answer to your second :-)
There’s an implicit “Sam fell prey to motivated reasoning, but I wouldn’t do that” in your comment, which itself seems like motivated reasoning :-)
(At least, it seems like motivated reasoning in the absence of a strong story for Sam being different from the rest of us. That’s why I’m so interested in what people like nbouscal have to say.)
So you think there’s too much danger of cutting yourself and everyone else via motivated reasoning, ala Dan Luu’s “Normalization of Deviance” and the principles have little room for errors in implementing them, is that right?
Here’s a link to it:
https://danluu.com/wat/
And a quote:
I’m not sure what you mean by “the principles have little room for errors in implementing them”.
That quote seems scarily plausible.
EDIT: Relevant Twitter thread
Specifically, I was saying that wrong results would come up if you failed in one of the steps of reasoning, and there’s no self-correction mechanism for bad reasoning like Sam Bankman-Fried was doing.