The mindset you describe seems like a big improvement compared to what I suspect is common among EAs (“naive trust”). However, it doesn’t sound entirely optimal to me either.
Sometimes it’s possible for people (with good people judgment)* to know specific other people well enough to confidently rule out various failure modes around prosociality like “this person certainly isn’t a serial killer” or “this person wouldn’t turn badly abusive even if lots of things go wrong.”**
Note that the emphasis here is on “sometimes.” The question isn’t just “do people with a low corruption threshold exist?” but also “can we identify them ex ante?” Identifying is hard, but sometimes we know someone well enough to do it. This holds especially if that person also makes themselves transparent (which greatly helps with trust-building).
Forming high-trust relationships with a few specific people (or even just one person) can be really valuable, so people might want to learn how to develop that sort of confidence even if they otherwise benefit from adopting the “no idols” mindset. (Note that it might be very rare to have the privilege to form close-enough relationships to make these assessments at all.)
I think the statement “power corrupts” is true, so with power-related failure modes, the world might be more grey (e.g., maybe there’s no person of whom we can tell that they’d never end up corrupted). Even so, you can sometimes say something like “this person seems about as good as it gets for this type of leadership.” That still makes a big difference!
In fact, I think focusing on “people and their traits” gives us more leverage than focusing on “situations” or “temptations/risk factors.” This is why, in organizational contexts, I think focusing on “checks and balances” is less important than choice of leadership. Apart from personality IMO being responsible for most of the variance around good/bad outcomes (no citation; that’s just my impression), I also think that “checks and balances” only control downside risk. They don’t really help much with cases where leadership subtly caps their upside potential for success at the mission in exchange for more easily legible (selfish) “benefits.” (Altruistic missions are much harder to verify than “make $ profits for shareholders,” which is a reason why EA orgs have it harder.) By contrast, good leadership seems like the only way to succeed with a mission where tracking progress is difficult and not really legible for people who aren’t deeply immersed in things.
*People who are good at “people judgment.” Some people are really bad at it, in which case they can only limit downsides.
**I say “badly abusive” because I accept that it can probably happen to even the best of people that they end up unhappy with themselves and let it out on others, to some degree. That said, I think there are people who would notice and care (because they derive their life satisfaction from being a good partner or parent) if they started to do this and who would have the strength of character to accept that they’re doing this (as opposed to going into denial about it and taking it out on the other person even more out of convoluted self-hatred). I expect that “awareness + caring + resistance to strong levels of self-deception” protect quite well against the worst relationship failure modes.
The mindset you describe seems like a big improvement compared to what I suspect is common among EAs (“naive trust”). However, it doesn’t sound entirely optimal to me either.
Sometimes it’s possible for people (with good people judgment)* to know specific other people well enough to confidently rule out various failure modes around prosociality like “this person certainly isn’t a serial killer” or “this person wouldn’t turn badly abusive even if lots of things go wrong.”**
Note that the emphasis here is on “sometimes.” The question isn’t just “do people with a low corruption threshold exist?” but also “can we identify them ex ante?” Identifying is hard, but sometimes we know someone well enough to do it. This holds especially if that person also makes themselves transparent (which greatly helps with trust-building).
Forming high-trust relationships with a few specific people (or even just one person) can be really valuable, so people might want to learn how to develop that sort of confidence even if they otherwise benefit from adopting the “no idols” mindset. (Note that it might be very rare to have the privilege to form close-enough relationships to make these assessments at all.)
I think the statement “power corrupts” is true, so with power-related failure modes, the world might be more grey (e.g., maybe there’s no person of whom we can tell that they’d never end up corrupted). Even so, you can sometimes say something like “this person seems about as good as it gets for this type of leadership.” That still makes a big difference!
In fact, I think focusing on “people and their traits” gives us more leverage than focusing on “situations” or “temptations/risk factors.” This is why, in organizational contexts, I think focusing on “checks and balances” is less important than choice of leadership. Apart from personality IMO being responsible for most of the variance around good/bad outcomes (no citation; that’s just my impression), I also think that “checks and balances” only control downside risk. They don’t really help much with cases where leadership subtly caps their upside potential for success at the mission in exchange for more easily legible (selfish) “benefits.” (Altruistic missions are much harder to verify than “make $ profits for shareholders,” which is a reason why EA orgs have it harder.) By contrast, good leadership seems like the only way to succeed with a mission where tracking progress is difficult and not really legible for people who aren’t deeply immersed in things.
*People who are good at “people judgment.” Some people are really bad at it, in which case they can only limit downsides.
**I say “badly abusive” because I accept that it can probably happen to even the best of people that they end up unhappy with themselves and let it out on others, to some degree. That said, I think there are people who would notice and care (because they derive their life satisfaction from being a good partner or parent) if they started to do this and who would have the strength of character to accept that they’re doing this (as opposed to going into denial about it and taking it out on the other person even more out of convoluted self-hatred). I expect that “awareness + caring + resistance to strong levels of self-deception” protect quite well against the worst relationship failure modes.