This is in relation to the Keep EA high-trust idea, but it seemed tangential enough and butterfly idea-ish that it didn’t make sense to share this as a comment on that post.
Rough thoughts: focus a bit less on people and a bit more on systems. Some failures are ‘bad actors,’ but my rough impression is that far more often bad things happen because either:
the system/structures/incentives nudge people toward bad behavior, or
the system/structures/incentives allow bad behavior
I think it is great to be able to trust people, but I also want institutions designed in such a way that it is okay if someone is in the 70th percentile of trustworthiness rather than the 95th percentile of trustworthiness.
Low confidence guess: small failures often occur not because people are malicious or selfish, but because they aren’t aware of better ways to do things. An employee that isn’t aware of EEO in the United States is more likely to make costly mistakes. A manager who has not received good training on how to be a manager is going to fumble more often.
I don’t want to imply that designing systems well is easy, not that I am somehow an expert in it. But my (very) rough impression is that in EA we trust individuals a lot, and we don’t spend as much time thinking about organizational design.
This is in relation to the Keep EA high-trust idea, but it seemed tangential enough and butterfly idea-ish that it didn’t make sense to share this as a comment on that post.
Rough thoughts: focus a bit less on people and a bit more on systems. Some failures are ‘bad actors,’ but my rough impression is that far more often bad things happen because either:
the system/structures/incentives nudge people toward bad behavior, or
the system/structures/incentives allow bad behavior
It very much reminds me of “Good engineering eliminates users being able to do the wrong thing as much as possible. . . . You don’t design a feature that invites misuse and then use instructions to try to prevent that misuse.” I’ve also just learned about the hierarchy of hazard controls, which seems like a nice framework for thinking about ‘bad things.’
I think it is great to be able to trust people, but I also want institutions designed in such a way that it is okay if someone is in the 70th percentile of trustworthiness rather than the 95th percentile of trustworthiness.
Low confidence guess: small failures often occur not because people are malicious or selfish, but because they aren’t aware of better ways to do things. An employee that isn’t aware of EEO in the United States is more likely to make costly mistakes. A manager who has not received good training on how to be a manager is going to fumble more often.
I don’t want to imply that designing systems well is easy, not that I am somehow an expert in it. But my (very) rough impression is that in EA we trust individuals a lot, and we don’t spend as much time thinking about organizational design.