I agree with what you said and I am concerned and genuinely worried because I interpret your post as expressing sincere concerns of yours and view your posts highly and update.
At the same time, I have different models of the underlying issue and these have different predictions.
Basically, have you considered the perspective that “some EA orgs aren’t very good” to be a better explanation for the problems?
This model/perspective has very different predictions and remedies, and some of your remedies make it worse.
What does it mean to be “not motivated” or “unbiased”?
I can’t think of any strong, successful movement where there isn’t “motivated” reasoning.
Ioften literally say that “I am biased towards [X]” and “my ideology/aesthetics [say this]”.
That is acceptable because that’s the truth.
As far as I can tell, that is how all people, including very skilled and extraordinary people/leaders reason. Ideally (often?) it turns out the “bias” is “zero” or at least, the “leaders are right”.
I rapidly change my biases and ideology/aesthetics (or at least I think I do) when updated.
In my model, for the biggest decisions, people rarely spend effort to be “unbiased” or “unmotivated”.
It’s more like, what’s the plan/vision/outcomes that I will see fulfilled with my “motivated reasoning”? How will this achieve impact?
Impractical to fix things by “adding CEA” or undergirding orgs with dissent and positivism
My models of empiricism says it’s hard to execute CEAs well. There isn’t some CEA template/process that we can just apply reliably. Even the best CEA or RCTs involve qualitative decisions that have methods/worldviews embedded. Think of the Science/AER papers that you have seen fall apart in your hands.
Also, in my model, one important situation is that sometimes leaders and organizations specifically need this “motivated” reasoning.
This is because, in some sense, all great new initiatives are going to lack a clear cost effectiveness formula. It takes a lot of “activation energy” to get a new intervention or cause area going.
Extraordinary leaders are going to have perspectives and make decisions with models/insights that are difficult to verify, and sometimes difficult to even conceptualize.
CEA or dissent isn’t going to be helpful in these situations.
Promoting empiricism or dissent may forestall great initiatives and may create environments where mediocre empiricism supports mediocre leadership.
It seems like we should expect the supply of high quality CEA or dissent to be as limited as good leadership.
I interpret your examples as evidence for my model:
Back in the 90’s I did some consulting work for a startup that was developing a new medical device...Peer review did not discover any of this during the publication process, because each individual estimate was reasonable. When I wrote the paper, I was not in the least bit aware that I was doing this; I truly thought I was being “objective.”
How would we fix the above, besides “getting good”?
As another example, ALLFED may have gotten dinged in a way that demonstrates my concern:
It seems likely that the underlying issues that undermine success on the object level would also make “meta” processes just as hard to execute, or worse.
As mentioned at the top, this isn’t absolving or fixing any problems you mentioned. Again, I share the underlying concerns and also update to you.
Maybe an alternative? Acknowledge these flaws?
A sketch of a solution might be:
1) Choose good leaders and people 2) Have knowledge of the “institutional space” being occupied by organizations, and have extraordinarily high standards for those that can govern/interact/filter the community 3) Allow distinct, separate cause areas and interventions to flourish and expect some will fail
This is just a sketch and there’s issues (how do you adequately shutdown and fairly compensate interests who fail, because non-profits and especially meta-orgs often perpetuate their own interests, for good reasons. We can’t really create an “executioner org” or rely on orgs getting immolated on the EA forum).
I think the value of this sketch is that it draws attention to the institutional space occupied by orgs and how it affects the community.
I think what you said is insightful and worth considering further. Nonetheless, I will only address a specific subpoint for now, and revisit this later.
Basically, have you considered the perspective that “some EA orgs aren’t very good” to be a better explanation for the problems?
Hmm I’m not sure what you mean, and I think it’s very likely we’re talking about different problems. But assuming we’re talking about the same problems, at a high-level any prediction problem can be decomposed to bias vs error (aka noise, aka variance).
I perceive that many of the issues I’ve mentioned to be better explained by bias than error. In particular I just don’t think we’ll see equivalently many errors in the opposite direction. This is an empirical question however, and I’d be excited to see more careful followups to test this hypothesis.
(as a separate point, I do think some EA orgs aren’t very good, with “very good” defined as I’d rather the $s be spent on their work rather than in Open Phil coffers, or my own bank account. I imagine many other EAs would feel similarly about my own work).
Thank you for your thoughtful reply. I think you are generous here:
I perceive that many of the issues I’ve mentioned to be better explained by bias than error. In particular I just don’t think we’ll see equivalently many errors in the opposite direction. This is an empirical question however, and I’d be excited to see more careful followups to test this hypothesis.
I think you are pointing out that, when I said I think I have many biases and these are inevitable, that I am confusing bias with error.
What you are pointing out seems right to me.
Now, at the very least, this undermines my comment (and at the worst suggests I am promoting/suffering from some other form of arrogance). I’m less confident about my comment now. I think now I will reread and think about your post a lot more.
Hi. I’m glad you appear to have gained a lot from my quick reply, but for what it’s worth I did not intend my reply as an admonishment.
I think the core of what I read as your comment is probably still valid. Namely, that if I misidentified problems as biases when almost all of the failures are due to either a) noise/error or b) incompetence unrelated to decision quality (eg mental health, insufficient technical skills, we aren’t hardworking enough), then the bias identification isn’t true or useful. Likewise, debiasing is somewhere between neutral to worse than useless if the problem was never bias to begin with.
I agree with what you said and I am concerned and genuinely worried because I interpret your post as expressing sincere concerns of yours and view your posts highly and update.
At the same time, I have different models of the underlying issue and these have different predictions.
Basically, have you considered the perspective that “some EA orgs aren’t very good” to be a better explanation for the problems?
This model/perspective has very different predictions and remedies, and some of your remedies make it worse.
What does it mean to be “not motivated” or “unbiased”?
I can’t think of any strong, successful movement where there isn’t “motivated” reasoning.
I often literally say that “I am biased towards [X]” and “my ideology/aesthetics [say this]”.
That is acceptable because that’s the truth.
As far as I can tell, that is how all people, including very skilled and extraordinary people/leaders reason. Ideally (often?) it turns out the “bias” is “zero” or at least, the “leaders are right”.
I rapidly change my biases and ideology/aesthetics (or at least I think I do) when updated.
In my model, for the biggest decisions, people rarely spend effort to be “unbiased” or “unmotivated”.
It’s more like, what’s the plan/vision/outcomes that I will see fulfilled with my “motivated reasoning”? How will this achieve impact?
Impractical to fix things by “adding CEA” or undergirding orgs with dissent and positivism
My models of empiricism says it’s hard to execute CEAs well. There isn’t some CEA template/process that we can just apply reliably. Even the best CEA or RCTs involve qualitative decisions that have methods/worldviews embedded. Think of the Science/AER papers that you have seen fall apart in your hands.
Also, in my model, one important situation is that sometimes leaders and organizations specifically need this “motivated” reasoning.
This is because, in some sense, all great new initiatives are going to lack a clear cost effectiveness formula. It takes a lot of “activation energy” to get a new intervention or cause area going.
Extraordinary leaders are going to have perspectives and make decisions with models/insights that are difficult to verify, and sometimes difficult to even conceptualize.
CEA or dissent isn’t going to be helpful in these situations.
Promoting empiricism or dissent may forestall great initiatives and may create environments where mediocre empiricism supports mediocre leadership.
It seems like we should expect the supply of high quality CEA or dissent to be as limited as good leadership.
I interpret your examples as evidence for my model:
How would we fix the above, besides “getting good”?
As another example, ALLFED may have gotten dinged in a way that demonstrates my concern:
It seems likely that the underlying issues that undermine success on the object level would also make “meta” processes just as hard to execute, or worse.
As mentioned at the top, this isn’t absolving or fixing any problems you mentioned. Again, I share the underlying concerns and also update to you.
Maybe an alternative? Acknowledge these flaws?
A sketch of a solution might be:
1) Choose good leaders and people
2) Have knowledge of the “institutional space” being occupied by organizations, and have extraordinarily high standards for those that can govern/interact/filter the community
3) Allow distinct, separate cause areas and interventions to flourish and expect some will fail
This is just a sketch and there’s issues (how do you adequately shutdown and fairly compensate interests who fail, because non-profits and especially meta-orgs often perpetuate their own interests, for good reasons. We can’t really create an “executioner org” or rely on orgs getting immolated on the EA forum).
I think the value of this sketch is that it draws attention to the institutional space occupied by orgs and how it affects the community.
I think what you said is insightful and worth considering further. Nonetheless, I will only address a specific subpoint for now, and revisit this later.
Hmm I’m not sure what you mean, and I think it’s very likely we’re talking about different problems. But assuming we’re talking about the same problems, at a high-level any prediction problem can be decomposed to bias vs error (aka noise, aka variance).
I perceive that many of the issues I’ve mentioned to be better explained by bias than error. In particular I just don’t think we’ll see equivalently many errors in the opposite direction. This is an empirical question however, and I’d be excited to see more careful followups to test this hypothesis.
(as a separate point, I do think some EA orgs aren’t very good, with “very good” defined as I’d rather the $s be spent on their work rather than in Open Phil coffers, or my own bank account. I imagine many other EAs would feel similarly about my own work).
Hi,
Thank you for your thoughtful reply. I think you are generous here:
I think you are pointing out that, when I said I think I have many biases and these are inevitable, that I am confusing bias with error.
What you are pointing out seems right to me.
Now, at the very least, this undermines my comment (and at the worst suggests I am promoting/suffering from some other form of arrogance). I’m less confident about my comment now. I think now I will reread and think about your post a lot more.
Thanks again.
Hi. I’m glad you appear to have gained a lot from my quick reply, but for what it’s worth I did not intend my reply as an admonishment.
I think the core of what I read as your comment is probably still valid. Namely, that if I misidentified problems as biases when almost all of the failures are due to either a) noise/error or b) incompetence unrelated to decision quality (eg mental health, insufficient technical skills, we aren’t hardworking enough), then the bias identification isn’t true or useful. Likewise, debiasing is somewhere between neutral to worse than useless if the problem was never bias to begin with.