I like this criticism, but I think there are two essentially disjoint parts here that are being criticized. The first is excess legibility, i.e., the issue of having explicit metrics and optimizing to the metrics at all. The second is that a few of the measurements that determine how many resources a group gets/how quickly it grows are correlated with things that are not inherently valuable at best and harmful at worst.
The first problem seems really hard to me: the legibility/autonomy trade-off is an age-old problem that happens in politics, business, and science, and seems to involve a genuine trade-off between organizational efficiency and the ability to capitalize on good but unorthodox ideas and individuals.
The second seems more accessible (though still hard), and reasonably separable from the first. Here I see a couple of things you flag (other than legibility/”corporateness” by itself) as parameters that positively contribute to growth but negatively contribute to the ability of EA to attract intellectually autonomous people. The first is “fire-and-brimstone” style arguments, where EA outreach tends to be all-or-nothing, “you either help save the sick children or you burn in Utilitarian Hell”, and the second is common-denominator level messaging that is optimized to build community (so things like slogans, manufactured community and sense of purpose; things that attract the people like Bob in your thought experiment), but not optimized to appeal to meta-level thinkers who understand the reasoning behind the slogans. Both are vaguely correlated with EA having commonalities with religious communities, and so I’m going to borrow the adjective “pious” to refer to ideas and individuals for which these factors are salient.
I like that you are pointing out that a lot of EA outreach is, in one way or another, an “appeal to piety”, and this is possibly bad. There might be a debate about whether this is actually bad and to what extent (e.g., the Catholic church is inefficient, but the sheer volume of charity it generates is nothing to sneer at), but I think I agree with the intuition that this is suboptimal, and that by Goodhart’s law, if pious people are more likely to react to outreach, eventually they will form a supermajority.
I don’t want to devalue the criticism that legibility is in itself a problem, and particularly ugh-y to certain types of people (e.g. to smart humanities majors). But I think that the problem of piety can be solved without giving up on legibility, and instead by using better metrics, that have more entanglement with the real world. This is something I believed before this post, so I might be shoe-horning it here: take this with a grain of salt.
But I want to point out that organizations that are constantly evaluated on some measurable parameter don’t necessarily tend to end up excessively pious. A sports team can’t survive by having the best team spirit; a software company will not see any profit if it only hires people who fervently believe in its advertising slogans. So maybe a solution to the problem of appeals to piety is to, as you say, reduce the importance of the metric of “HEA” generation in determining funding, clout, etc., but replace it with other hard-to-fake metrics that are less correlated with piety and more correlated with actually being effective at what you do.
I haven’t thought much about what the best metrics would be and am probably not qualified to make recommendations, but I just for plausibility’s sake here are a couple of examples of things that I think would be cool (if not necessarily realistic):
First, it would be neat (though potentially expensive) if there were a yearly competition between teams of EA’s (maybe student groups, or maybe something on a larger level) to use a funding source to create an independent real-world project and have their impact in QALY’s judged by an impartial third party.
Second, I think it would be nice to make “intramural” forms of existing competitions, such as the fiction contest, Scott Alexander’s book review contest, various super-forecasting contests, etc., and grading University groups on success (relative to past results). If something like this is implemented, I’d also like to see the focus of things like the fiction competition move away from “good messaging” (which smacks of piety) and towards “good fiction that happens to have an EA component, if you look hard enough”.
I think that if the funding culture becomes more explicitly focused on concentration of talent and on real-world effects and less on sheer numbers or uncritical mission alignment, then outreach will follow suit and some of the issues that you address will be addressed.
I like this criticism, but I think there are two essentially disjoint parts here that are being criticized. The first is excess legibility, i.e., the issue of having explicit metrics and optimizing to the metrics at all. The second is that a few of the measurements that determine how many resources a group gets/how quickly it grows are correlated with things that are not inherently valuable at best and harmful at worst.
The first problem seems really hard to me: the legibility/autonomy trade-off is an age-old problem that happens in politics, business, and science, and seems to involve a genuine trade-off between organizational efficiency and the ability to capitalize on good but unorthodox ideas and individuals.
The second seems more accessible (though still hard), and reasonably separable from the first. Here I see a couple of things you flag (other than legibility/”corporateness” by itself) as parameters that positively contribute to growth but negatively contribute to the ability of EA to attract intellectually autonomous people. The first is “fire-and-brimstone” style arguments, where EA outreach tends to be all-or-nothing, “you either help save the sick children or you burn in Utilitarian Hell”, and the second is common-denominator level messaging that is optimized to build community (so things like slogans, manufactured community and sense of purpose; things that attract the people like Bob in your thought experiment), but not optimized to appeal to meta-level thinkers who understand the reasoning behind the slogans. Both are vaguely correlated with EA having commonalities with religious communities, and so I’m going to borrow the adjective “pious” to refer to ideas and individuals for which these factors are salient.
I like that you are pointing out that a lot of EA outreach is, in one way or another, an “appeal to piety”, and this is possibly bad. There might be a debate about whether this is actually bad and to what extent (e.g., the Catholic church is inefficient, but the sheer volume of charity it generates is nothing to sneer at), but I think I agree with the intuition that this is suboptimal, and that by Goodhart’s law, if pious people are more likely to react to outreach, eventually they will form a supermajority.
I don’t want to devalue the criticism that legibility is in itself a problem, and particularly ugh-y to certain types of people (e.g. to smart humanities majors). But I think that the problem of piety can be solved without giving up on legibility, and instead by using better metrics, that have more entanglement with the real world. This is something I believed before this post, so I might be shoe-horning it here: take this with a grain of salt.
But I want to point out that organizations that are constantly evaluated on some measurable parameter don’t necessarily tend to end up excessively pious. A sports team can’t survive by having the best team spirit; a software company will not see any profit if it only hires people who fervently believe in its advertising slogans. So maybe a solution to the problem of appeals to piety is to, as you say, reduce the importance of the metric of “HEA” generation in determining funding, clout, etc., but replace it with other hard-to-fake metrics that are less correlated with piety and more correlated with actually being effective at what you do.
I haven’t thought much about what the best metrics would be and am probably not qualified to make recommendations, but I just for plausibility’s sake here are a couple of examples of things that I think would be cool (if not necessarily realistic):
First, it would be neat (though potentially expensive) if there were a yearly competition between teams of EA’s (maybe student groups, or maybe something on a larger level) to use a funding source to create an independent real-world project and have their impact in QALY’s judged by an impartial third party.
Second, I think it would be nice to make “intramural” forms of existing competitions, such as the fiction contest, Scott Alexander’s book review contest, various super-forecasting contests, etc., and grading University groups on success (relative to past results). If something like this is implemented, I’d also like to see the focus of things like the fiction competition move away from “good messaging” (which smacks of piety) and towards “good fiction that happens to have an EA component, if you look hard enough”.
I think that if the funding culture becomes more explicitly focused on concentration of talent and on real-world effects and less on sheer numbers or uncritical mission alignment, then outreach will follow suit and some of the issues that you address will be addressed.