I really like the general class of improving community epistemics :)
That being said, I feel pretty pessimistic about having dedicated “community builders” come in to create good institutions that would then improve the epistemics of the field: in my experience, most such attempts fail, because they don’t actually solve a problem in a way that works for the people in the field (and in addition, they “poison the well”, in that it makes it harder for someone else to build an actually-functioning version of the solution, because everyone in the field now expects it to fail and so doesn’t buy in to it).
I feel much better about people within the field figuring out ways to improve the epistemics of the community they’re in, trialing them out themselves, and if they seem to work well only then attempting to formalize them into an institution.
Take me as an example. I’ve done a lot of work that could be characterized as “trying to improve the epistemics of a community”, such as:
Organizing calls for local EA group organizers to share best practices
The first five couldn’t have been done by a person without the relevant expertise (in AI alignment for the first four, and in EA group organizing for the fifth). If they were trying to build institutions that would lead to any of these six things happening, I think they might have succeeded, but it probably would have taken multiple years, as opposed to it taking ~a month each for me. (Here I’m assuming that an institution is “built” once it operates through the effort of people within the field, with no or very little ongoing effort from the person who started the institution.) It’s just quite hard to build institutions for a field without significant buy-in from people in the field, and creating that buy-in is hard.
I think people who find the general approach in this post interesting should probably be becoming very knowledgeable about a particular field (both the technical contents of the field, as well as the landscape of people who work on it), and then trying to improve the field from within.
It’s also of course fine to think of ideas for better institutions and pitch them to people in the field; what I want to avoid is coming up with a clever idea and then trying to cause it to exist without already having a lot of buy in from people in the field.
I really like the general class of improving community epistemics :)
That being said, I feel pretty pessimistic about having dedicated “community builders” come in to create good institutions that would then improve the epistemics of the field: in my experience, most such attempts fail, because they don’t actually solve a problem in a way that works for the people in the field (and in addition, they “poison the well”, in that it makes it harder for someone else to build an actually-functioning version of the solution, because everyone in the field now expects it to fail and so doesn’t buy in to it).
I feel much better about people within the field figuring out ways to improve the epistemics of the community they’re in, trialing them out themselves, and if they seem to work well only then attempting to formalize them into an institution.
Take me as an example. I’ve done a lot of work that could be characterized as “trying to improve the epistemics of a community”, such as:
The Alignment Newsletter
An AI alignment literature review
Clarifying key hypotheses that alignment researchers disagree about (mostly an advisory role)
Questioning the case for AI risk (1, 2)
Writing community building retrospectives
Organizing calls for local EA group organizers to share best practices
The first five couldn’t have been done by a person without the relevant expertise (in AI alignment for the first four, and in EA group organizing for the fifth). If they were trying to build institutions that would lead to any of these six things happening, I think they might have succeeded, but it probably would have taken multiple years, as opposed to it taking ~a month each for me. (Here I’m assuming that an institution is “built” once it operates through the effort of people within the field, with no or very little ongoing effort from the person who started the institution.) It’s just quite hard to build institutions for a field without significant buy-in from people in the field, and creating that buy-in is hard.
I think people who find the general approach in this post interesting should probably be becoming very knowledgeable about a particular field (both the technical contents of the field, as well as the landscape of people who work on it), and then trying to improve the field from within.
It’s also of course fine to think of ideas for better institutions and pitch them to people in the field; what I want to avoid is coming up with a clever idea and then trying to cause it to exist without already having a lot of buy in from people in the field.
I agree with all of what you say here. Building things for others can often go badly wrong. Thanks for sharing this perspective!