“I’m not suggesting that quantitive facts should be ignored during the hypothesis generation stage, just that we need to understand the hypothesis space before we can choose appropriate metrics, otherwise we may artificially limit the set of theories that we consider.”
I very much agree with this view methodologically. This is why we used qualitative research methods in addition to quantitative for the LEAN impact assessment. There is real risk of narrowing perspective and obscuring important factors from view if you commit to specific metrics prematurely. Qualitative research design is based on the aim of keeping the research process grounded and inductive, always responsive to unanticipated factors, regularly revisiting fundamental problem framing and steering sharply clear of methodological individualism, which is the approach you described (https://en.wikipedia.org/wiki/Methodological_individualism).
In the case of the impact assessment (where LEAN was trying to judge how effective our group support programme is, and how much impact groups have), we could look at metrics like group size, the number of individuals converted to EA, lifestyle changes, donations, pledges, events held and so forth. However qualitative interviews were used to piece together more complicated pathways that connect different nodes. The EA network is relatively small, which means that detailed examples can be very informative. I would like to see mixed methods of this kind used more.
If people want to avoid methodological individualism while still using quantitative techniques,
social network analysis and multiple correspondence analysis are two quantitative techniques that many sociologists have used in order to tackle similar issues when working with much larger datasets. Social network analysis allows you to map out ‘pipelines’ of the kind you described in order to identify which nodes in the community are the most prominent and influential in terms of providing critical connections.
We don’t, however, even need to do any more empirical analysis of EA to know that what you say is true… that many of the most important, high impact and high yield developments and achievements come down to an interaction between different community and information sources all coming together in a fortuitous way for a given trajectory. We can be sure of this not only by reflecting on examples in EA but also because this is simply a sociological human fact (often analysed and illustrated in the sprawling ‘social capital’ research field). The question then becomes, as you suggest, how do we cultivate the right environment for these vital spontaneous connections and interactions to take place?
My opinion on this is that we already do very well in this regard. Not through any virtue per se, other than the fact that the smaller a community is, the faster and more readily connections will arise (too small, of course, and you run out of useful nodes). However we definitely can do better, and the most urgent area for practical intervention is restructuring this forum in order to better serve the EA online community. This is something that has come out very clearly both in the 2017 Local Group Survey but also our interviews with group organisers. I think it is also quite self evident. On offer for budding EAs are either dead backwater Facebook groups with no life, or monstrous central groups with hundreds of members where only the most confident EAs feel comfortable posting. The forum is similar. Although the option of anonymity probably empowers some people to speak up, there is a much larger collective of lurkers who will feel too intimidated to contribute. A system of subforums that allow sheltered zones targetted at different kinds of EA would encourage a good deal more to come out of the woodwork and allow them to connect to one another. Individuals could then progress from a newbie friendly subforum to more ‘advanced’ or in depth content and conversations. I’m very happy that CEA will be taking on a restructure of the forum in the coming months.
Another area that can be optimised is the streamlining and organisation of content into a more user friendly and accessible format. This is something LEAN will be working on in the near future both in terms of making existing content more navigable but also in terms of continuing to make bespoke introductions between aligned individuals and organisations, but also helping EAs and EA groups to find one another more easily (like through our map of EAs) and through maintaining up to date contact information, and ensuring that it is easily found.
I very much agree with this view methodologically. This is why we used qualitative research methods in addition to quantitative for the LEAN impact assessment. There is real risk of narrowing perspective and obscuring important factors from view if you commit to specific metrics prematurely. Qualitative research design is based on the aim of keeping the research process grounded and inductive, always responsive to unanticipated factors, regularly revisiting fundamental problem framing and steering sharply clear of methodological individualism, which is the approach you described (https://en.wikipedia.org/wiki/Methodological_individualism).
In the case of the impact assessment (where LEAN was trying to judge how effective our group support programme is, and how much impact groups have), we could look at metrics like group size, the number of individuals converted to EA, lifestyle changes, donations, pledges, events held and so forth. However qualitative interviews were used to piece together more complicated pathways that connect different nodes. The EA network is relatively small, which means that detailed examples can be very informative. I would like to see mixed methods of this kind used more.
If people want to avoid methodological individualism while still using quantitative techniques, social network analysis and multiple correspondence analysis are two quantitative techniques that many sociologists have used in order to tackle similar issues when working with much larger datasets. Social network analysis allows you to map out ‘pipelines’ of the kind you described in order to identify which nodes in the community are the most prominent and influential in terms of providing critical connections.
We don’t, however, even need to do any more empirical analysis of EA to know that what you say is true… that many of the most important, high impact and high yield developments and achievements come down to an interaction between different community and information sources all coming together in a fortuitous way for a given trajectory. We can be sure of this not only by reflecting on examples in EA but also because this is simply a sociological human fact (often analysed and illustrated in the sprawling ‘social capital’ research field). The question then becomes, as you suggest, how do we cultivate the right environment for these vital spontaneous connections and interactions to take place?
My opinion on this is that we already do very well in this regard. Not through any virtue per se, other than the fact that the smaller a community is, the faster and more readily connections will arise (too small, of course, and you run out of useful nodes). However we definitely can do better, and the most urgent area for practical intervention is restructuring this forum in order to better serve the EA online community. This is something that has come out very clearly both in the 2017 Local Group Survey but also our interviews with group organisers. I think it is also quite self evident. On offer for budding EAs are either dead backwater Facebook groups with no life, or monstrous central groups with hundreds of members where only the most confident EAs feel comfortable posting. The forum is similar. Although the option of anonymity probably empowers some people to speak up, there is a much larger collective of lurkers who will feel too intimidated to contribute. A system of subforums that allow sheltered zones targetted at different kinds of EA would encourage a good deal more to come out of the woodwork and allow them to connect to one another. Individuals could then progress from a newbie friendly subforum to more ‘advanced’ or in depth content and conversations. I’m very happy that CEA will be taking on a restructure of the forum in the coming months.
Another area that can be optimised is the streamlining and organisation of content into a more user friendly and accessible format. This is something LEAN will be working on in the near future both in terms of making existing content more navigable but also in terms of continuing to make bespoke introductions between aligned individuals and organisations, but also helping EAs and EA groups to find one another more easily (like through our map of EAs) and through maintaining up to date contact information, and ensuring that it is easily found.