asked an ‘NPS’ question about EA, asked for qualitative information about positive/negative experiences of the community and examined predictors
asked about barriers to becoming more involved in EA
asked about reasons for people’s interest in EA declining or increasing
asked about what factors were important for retaining people in EA
asked about why people who the respondent knew dropped out
I’m pretty sceptical about the utility of Net Promoter Score in the classical sense for EA. I don’t think there’s any good evidence for the prescribed way of calculating Net Promoter Score (ignoring respondents who answer in the upper-middle of the scale, and then subtracting the proportion of people who selected one of the bottom 7 response levels from the proportion who selected one of the top two response items). And, as I mentioned in our original post, its validity and predictive power has been questioned. Furthermore, one of the most common uses is comparing the NPS score of an entity to an industry benchmark (e.g. the average scores for other companies in the same industry), but it’s very unclear what reference class would be relevant for EA, the community, as a whole, so it’s fundamentally not clear whether EA’s NPS score is good or bad. In the specific case of EA, I also suspect that the question of how excited one would be to recommend EA to a suitable friend may well be picking up on attitudes other than satisfaction with EA, i.e. literally how people would feel about recommending EA to someone. This might explain why the people with the highest ‘NPS’ scores (we just treated the measure as a straightforward ordinal varlable in our own analyses) were people who had just joined EA, and fairly reliably became lower over time.
Yeh much of this is in our Community Information post where we:
asked an ‘NPS’ question about EA, asked for qualitative information about positive/negative experiences of the community and examined predictors
asked about barriers to becoming more involved in EA
asked about reasons for people’s interest in EA declining or increasing
asked about what factors were important for retaining people in EA
asked about why people who the respondent knew dropped out
I’m pretty sceptical about the utility of Net Promoter Score in the classical sense for EA. I don’t think there’s any good evidence for the prescribed way of calculating Net Promoter Score (ignoring respondents who answer in the upper-middle of the scale, and then subtracting the proportion of people who selected one of the bottom 7 response levels from the proportion who selected one of the top two response items). And, as I mentioned in our original post, its validity and predictive power has been questioned. Furthermore, one of the most common uses is comparing the NPS score of an entity to an industry benchmark (e.g. the average scores for other companies in the same industry), but it’s very unclear what reference class would be relevant for EA, the community, as a whole, so it’s fundamentally not clear whether EA’s NPS score is good or bad. In the specific case of EA, I also suspect that the question of how excited one would be to recommend EA to a suitable friend may well be picking up on attitudes other than satisfaction with EA, i.e. literally how people would feel about recommending EA to someone. This might explain why the people with the highest ‘NPS’ scores (we just treated the measure as a straightforward ordinal varlable in our own analyses) were people who had just joined EA, and fairly reliably became lower over time.