Selection bias likely makes looking at average NPS unwise. People willing to take flights to go to a conference about X are likely more enthusiastic about EA, and so willing to promote it to others. If the point is merely there is this cohort of ~ 2000 EAs who are very keen, fair enough. Yet this does not provide a huge amount of information about the perception of the ‘EA brand’ - EA global might have selected the dedicated fanbase out of a much larger pool of the ambivalent or antagonistic.
It’s fair to suggest that we don’t get carried away with NPS and it’s fair to argue that NPS may not represent EA’s brand as a whole.
But, for what it’s worth, asking EAG attendees about EA doesn’t seem like a stronger selection effect than the usual context for this question. NPS is about consumer loyalty. That means someone has to purchase the product before you can ask it.
If you ask someone for their NPS on an Apple Laptop, they have to spend $1K+ on the laptop first. It’s not clear that asking this question of people that attended a conference is substantially different.
I think the point is that for NPS, we’re interested in what all effective altruists think, since they’re the users of the product. But EAG attendees are not likely to be typical effective altruists: they will probably be more committed, and more positive about EA than a typical EA is.
To continue the Apple analogy, it’s a bit like basing your NPS score not on everyone that buys a laptop, but on the people who comment most on Apple product forums: these people won’t be typical of Apple’s consumers.
Selection bias likely makes looking at average NPS unwise. People willing to take flights to go to a conference about X are likely more enthusiastic about EA, and so willing to promote it to others. If the point is merely there is this cohort of ~ 2000 EAs who are very keen, fair enough. Yet this does not provide a huge amount of information about the perception of the ‘EA brand’ - EA global might have selected the dedicated fanbase out of a much larger pool of the ambivalent or antagonistic.
It’s fair to suggest that we don’t get carried away with NPS and it’s fair to argue that NPS may not represent EA’s brand as a whole.
But, for what it’s worth, asking EAG attendees about EA doesn’t seem like a stronger selection effect than the usual context for this question. NPS is about consumer loyalty. That means someone has to purchase the product before you can ask it.
If you ask someone for their NPS on an Apple Laptop, they have to spend $1K+ on the laptop first. It’s not clear that asking this question of people that attended a conference is substantially different.
I think the point is that for NPS, we’re interested in what all effective altruists think, since they’re the users of the product. But EAG attendees are not likely to be typical effective altruists: they will probably be more committed, and more positive about EA than a typical EA is.
To continue the Apple analogy, it’s a bit like basing your NPS score not on everyone that buys a laptop, but on the people who comment most on Apple product forums: these people won’t be typical of Apple’s consumers.