I spent one day trying to compare retention in EA to other movements. I was unable to find anything that I was very happy with, but I’m sharing these comparisons in case they are helpful to others. Hopefully Cunningham’s law will kick in and someone will be spurred to do a better analysis.
I found semi-comparable estimates of retention for Southern Baptist megachurch youth ministry participants, vegetarians, and civil rights activists who participated in Freedom Summer. While none of these data sources seemed precise or perfectly comparable, my best guess is that EA is slightly better at retention than they are, and unlikely to be substantially worse.
How does EA compare to other movements?
Most interviewees remarked that, once someone gets to the point where their career is driven by EA considerations, it’s unlikely that they will leave. This might suggest that CEA should not focus on retention, as it is already going well.
For some insight on this, I tried to compare Ben Toddd’s estimates with estimates from other movements. My best guess is that EA has better retention than Southern Baptist megachurches and Freedom Summer participants, and similar or slightly worse retention to vegetarianism. Most of my uncertainty comes from how reliable these estimates are and how comparable they are to each other.
Because of how wide my error bars are, it’s hard to say too much with confidence. I don’t think EA has a severe retention problem (compared to e.g. Baptist churches). We also don’t seem substantially better at retention than others (e.g. vegetarianism). So my guess is that EA is slightly above average at retaining highly engaged members (compared to other movements), which makes me think it’s worth investing a moderate amount of resources into retention.
Five-year dropout rate
Ben Todd’s estimate for top 2000 most engaged EA’s
Dropout rate (don’t self-report as moderate or high involvement)
Dropout rate (don’t self-report as high involvement)
Disengaged prospect
100%
100%
Normative attender
26%
58%
Enthusiastic follower
21%
65%
Engaged disciple
8%
42%
I think we are mostly interested in those listed as “engaged disciples” who continue to self-report as “high involvement”, so the 42% number is what I’m going to focus on.
This study did not control for the fact that people who were older at the time of surveying were more likely to have dropped out.[2] Also note that the survey was retrospective and presumably suffers from response bias.
To try to handle the fact they didn’t control for age, I tried to find the parameters for a geometric distribution which fit their overall results of 42% dropout. A constant 14.5% annual dropout rate results in 42% of respondents having dropped out:
Age
Elapsed years
Percent of respondents
Expected
dropout rate
Percent of total sample who
dropped out from this age bracket
18
0
0.4
0.00
0.00
19
1
10
0.15
1.45
20
2
24.4
0.27
6.56
21
3
20.1
0.37
7.54
22
4
14
0.47
6.52
23
5
7.2
0.54
3.91
24
6
8.6
0.61
5.24
25
7
4.7
0.67
3.13
26
8
6.1
0.71
4.36
27
9
4.7
0.76
3.55
Total
45
100.2
42.26
This assumes that “engaged disciples” are evenly distributed across the ages, which is probably not completely true but I would guess mostly true. A 14.5% annual dropout rate is a 54% five year dropout rate.
I’m not very confident in the 54% number, but I feel relatively confident that the number is higher than Ben Todd’s estimate of 20%. The paper cites another survey which found that 70-90% of teenagers stop attending church within two years of high school graduation. (One wrinkle is that there is some evidence people reengage with the church in their late 20s/30s, meaning that it’s a little hard to define what “dropout” actually means.)
Vegetarian adherence
ACE estimates that the average vegetarian stays vegetarian for 3.9-7.2 years,[3] implying a five-year dropout rate of 53%-77%.[4] My guess is that the average vegetarian is equivalent to Ben’s lowest level of engagement (GWWC member but not otherwise involved) or even less engaged, and being vegetarian seems about as big of a commitment as keeping the GWWC pledge. This implies a dropout rate higher than what Ben estimated for EA.
US Civil Rights Activists—Freedom Summer
Freedom Summer was a 1964 project with extremely high asks of its participants: they were mostly college students who gave up their summer to be unpaid volunteers facing severe violence (several were murdered). Participants needed to apply beforehand, and those who applied but withdrew from the project before it started give some estimate of adherence amongst civil rights activists, split by engagement level.
I think that attending Freedom Summer is a sign of involvement in the civil rights movement very roughly comparable to how having attended early EA retreats is a sign of being at one of Ben’s two highest levels of EA involvement. So that’s what I’m going to compare it to, even though the comparison is quite inexact.[5]
In this and the rest of this section I’m going to assume a constant annual dropout rate, i.e. a geometric distribution, unless otherwise specified.
In the section where the author analyzes demographic variables, they say that only educational ones had a significant impact. Taken at face value, this implies that age does not correlate with dropout rate, which seems unlikely to me, but possible.
Taken from their “final estimate” but excluding the effect of cultured meat alternatives, death, and GCRs, since those do not seem relevant to the question I’m trying to answer here.
Assuming a geometric distribution whose expectation is 1/p
There are a lot of ways in which Freedom Summer is different from EA, the most notable being the violence participants experienced. Another major concern is that the Civil Rights Act of 1968 is seen by some as the culmination of the civil rights movement, meaning that many participants perhaps should have been expected to move to other causes after 1968 (e.g. anti-Vietnam War). But I think we would have expected many of them to be active in at least some social movement, if they were truly “retained”.
This is the percentage who were active in civil rights at some point within the five years following Freedom Summer, but I am simplifying this to assume it’s an annual dropout rate. My justification for this is a) there probably aren’t many people who didn’t engage for 4 years and then became active again in the fifth, and b) to the extent that these people do exist, it’s arguably more accurate to classify them as having disengaged and then reengaged instead of having been retained.
Retention in EA—Part III: Retention Comparisons
Summary
I spent one day trying to compare retention in EA to other movements. I was unable to find anything that I was very happy with, but I’m sharing these comparisons in case they are helpful to others. Hopefully Cunningham’s law will kick in and someone will be spurred to do a better analysis.
I found semi-comparable estimates of retention for Southern Baptist megachurch youth ministry participants, vegetarians, and civil rights activists who participated in Freedom Summer. While none of these data sources seemed precise or perfectly comparable, my best guess is that EA is slightly better at retention than they are, and unlikely to be substantially worse.
How does EA compare to other movements?
Most interviewees remarked that, once someone gets to the point where their career is driven by EA considerations, it’s unlikely that they will leave. This might suggest that CEA should not focus on retention, as it is already going well.
For some insight on this, I tried to compare Ben Toddd’s estimates with estimates from other movements. My best guess is that EA has better retention than Southern Baptist megachurches and Freedom Summer participants, and similar or slightly worse retention to vegetarianism. Most of my uncertainty comes from how reliable these estimates are and how comparable they are to each other.
Because of how wide my error bars are, it’s hard to say too much with confidence. I don’t think EA has a severe retention problem (compared to e.g. Baptist churches). We also don’t seem substantially better at retention than others (e.g. vegetarianism). So my guess is that EA is slightly above average at retaining highly engaged members (compared to other movements), which makes me think it’s worth investing a moderate amount of resources into retention.
Ben Todd’s estimates
Taken from More empirical data on ‘value drift’.
Youth Ministry Adherence
An Assessment of Dropout Rates of Former Youth Ministry Participants in Conservative Southern Baptist Megachurches classified youth ministry participants (teenagers who attended a certain type of religious education) into one of four engagement levels based on their answer to a survey (asking e.g. how frequently they attend church) and measured their retention.
I think we are mostly interested in those listed as “engaged disciples” who continue to self-report as “high involvement”, so the 42% number is what I’m going to focus on.
This study did not control for the fact that people who were older at the time of surveying were more likely to have dropped out.[2] Also note that the survey was retrospective and presumably suffers from response bias.
To try to handle the fact they didn’t control for age, I tried to find the parameters for a geometric distribution which fit their overall results of 42% dropout. A constant 14.5% annual dropout rate results in 42% of respondents having dropped out:
Expected
dropout rate
Percent of total sample who
dropped out from this age bracket
18
0
0.4
0.00
0.00
19
1
10
0.15
1.45
20
2
24.4
0.27
6.56
21
3
20.1
0.37
7.54
22
4
14
0.47
6.52
23
5
7.2
0.54
3.91
24
6
8.6
0.61
5.24
25
7
4.7
0.67
3.13
26
8
6.1
0.71
4.36
27
9
4.7
0.76
3.55
45
100.2
42.26
This assumes that “engaged disciples” are evenly distributed across the ages, which is probably not completely true but I would guess mostly true. A 14.5% annual dropout rate is a 54% five year dropout rate.
I’m not very confident in the 54% number, but I feel relatively confident that the number is higher than Ben Todd’s estimate of 20%. The paper cites another survey which found that 70-90% of teenagers stop attending church within two years of high school graduation. (One wrinkle is that there is some evidence people reengage with the church in their late 20s/30s, meaning that it’s a little hard to define what “dropout” actually means.)
Vegetarian adherence
ACE estimates that the average vegetarian stays vegetarian for 3.9-7.2 years,[3] implying a five-year dropout rate of 53%-77%.[4] My guess is that the average vegetarian is equivalent to Ben’s lowest level of engagement (GWWC member but not otherwise involved) or even less engaged, and being vegetarian seems about as big of a commitment as keeping the GWWC pledge. This implies a dropout rate higher than what Ben estimated for EA.
US Civil Rights Activists—Freedom Summer
Freedom Summer was a 1964 project with extremely high asks of its participants: they were mostly college students who gave up their summer to be unpaid volunteers facing severe violence (several were murdered). Participants needed to apply beforehand, and those who applied but withdrew from the project before it started give some estimate of adherence amongst civil rights activists, split by engagement level.
I think that attending Freedom Summer is a sign of involvement in the civil rights movement very roughly comparable to how having attended early EA retreats is a sign of being at one of Ben’s two highest levels of EA involvement. So that’s what I’m going to compare it to, even though the comparison is quite inexact.[5]
The Biographical Consequences of Activism followed up with participants approximately 20 years after Freedom Summer and found the following levels of engagement:
Again, the methodology makes comparisons hard, but it seems like the dropout rate from Freedom Summer is higher than Ben Todd’s estimates for EA.
Religious adherence
Social Predictors of Retention in and Switching from the Religious Faith of Family of Origin: Another Look Using Religious Tradition Self-Identification found that age was positively correlated with remaining in a religion, implying a negative dropout rate. I’m not sure whether this is because people drop out when they are young and then rejoin when they are older or if people who were born longer ago are just more committed to their religion, but either way it seems different enough from EA that the comparison isn’t useful.
Footnotes
In this and the rest of this section I’m going to assume a constant annual dropout rate, i.e. a geometric distribution, unless otherwise specified.
In the section where the author analyzes demographic variables, they say that only educational ones had a significant impact. Taken at face value, this implies that age does not correlate with dropout rate, which seems unlikely to me, but possible.
Taken from their “final estimate” but excluding the effect of cultured meat alternatives, death, and GCRs, since those do not seem relevant to the question I’m trying to answer here.
Assuming a geometric distribution whose expectation is 1/p
There are a lot of ways in which Freedom Summer is different from EA, the most notable being the violence participants experienced. Another major concern is that the Civil Rights Act of 1968 is seen by some as the culmination of the civil rights movement, meaning that many participants perhaps should have been expected to move to other causes after 1968 (e.g. anti-Vietnam War). But I think we would have expected many of them to be active in at least some social movement, if they were truly “retained”.
This is the percentage who were active in civil rights at some point within the five years following Freedom Summer, but I am simplifying this to assume it’s an annual dropout rate. My justification for this is a) there probably aren’t many people who didn’t engage for 4 years and then became active again in the fifth, and b) to the extent that these people do exist, it’s arguably more accurate to classify them as having disengaged and then reengaged instead of having been retained.