Is it possible to compare overall satisfaction to previous EA surveys? As you say, all the methods for extracting the impact of FTX from this data seem a bit suspect. Satisfaction now vs. satisfaction 1-2 years ago is simpler and arguably more decision-relevant.
Is it possible to compare overall satisfaction to previous EA surveys?
It certainly is! (That said, I don’t think comparing 2020 to 2022 is necessarily simpler. If we want to know where we are now, we presumably want to distinguish pre-FTX and post-FTX responses. And, if we want to know the effect of FTX, we may want to control for engagement, like in our other analyses, to account for differences between groups).
This plot shows the raw mean satisfaction for EAS 2020, EAS 2022, and for EAS 2022 (but only post-FTX).
However, if we control for engagement, and distinguish 2022 pre-FTX and post-FTX, we see that satisfaction actually seemed to be slightly higher in 2022 pre-FTX, before falling below 2020 levels post-FTX. (All differences significant.)
This is also what we see if we plot satisfaction across time for 2020 and 2022. 2022 appears to start out higher than 2022, before falling lower.
Is it possible to compare overall satisfaction to previous EA surveys? As you say, all the methods for extracting the impact of FTX from this data seem a bit suspect. Satisfaction now vs. satisfaction 1-2 years ago is simpler and arguably more decision-relevant.
It certainly is! (That said, I don’t think comparing 2020 to 2022 is necessarily simpler. If we want to know where we are now, we presumably want to distinguish pre-FTX and post-FTX responses. And, if we want to know the effect of FTX, we may want to control for engagement, like in our other analyses, to account for differences between groups).
This plot shows the raw mean satisfaction for EAS 2020, EAS 2022, and for EAS 2022 (but only post-FTX).
However, if we control for engagement, and distinguish 2022 pre-FTX and post-FTX, we see that satisfaction actually seemed to be slightly higher in 2022 pre-FTX, before falling below 2020 levels post-FTX. (All differences significant.)
This is also what we see if we plot satisfaction across time for 2020 and 2022. 2022 appears to start out higher than 2022, before falling lower.
Thank, that’s very helpful!
(I’m not surprised satisfaction was higher in 2022 than 2020 before FTX.)