Yeah it’s fair to worry about the causal interpretation of the satisfaction results, although I think the pre-trend you mention is mostly a result of the GAM model pulling the regression line down a bit due to the later observations. Limiting the analysis to the period before, say, November 9th, shows no such pre-trend. I’m personally more worried about possible confounds such as the one we note about engagement. It could also be that the FTX crisis motivated a certain group of people to take the survey and share their dissatisfaction. So, it’s true that the design of the survey does not lend itself to easily answer questions of causality, leaving it a matter of interpretation based on the results as a whole and context.
Regarding the overall interpretation, I think it might still be fair to conclude that the FTX crisis has decreased satisfaction if we take into account the other results, including the recalled satisfaction pre and before the FTX crisis and the explicit reports of concerns related to FTX, decreased trust etc.
Yeah it’s fair to worry about the causal interpretation of the satisfaction results, although I think the pre-trend you mention is mostly a result of the GAM model pulling the regression line down a bit due to the later observations. Limiting the analysis to the period before, say, November 9th, shows no such pre-trend. I’m personally more worried about possible confounds such as the one we note about engagement. It could also be that the FTX crisis motivated a certain group of people to take the survey and share their dissatisfaction. So, it’s true that the design of the survey does not lend itself to easily answer questions of causality, leaving it a matter of interpretation based on the results as a whole and context.
Regarding the overall interpretation, I think it might still be fair to conclude that the FTX crisis has decreased satisfaction if we take into account the other results, including the recalled satisfaction pre and before the FTX crisis and the explicit reports of concerns related to FTX, decreased trust etc.
Thanks, Willem, that all makes sense! I agree, the overall conclusion certainly seems fair, especially given the convergent evidence you cite.