I think this is interesting research but I would quibble with your interpretation of the top part of figure 4 as a causal effect. As far as I can tell, that part is a cross-sectional analysis that is only valid if individuals with greater knowledge of climate organisations are the same in all relevant ways as those with lower levels of knowledge that identify with Friends of the Earth to the same extent. This seems unlikely to be true and indeed does not have to hold for the fixed effect analyses that make up the majority of this piece to be unbiased. If I have not misinterpreted something here, I would recommend being much clearer in future about when switching between fixed and random effects models as they estimate very different parameters, with fixed effects usually being much more reliable at retrieving causal effects.
Hi Joseph, thanks for your comment! It might have been a misunderstanding but we’re definitely not claiming the top panel of Figure 4 is causal. It might have been because I cut out parts of our result section from the full report to make the EA Forum post shorter, but we discuss that the top part is cross-sectional as you say, and the bottom panel of Figure 4 (with difference scores) is more causal evidence.
Thanks. I think the issue is the use of the word effect, which usually implies causality in my field (Economics), rather than association when referring to the cross-sectional analysis alongside the fact that context was lost when it was edited down for the forum.
I think this is interesting research but I would quibble with your interpretation of the top part of figure 4 as a causal effect. As far as I can tell, that part is a cross-sectional analysis that is only valid if individuals with greater knowledge of climate organisations are the same in all relevant ways as those with lower levels of knowledge that identify with Friends of the Earth to the same extent. This seems unlikely to be true and indeed does not have to hold for the fixed effect analyses that make up the majority of this piece to be unbiased. If I have not misinterpreted something here, I would recommend being much clearer in future about when switching between fixed and random effects models as they estimate very different parameters, with fixed effects usually being much more reliable at retrieving causal effects.
Hi Joseph, thanks for your comment! It might have been a misunderstanding but we’re definitely not claiming the top panel of Figure 4 is causal. It might have been because I cut out parts of our result section from the full report to make the EA Forum post shorter, but we discuss that the top part is cross-sectional as you say, and the bottom panel of Figure 4 (with difference scores) is more causal evidence.
Thanks. I think the issue is the use of the word effect, which usually implies causality in my field (Economics), rather than association when referring to the cross-sectional analysis alongside the fact that context was lost when it was edited down for the forum.