Hi Jared and Maya, I’m Richie the (recently promoted) Director of Research for Bryant. Thank you so much for this!
Having just read your post here but not yet read your published comment, I broadly agree with your approach, it’s certainly an improvement on the original paper in my view. Great to see the analysis cleared up some puzzling results too.
This research was conducted before I was hired. I am self-taught in non-experimental causal inference / econometrics, and see it as an essential area for behavioural scientists to incorporate in their work (economists have a great handle on it, but in psychology where I’m originally from it’s woefully lacking). Going forward, I’ll be looking to incorporate more of a causal approach in our research.
In general, I think there are two useful, general pieces of statistical advice here I’d want to highlight:
Don’t condition on post treatment variables
Use the raw variables where possible. I.e. avoid change scores, be careful about making new variables by combining others. I personally have a vendetta against creating unnecessary ratio variables!
Also, for readers who are interested in learning a bit more about causal inference: The Effect by Nick Huntingdon-Klein is an excellent, free resource. So good, I paid for it when I didn’t have to.
Hi Jared and Maya, I’m Richie the (recently promoted) Director of Research for Bryant. Thank you so much for this!
Having just read your post here but not yet read your published comment, I broadly agree with your approach, it’s certainly an improvement on the original paper in my view. Great to see the analysis cleared up some puzzling results too.
This research was conducted before I was hired. I am self-taught in non-experimental causal inference / econometrics, and see it as an essential area for behavioural scientists to incorporate in their work (economists have a great handle on it, but in psychology where I’m originally from it’s woefully lacking). Going forward, I’ll be looking to incorporate more of a causal approach in our research.
In general, I think there are two useful, general pieces of statistical advice here I’d want to highlight:
Don’t condition on post treatment variables
Use the raw variables where possible. I.e. avoid change scores, be careful about making new variables by combining others. I personally have a vendetta against creating unnecessary ratio variables!
Also, for readers who are interested in learning a bit more about causal inference: The Effect by Nick Huntingdon-Klein is an excellent, free resource. So good, I paid for it when I didn’t have to.
Thanks, Richie – causal inference is definitely tricky and subtle! Thanks for sharing “The Effect”. I had not seen this.