Should we limit this to social phenomena? Is it more likely for them because social phenomena are more complex and have more moving parts, so it’s easy to miss the most important effects and use vastly oversimplified models? Also, studies of social phenomena (+biology and medical research) often don’t replicate, possibly for that reason and also having high p-value cutoffs and many ways to do analyses until you get a result.
In general, I think we should be careful about claims of causal effects X → Y ->Z (or longer causal paths from X to Z) capturing most of the impact or even having the right sign for overall effects of X on Z. Ideally, you should manipulate X and directly measure Z.
Furthermore, the longer the causal path in the argument, the more places it could be missing parallel paths that are more important or have the opposite sign (and be more likely to have an error at some point). So, we should be more skeptical of longer causal paths in general. Plus, the intuitive examples you give (at least as stated) don’t establish that the specific effects are practically significant even if they exist.
Should we limit this to social phenomena? Is it more likely for them because social phenomena are more complex and have more moving parts, so it’s easy to miss the most important effects and use vastly oversimplified models? Also, studies of social phenomena (+biology and medical research) often don’t replicate, possibly for that reason and also having high p-value cutoffs and many ways to do analyses until you get a result.
In general, I think we should be careful about claims of causal effects X → Y ->Z (or longer causal paths from X to Z) capturing most of the impact or even having the right sign for overall effects of X on Z. Ideally, you should manipulate X and directly measure Z.
Furthermore, the longer the causal path in the argument, the more places it could be missing parallel paths that are more important or have the opposite sign (and be more likely to have an error at some point). So, we should be more skeptical of longer causal paths in general. Plus, the intuitive examples you give (at least as stated) don’t establish that the specific effects are practically significant even if they exist.
Yeah this seems right.
I think I don’t understand the point you’re making with your last sentence.