Really interesting stuff! How many statements did you test in total? The depression only side resonates much more strongly with my priors than the anxiety only side; I’d have forecast that at least half of the anxiety-only side would correlated with both anxiety and depression.
Which results surprised you most? Any plans to replicate with a larger sample (maybe 10k or so)?
Glad you find it interesting! We tested maybe about 150 statements. Just to clarify, it’s not that the depression side doesn’t correlate with the anxiety side—because anxiety and depression are so correlated, any statement correlating with one is likely to correlate with the other. But when you statistically separate them (i.e., you look at what correlates with anxiety when you’ve controlled for depression, or the reverse), this clearer picture emerges).
While it would be great for someone to replicate these findings (to increase confidence in them), and I hope someone does that, the sample size (n=500) is fine in my view for this kind of result. There is diminishing benefit to larger sample sizes (the right sample size depends on the analyses being performed and the level of noise). So 10,000 people isn’t as much better than 500 people as it may sound.
For instance, at n=500 a measured correlation of r=0.50 has a 95th percentile confidence interval of r=0.43 to r=0.56. At n=10,000 it’s r=0.49 to r=0.51. For many purposes the latter isn’t much more useful than the former. See this confidence interval calculation tool for correlations for more details: http://vassarstats.net/rho.html
Thank you! I underestimated the rate at which sample size diminishes in returns, or perhaps over-valued much larger sample size increases you’d need to correct for multiple comparisons.
I followed the logic re: the statistical correction covariance but still found some of the results surprising. Especially the having to leave work part; I would have predicted that depression was more disabling with respect to workplace performance than anxiety. It’s thus very surprising to me that it’s only anxiety (once the covariance is corrected for) that correlates with having to leave work. I might be underestimating how hopeless people with depression feel, and thus their unwillingness to make life changes, or how high-functioning the average depressive is. Was this a result you found surprising?
Really interesting stuff! How many statements did you test in total? The depression only side resonates much more strongly with my priors than the anxiety only side; I’d have forecast that at least half of the anxiety-only side would correlated with both anxiety and depression.
Which results surprised you most? Any plans to replicate with a larger sample (maybe 10k or so)?
Glad you find it interesting! We tested maybe about 150 statements. Just to clarify, it’s not that the depression side doesn’t correlate with the anxiety side—because anxiety and depression are so correlated, any statement correlating with one is likely to correlate with the other. But when you statistically separate them (i.e., you look at what correlates with anxiety when you’ve controlled for depression, or the reverse), this clearer picture emerges).
While it would be great for someone to replicate these findings (to increase confidence in them), and I hope someone does that, the sample size (n=500) is fine in my view for this kind of result. There is diminishing benefit to larger sample sizes (the right sample size depends on the analyses being performed and the level of noise). So 10,000 people isn’t as much better than 500 people as it may sound.
For instance, at n=500 a measured correlation of r=0.50 has a 95th percentile confidence interval of r=0.43 to r=0.56. At n=10,000 it’s r=0.49 to r=0.51. For many purposes the latter isn’t much more useful than the former. See this confidence interval calculation tool for correlations for more details: http://vassarstats.net/rho.html
Thank you! I underestimated the rate at which sample size diminishes in returns, or perhaps over-valued much larger sample size increases you’d need to correct for multiple comparisons.
I followed the logic re: the statistical correction covariance but still found some of the results surprising. Especially the having to leave work part; I would have predicted that depression was more disabling with respect to workplace performance than anxiety. It’s thus very surprising to me that it’s only anxiety (once the covariance is corrected for) that correlates with having to leave work. I might be underestimating how hopeless people with depression feel, and thus their unwillingness to make life changes, or how high-functioning the average depressive is. Was this a result you found surprising?