I think I agree with the central theses here, as I read them: indeed, ideally we would (1) measure what happens to people individually, rather than on average, due to taking psychiatric drugs, and (2) measure an outcome that reflects people’s aggregate preference for their experience of life with the drug versus the counterfactual experience of life without the drug.
However, I think these problems are harder to resolve than the post suggests. Neither can be measured directly (outside circumscribed / assumption-laden situations) due to the fundamental problem of causal inference, which is not resolved by people’s self-reported estimates of individual causal effects. There are better approaches to consider than comparing averages, but, in my opinion, this is the default for practical causal inference reasons, rather than a failure to take phenomenology seriously.
I agree that (2) is more tractable; however, these improvements are non-trivial to implement. Continuing your example, if we reanalyze a trial to focus on patients with high baseline akathisia, who may be most affected by either a benefit or a harm, we have far fewer patients to analyze. What was once an adequately powered trial to detect a moderate effect in the full sample is now under-powered. The same issue arises when analyzing complex interactions: precisely estimating interaction effects generally requires far larger sample sizes than estimating main effects. So a trial designed to measure a main effect of a drug is unlikely to be sufficiently powered to estimate several interaction effects.
For either issue, the data is not already there in my view. That said, I may not be fully understanding what exactly you propose doing; are there examples of “[using] criticality and complex systems modeling tools to deal with symptom interactions” in a healthcare context that illustrate this sort of analysis?
Thank you! I will get back to you. My honest reaction to this comment: “AAAAAHHHH damn, this should be so incredibly obvious!! where to even start?” But I recognize I’m deep into the inside view of log scales, phenomenology visualization, valence quantification, etc. What seems mind-numbingly foot-stompingly obvious to me might not be to others. Do I say: “PLEASE BE CREATIVE” or should I write a treaties expanding on every point? I’ll do the latter. And I apologize for my candid reaction comment XD Seriously, I appreciarte the comment. It’s just… ahhhhhh!
[to be clear you raise very good points—I’m just trying to do the prosocial thing and communicate the large “inference gap” on this topic that currently exists—I’ll do better! Will respond object level. Cheers!]
I think I agree with the central theses here, as I read them: indeed, ideally we would (1) measure what happens to people individually, rather than on average, due to taking psychiatric drugs, and (2) measure an outcome that reflects people’s aggregate preference for their experience of life with the drug versus the counterfactual experience of life without the drug.
However, I think these problems are harder to resolve than the post suggests. Neither can be measured directly (outside circumscribed / assumption-laden situations) due to the fundamental problem of causal inference, which is not resolved by people’s self-reported estimates of individual causal effects. There are better approaches to consider than comparing averages, but, in my opinion, this is the default for practical causal inference reasons, rather than a failure to take phenomenology seriously.
I agree that (2) is more tractable; however, these improvements are non-trivial to implement. Continuing your example, if we reanalyze a trial to focus on patients with high baseline akathisia, who may be most affected by either a benefit or a harm, we have far fewer patients to analyze. What was once an adequately powered trial to detect a moderate effect in the full sample is now under-powered. The same issue arises when analyzing complex interactions: precisely estimating interaction effects generally requires far larger sample sizes than estimating main effects. So a trial designed to measure a main effect of a drug is unlikely to be sufficiently powered to estimate several interaction effects.
For either issue, the data is not already there in my view. That said, I may not be fully understanding what exactly you propose doing; are there examples of “[using] criticality and complex systems modeling tools to deal with symptom interactions” in a healthcare context that illustrate this sort of analysis?
Thank you! I will get back to you. My honest reaction to this comment: “AAAAAHHHH damn, this should be so incredibly obvious!! where to even start?” But I recognize I’m deep into the inside view of log scales, phenomenology visualization, valence quantification, etc. What seems mind-numbingly foot-stompingly obvious to me might not be to others. Do I say: “PLEASE BE CREATIVE” or should I write a treaties expanding on every point? I’ll do the latter. And I apologize for my candid reaction comment XD Seriously, I appreciarte the comment. It’s just… ahhhhhh!
[to be clear you raise very good points—I’m just trying to do the prosocial thing and communicate the large “inference gap” on this topic that currently exists—I’ll do better! Will respond object level. Cheers!]
I’m curious to circle back here if you have a chance.