My response to this post overall is that I think some of what is going on here is that different people and different organizations mean very different things when we say “Depression”. Since “depression” is not really a binary, the value of averting “1 case of severe depression” can change a lot depending on how you define severity, in such a way that differences in reasonable definitions of “sufficiently bad depression” can plausibly differ by 1-3x when you break it down into “how many SD counts as curing depression” terms.
However, the in-progress nature of SoGives’ mental health work makes pinning down what we do mean sort of tricky. What exactly did the participants in the SoGive Delphi Process mean when they said “severe depression”? How should I, as an analyst who isn’t aiming to set the moral weights but is attempting to advise people using them, interpret that? These things are currently in flux, in the sense that I’m basically in the process of making various judgement calls about them right now, which I’ll describe below.
You commented:
I’m not sure 2-5 SD-years is plausible for severe depression. 3 SDs would saturate the entire scale 0-24.
It’s true that the PHQ-9 score of 27 points maxes out around 2-4sd. How many SD it is exactly depends on the spread of your population of course (for example if 1sd=6.1 points then the range of a 27 point scale spans 4.42sd ), and for some population spreads it would be 3sd.
These two things are related actually! I think the trouble is that the word “severity depression” is ambiguous as to how bad it is, so different people can mean different things by it.
One might argue that the following was an awkward workaround which should have been done differently, but basically, to make transparent my internal thought process here (In terms of what I thought after joining sogive, starting this analysis, and encountering these weights) was the following:
-> “hm, this implies we’re willing to trade averting 25 years of depression against one (mostly neonatal) death. Is this unusual?”
→ “Maybe we are thinking about the type of severe, suicidal depression that is an extremely net negative experience, a state which is worse than death.”
→ “Every questionnaire creator seems to have recommended cut-offs for gradients of depression such as “mild” and “moderate” (e.g. the creators of the PHQ-9 scale are recommending 20 points as the cut-off for “severe” depression) but these aren’t consistent between scales and are ultimately arbitrary choices.”
-> “extrapolating linearly from the time-trade-off literature people seemed to think that a year of depression breaks even with dying a year earlier around 5.5sd. Maybe less if it’s not linear.”
-> “But maybe it should be more because what’s really happening here is that we’re seeing multiple patients improve by 0.5-0.8 sd. The people surveyed in that paper think that the difference between 2sd->3sd is bigger than 1sd->2sd. People might disagree on the correct way to sum these up.”
→ concluding with me thinking that various reasonable people might set the standard for “averting severe depression” between 2-6 sd, depending on whether they wanted ordinary severity or worse than death severity
So, hopefully that answers your question as to why I wrote to you that 2-5sd is reasonable for severe depression. I’m going to try to justify this further in subsequent posts. Some additional thoughts that I had were:
-> I notice that this is still weighting depression more heavily than the people surveyed in the time-trade-off, but if we set it on the higher range of 3-6sd it still feels like a morally plausible view (especially considering that some people might have assigned lower moral weight to neonates).
→ My role is to tell people what the effect is, not to tell them what moral weights to use. However, I’m noticing that all the wiggle room to interpret what “severe” means is on me, and I notice that I keep wanting to nudge the SD-years I accept as higher in order to make the view match what I think is morally plausible.
-> I’ll just provisionally use something between 3-5 sd-years for the purpose of completing analysis, because my main aim is to figure out what therapy does in terms of sd.
→ But I should probably publish a tool that allows people to think about moral weights in terms of standard deviation, and maybe we can survey people for moral weights again in the future in a manner that lets them talk about standard deviations rather than whatever connotations they attached to “severe depression”. Then we can figure out what people really think about various grades of depression and how much income and life they’re willing to trade about it.
In fact the next thing I’m scheduled to publish is a write up that talks in detail about how to translate SD into something more morally intuitive. So hopefully that will help us make some progress on the moral weights issue.
So to summarize, I think (assuming your calculations w.r.t. everyone else’s weights are correct) what’s going on here is that it looks like SoGive is weighing depression 4x more than everyone, but those moral weights were set in the absence of a concrete recommendations, and in the end …and arguably this is an artifact me choosing after the fact to set a really high SD threshold for “severity” as a reaction to the weights, and what really needs to happen is that we need to go through that process I described of polling people again in a way that breaks down “severity” differently… in the final analysis, once a concrete recommendation comes out, it probably won’t be that different? (Though you’ve added two items, sd<->daly/wellby and cash<->sd, on my list of things to check for robustness and if it ends up being notable I’m definitely going to flag it, so thank you for that). I do think that this story will ultimately end with some revisiting of moral weights, how they should be set, and what they mean, and how to communicate them.
(There’s another point that came up in the other thread though, regarding “does it pass the sanity check w.r.t. cash transfer effects on well being”, which this doesn’t address. although it falls outside the scope of my current work I have been wanting to get a firmer sense of the empirical cash <-> wellby <-> sd depression correlations and apropos of your comments perhaps this should be made more explicit in moral weights agendas.)
My response to this post overall is that I think some of what is going on here is that different people and different organizations mean very different things when we say “Depression”. Since “depression” is not really a binary, the value of averting “1 case of severe depression” can change a lot depending on how you define severity, in such a way that differences in reasonable definitions of “sufficiently bad depression” can plausibly differ by 1-3x when you break it down into “how many SD counts as curing depression” terms.
However, the in-progress nature of SoGives’ mental health work makes pinning down what we do mean sort of tricky. What exactly did the participants in the SoGive Delphi Process mean when they said “severe depression”? How should I, as an analyst who isn’t aiming to set the moral weights but is attempting to advise people using them, interpret that? These things are currently in flux, in the sense that I’m basically in the process of making various judgement calls about them right now, which I’ll describe below.
You commented:
It’s true that the PHQ-9 score of 27 points maxes out around 2-4sd. How many SD it is exactly depends on the spread of your population of course (for example if 1sd=6.1 points then the range of a 27 point scale spans 4.42sd ), and for some population spreads it would be 3sd.
These two things are related actually! I think the trouble is that the word “severity depression” is ambiguous as to how bad it is, so different people can mean different things by it.
One might argue that the following was an awkward workaround which should have been done differently, but basically, to make transparent my internal thought process here (In terms of what I thought after joining sogive, starting this analysis, and encountering these weights) was the following:
-> “hm, this implies we’re willing to trade averting 25 years of depression against one (mostly neonatal) death. Is this unusual?”
→ “Maybe we are thinking about the type of severe, suicidal depression that is an extremely net negative experience, a state which is worse than death.”
→ “Every questionnaire creator seems to have recommended cut-offs for gradients of depression such as “mild” and “moderate” (e.g. the creators of the PHQ-9 scale are recommending 20 points as the cut-off for “severe” depression) but these aren’t consistent between scales and are ultimately arbitrary choices.”
-> “extrapolating linearly from the time-trade-off literature people seemed to think that a year of depression breaks even with dying a year earlier around 5.5sd. Maybe less if it’s not linear.”
-> “But maybe it should be more because what’s really happening here is that we’re seeing multiple patients improve by 0.5-0.8 sd. The people surveyed in that paper think that the difference between 2sd->3sd is bigger than 1sd->2sd. People might disagree on the correct way to sum these up.”
→ concluding with me thinking that various reasonable people might set the standard for “averting severe depression” between 2-6 sd, depending on whether they wanted ordinary severity or worse than death severity
So, hopefully that answers your question as to why I wrote to you that 2-5sd is reasonable for severe depression. I’m going to try to justify this further in subsequent posts. Some additional thoughts that I had were:
-> I notice that this is still weighting depression more heavily than the people surveyed in the time-trade-off, but if we set it on the higher range of 3-6sd it still feels like a morally plausible view (especially considering that some people might have assigned lower moral weight to neonates).
→ My role is to tell people what the effect is, not to tell them what moral weights to use. However, I’m noticing that all the wiggle room to interpret what “severe” means is on me, and I notice that I keep wanting to nudge the SD-years I accept as higher in order to make the view match what I think is morally plausible.
-> I’ll just provisionally use something between 3-5 sd-years for the purpose of completing analysis, because my main aim is to figure out what therapy does in terms of sd.
→ But I should probably publish a tool that allows people to think about moral weights in terms of standard deviation, and maybe we can survey people for moral weights again in the future in a manner that lets them talk about standard deviations rather than whatever connotations they attached to “severe depression”. Then we can figure out what people really think about various grades of depression and how much income and life they’re willing to trade about it.
In fact the next thing I’m scheduled to publish is a write up that talks in detail about how to translate SD into something more morally intuitive. So hopefully that will help us make some progress on the moral weights issue.
So to summarize, I think (assuming your calculations w.r.t. everyone else’s weights are correct) what’s going on here is that it looks like SoGive is weighing depression 4x more than everyone, but those moral weights were set in the absence of a concrete recommendations, and in the end …and arguably this is an artifact me choosing after the fact to set a really high SD threshold for “severity” as a reaction to the weights, and what really needs to happen is that we need to go through that process I described of polling people again in a way that breaks down “severity” differently… in the final analysis, once a concrete recommendation comes out, it probably won’t be that different? (Though you’ve added two items, sd<->daly/wellby and cash<->sd, on my list of things to check for robustness and if it ends up being notable I’m definitely going to flag it, so thank you for that). I do think that this story will ultimately end with some revisiting of moral weights, how they should be set, and what they mean, and how to communicate them.
(There’s another point that came up in the other thread though, regarding “does it pass the sanity check w.r.t. cash transfer effects on well being”, which this doesn’t address. although it falls outside the scope of my current work I have been wanting to get a firmer sense of the empirical cash <-> wellby <-> sd depression correlations and apropos of your comments perhaps this should be made more explicit in moral weights agendas.)