Thanks for your amazing work! I have two questions.
StrongMinds’ work seems to suggest that depression in Uganda and Zambia is at least as treatment-responsive as depression in high-income countries, if not more. This is true despite your intervention being psychotherapeutic rather than improving income directly. How do we reconcile this with the intuition that economic conditions are very important in causing people’s depressive experiences? Should we infer from this that economic conditions are not very important in causing depression?
How does StrongMinds think about the Hawthorne effect—the possibility that clients feel obligated to report improvements in their health even if they don’t feel them? My anecdotal observations of therapy in high-income contexts suggest that people feel like if they aren’t being helped, then they must be the problem, so they report that therapy helps them in order to avoid being a burden. This seems like a challenge for measuring impact.
Haushofer et al., (2020), a trial of both psychotherapy and cash transfers in a LMIC, perform a test ‘experimenter demand effect’, where they explicitly state to the participants whether they expect the research to have a positive or negative effect on the outcome in question. We take it this would generate the maximum effect, as participants would know (rather than have to guess) what the experimenter would like to hear. Haushofer et al., (2020), found no impact of explicitly stating that they expected the intervention to increase (or decrease) self-reports of depression. The results were non-significant and close to zero (n = 1,545). We take this research to suggest social desirability bias is not a major issue with psychotherapy. Moreover, it’s unclear why, if there were a social desirability bias, it would be proportionally more acute for psychotherapy than other interventions. Further tests of experimenter demand effects would be welcome.
Other less relevant evidence of experimenter demand effects finds that it results in effects that are small or close to zero. Bandiera et al., (n =5966; 2020) studied a trial that attempted to improve the human capital of women in Uganda. They found that experimenter demand effects were close to zero. In an online experiment Mummolo & Peterson, (2019) found that “Even financial incentives to respond in line with researcher expectations fail to consistently induce demand effects.” Finally, in de Quidt et al., (2018) while they find experimenter demand effects they conclude by saying “Across eleven canonical experimental tasks we … find modest responses to demand manipulations that explicitly signal the researcher’s hypothesis… We argue that these treatments reasonably bound the magnitude of demand in typical experiments, so our … findings give cause for optimism.”
The experimenter demand test is quite reassuring! Although I disagree with the rest of that section, since I didn’t have in mind a conventional social desirability bias. I disagree with the idea that psychotherapy is no different from other interventions in this regard—anecdotally, depressed people are much more sensitive than average to feeling like a burden and not wanting other people to worry about their problems.
Thanks for your kind words and for the great questions!
The relationship between poverty and mental health disorders, including depression, is complex. Poverty exacerbates depression, and depression, alongside other mental health disorders, can drive people further into poverty through reduced productivity, decreased income, and isolation. (Lund et al. 2011) But it is incorrect to conclude that poverty on its own causes depression. If it did, we would see rates of depression at 100% rather than 25% in the slums of Kampala. Our model looks at four distinct triggers for depression: grief, life changes, loneliness/isolation, and conflict. Interestingly, we see from our well-being indicators that many times, restoring a woman’s mental health can increase her productivity and livelihood; for instance, 16% of our former clients reported increases in work attendance after the conclusion of therapy.
Yes, in self-reported data, we have to be very careful to ensure we measure correctly and there aren’t existing biases or misrepresentations. We work directly with an external firm to help with this. They conduct our endline evaluations. As mentioned above, we also look at well-being indicators to help demonstrate our impact, such as a 13% increase in family food security, a 30% increase in school attendance, and a 28% increase in women feeling socially connected.
Poverty exacerbates depression, and depression, alongside other mental health disorders, can drive people further into poverty through reduced productivity, decreased income, and isolation. (Lund et al. 2011) But it is incorrect to conclude that poverty on its own causes depression. If it did, we would see rates of depression at 100% rather than 25% in the slums of Kampala.
What I was wondering about was not the prevalence of depression, but rather how treatable it is. For the reasons you described of poverty exacerbating mental health, it seems like depression should be much less treatable in Kampala than in the US. Yet SM’s success rate indicates that depression is at least as treatable and possibly more treatable than depression in the US. Those are the two things that I struggle to reconcile.
Measurement of other well-being indicators is a great corroboration of your results.
I agree that based on all of the barriers and life challenges that it seems like it would be harder to treat depression in Kampala. As mentioned in some of the other answers, I think the model itself, being culturally appropriate and community-centered, as well as the interpersonal bonds, play a significant role in the program’s success. Also, the fact that we work where the women are, in refugee camps, slums, or schools, to deliver our services.
Thanks for your amazing work! I have two questions.
StrongMinds’ work seems to suggest that depression in Uganda and Zambia is at least as treatment-responsive as depression in high-income countries, if not more. This is true despite your intervention being psychotherapeutic rather than improving income directly. How do we reconcile this with the intuition that economic conditions are very important in causing people’s depressive experiences? Should we infer from this that economic conditions are not very important in causing depression?
How does StrongMinds think about the Hawthorne effect—the possibility that clients feel obligated to report improvements in their health even if they don’t feel them? My anecdotal observations of therapy in high-income contexts suggest that people feel like if they aren’t being helped, then they must be the problem, so they report that therapy helps them in order to avoid being a burden. This seems like a challenge for measuring impact.
On (2): Here’s the section on social desirability bias from HLI’s cost-effectiveness analysis.
The experimenter demand test is quite reassuring! Although I disagree with the rest of that section, since I didn’t have in mind a conventional social desirability bias. I disagree with the idea that psychotherapy is no different from other interventions in this regard—anecdotally, depressed people are much more sensitive than average to feeling like a burden and not wanting other people to worry about their problems.
Thanks for your kind words and for the great questions!
The relationship between poverty and mental health disorders, including depression, is complex. Poverty exacerbates depression, and depression, alongside other mental health disorders, can drive people further into poverty through reduced productivity, decreased income, and isolation. (Lund et al. 2011) But it is incorrect to conclude that poverty on its own causes depression. If it did, we would see rates of depression at 100% rather than 25% in the slums of Kampala. Our model looks at four distinct triggers for depression: grief, life changes, loneliness/isolation, and conflict. Interestingly, we see from our well-being indicators that many times, restoring a woman’s mental health can increase her productivity and livelihood; for instance, 16% of our former clients reported increases in work attendance after the conclusion of therapy.
Yes, in self-reported data, we have to be very careful to ensure we measure correctly and there aren’t existing biases or misrepresentations. We work directly with an external firm to help with this. They conduct our endline evaluations. As mentioned above, we also look at well-being indicators to help demonstrate our impact, such as a 13% increase in family food security, a 30% increase in school attendance, and a 28% increase in women feeling socially connected.
What I was wondering about was not the prevalence of depression, but rather how treatable it is. For the reasons you described of poverty exacerbating mental health, it seems like depression should be much less treatable in Kampala than in the US. Yet SM’s success rate indicates that depression is at least as treatable and possibly more treatable than depression in the US. Those are the two things that I struggle to reconcile.
Measurement of other well-being indicators is a great corroboration of your results.
I agree that based on all of the barriers and life challenges that it seems like it would be harder to treat depression in Kampala. As mentioned in some of the other answers, I think the model itself, being culturally appropriate and community-centered, as well as the interpersonal bonds, play a significant role in the program’s success. Also, the fact that we work where the women are, in refugee camps, slums, or schools, to deliver our services.