I have been working on a research project into the scale, tractability and neglectedness of child marriage. After 80 hours of research, I thought that there was a relatively strong case that effective altruist funding organisations that fund projects addressing international poverty should consider funding child marriage interventions. I then came across a study that destabilised the premise that child marriage leads to decreased wellbeing across a number of health metrics. I describe in more detail my experience and findings below, and share some tips for those undertaking self-directed research projects to avoid making the mistakes I made (skip to ‘What I will do next time’ for these).
Context:
I had no direct experience researching child marriage, but I was interested to learn about effective interventions and whether it had potential as a possible cause area. I studied Political Science and International Relations at University, as well as some subjects on development, gender and economics, I have also worked as a government evaluator. My goal was to do some preliminary research and determine if child marriage was large scale problem, tractable and neglected. If so, I would share this research with effective altruist funders.
My model:
In October last year, I started a self-directed research project into the scale, tractability and neglectedness of child marriage. I read and collected dozens of sources, analyzed data, contacted a top researcher, compared effective interventions, built a mental model of what the charitable space looked like and identified potential interventions for EA support.
I came to the following findings, based on around 80 hours of research:
Scale/importance
Child marriage is a widespread practice that affects around 12 millions girls per year (UNICEF, 2022)
Child marriage is a harmful practice that increases the risk of negative maternal and sexual health outcomes, domestic and sexual violence, and reduces the likelihood that a girl will complete school. This is the consensus position held by global development institutions (see meeting report from leading global institutions on child marriage UNFPA, 2019).
Tractability
There are cost effective interventions that work to prevent child marriage, e.g. the ‘cost per marriage averted’ ranged between US$159 and US$732 in this study (Erulkar, Medhin and Weissman 2017).
The effect of child marriage on quality adjusted or disability adjusted life years has not been quantified so it difficult to compare cost effectiveness with other interventions (EA Forum explainer on these metrics).
Neglectedness
Population Council is a research body focussed on running quasi-experimental programs and creating scalable interventions (Population Council, date unknown).
The lead investigator into child marriage at Population Council informed me that it is not currently running programs to prevent child marriage because of lack of funding.
Conclusion
Effective altruist funding organisations focused on international health and poverty should consider funding effective interventions to prevent child marriage at a large scale.
What broke my model
Earlier this week, I decided it would be useful to try and quantify the harm of child marriage, or at least some of the harms, into commonly used metrics like quality adjusted or disability adjusted life years (QALYs or DALYs). I anticipated that this would be a key piece of information for EA funders, and it had not been done so far (finding 2b). In doing so, I came across a study that fundamentally challenged finding 1b: child marriage is an underlying cause of many harmful outcomes. Without strong evidence that child marriage causes harm, the other findings on its tractability and neglectedness are significantly less consequential.
Fan and Koski, 2022 examined the data from 58 studies on the effects of child marriage on a range of health outcomes (see my summary Table 1). They found that while data clearly shows that child marriage increases the risk of physical violence, results from other metrics, including impacts on contraceptive use, maternal health, nutrition and mental wellbeing are mixed (Fan and Koski, 2022). Some studies even showed that child marriage was associated with more positive outcomes, such as higher contraceptive use (Fan and Koski, 2022). It is unclear why results are so mixed, but worth noting that these studies take place over many different countries and cultural contexts across Sub-Saharan Africa and South Asia.
Additionally, Woden et al., 2017 reviewed a series of studies into decision-making power and found that it is unclear whether child marriage leads to lessened decision-making power.
This is not to say that child marriage is not harmful, it is linked to:
Increased risk of physical violence (Fan and Koski, 2022, citing 8 studies).
These metrics also do not measure broader societal effects on gender inequality.
Girls married before the age of 15 are a less studied cohort than girls under 18. The few studies that look at girls married before 15 as a separate group indicate negative impacts. While limited, studies that show that the following health risks are greater for girls married before the age of 15 for:
Table 1 below summarizes the information I have collected on the harms of child marriage so far.
Table 1
Harm
Girls married before the age of 15
Girls married before the age of 18
Sources
Physical violence
Increased risk
Increased risk
Fan and Koski, 2022 consistent results across eight studies. One study showed no effect but this study only looked at a period of three months (Erulkar, 2013).
Sexual violence
Unclear
Unclear
Fan and Koski, 2022, citing seven studies. Some showed increased risk and some showed no effect.
Unwanted pregnancy
Unclear
Unclear
Fan and Koski, 2022, citing seven studies: some found increased risk, some found decreased risk, some found no effect and some found variable effects in different countries.
Girls married before 15 studied separately by only one study. This study found there was increased risk for this group (Kamal, 2013).
Stillbirth and miscarriage
Unclear
Unclear
Fan and Koski, 2022, citing 2 studies. One found increased risk, one found no increased risk for 15-17 year olds but increased risk for girls married before age of 15 (Paul, 2018).
Use of contraception
Unclear
Unclear
Fan and Koski, 2022, citing 15 studies. Some found increased use, some found decreased use, some found no effect. One study showed girls married under 15 less likely to use contraception (Habyarimana F, Ramroop, 2018).
Use of maternal health care
Unclear
Unclear
Fan and Koski, 2022, citing nine studies. Some showed no effect and some showed decreased likelihood of using maternal health care. Consistent across studies was the decreased likelihood of giving birth in a healthcare facility. However, this was not adjusted for the potentially confounding variable that most child marriage occurs in rural areas where health care facilities are often far away.
Nutritional status
Unclear
Unclear
Fan and Koski, 2022, citing six studies. Some found increased likelihood of malnutrition, others found no effect. One study found decreased likelihood of malnutrition. One study found increased likelihood of malnutrition for girls married before 15 but no effect for those married between 15 and 18 (Yimer, 2016).
My updated views on child marriage after doing this research are that:
The causal model that child marriage leads to harmful outcomes, except for the metrics of increased risk of physical violence and decreased school completion, is not stable (see Table 1).
I should direct time into measuring:
How much child marriage leads to physical violence and decreased school years completed, and attempt to quantify this harm through the most appropriate metric (e.g. QALYs or DALYs).
The harms to girls married before the age of 15, and attempt to quantify this harm into the most appropriate metric (e.g. QALYs or DALYs).
If the findings of 2 or 3 are significant, I should research the tractability and neglectedness these problems.
It may be better to look outside of the causal model of child marriage at more specific harms and attempt to address these directly. For example, this study found that no more than 20% of girls who dropped out of school in francophone Africa did so because of marriage or pregnancy (Lloyd and Mensch, 2008).
What I know so far about scale and tractability: marriage before 15
UNICEF estimates that around 5% of women aged 20-24 alive today were married before the age of 15 (UNICEF, 2021). The Population Council demonstrated that providing educational materials to girls at a cost of $20 per girl reduced the risk of girls being married. The adjusted risk ratio was 0.09 (91% decreased chance) with a 95% confidence interval of (0.01, 0.71) or 99% to 29% decreased chance and (significance: p<0.05). However, the results of this intervention were not consistent in Burkina Faso and Tanzania (Erulkar et al., 2017). I plan to review further studies that measure interventions to delay marriage targeted at girls under 15.
What I will do next time
I am glad I undertook this project. I learnt lots of things about child marriage, taught myself some statistical concepts, and things that I will do differently next time to save time and more quickly develop my model.
Assumptions
Before committing time to a research project:
List my underlying assumptions (I likely would have listed 1b)
Search for sources that disagree with these assumptions (I would likely have found Fan and Koski, 2022 within the first 5 hours of doing this project, and disrupted 1b)
Consider whether those assumptions still hold.
Consensus and data
It is important not to automatically trust a position made by large institutions:
As soon as I see a claim being made, no matter which authority is making this claim, trace the claim back to its underlying data (I would have traced UNICEF and UNFPA’s claims about 1b back to their source material).
Thoroughly examine the methods and results sections of studies and come to my own conclusions based on the data before reading any discussion or conclusions (I would have found that 1b was weaker than I anticipated, and worked to refine my research questions).
Find someone to report on my projects to
I think the act of reporting my process and findings to someone else, and having them challenge my assumptions, would probably have led me to challenge my assumptions faster, and saved me lots of time. I will be reaching out to fellow EAs to fulfill a supervisor/challenger role for future self-directed projects. Also, if you have a project, I am happy to play this role for you. My email address is catherinefist@gmail.com.
If you found this post interesting, you may also like:
How science works and how best to read scientific papers
I really enjoyed listening to Spencer Greenberg and Christie Aschwanden speak about how slow and hard determining even the most simple things using scientific methods is in this podcast. They also spoke about how they read scientific papers, which I found useful.
Sarah Hough and Ben Williamson launched the Maternal Health Initiative (MHI) last year through Charity Entrepreneurship. MHI which aims to increase access to family planning in Sub-Saharan Africa. They have a robust theory of change and I am excited to see the impact they make.
Thanks :)
Thanks to RP and TF who reviewed this post for me and gave me wonderful feedback.
What I thought about child marriage as a cause area, and how I’ve changed my mind
Summary:
I have been working on a research project into the scale, tractability and neglectedness of child marriage. After 80 hours of research, I thought that there was a relatively strong case that effective altruist funding organisations that fund projects addressing international poverty should consider funding child marriage interventions. I then came across a study that destabilised the premise that child marriage leads to decreased wellbeing across a number of health metrics. I describe in more detail my experience and findings below, and share some tips for those undertaking self-directed research projects to avoid making the mistakes I made (skip to ‘What I will do next time’ for these).
Context:
I had no direct experience researching child marriage, but I was interested to learn about effective interventions and whether it had potential as a possible cause area. I studied Political Science and International Relations at University, as well as some subjects on development, gender and economics, I have also worked as a government evaluator. My goal was to do some preliminary research and determine if child marriage was large scale problem, tractable and neglected. If so, I would share this research with effective altruist funders.
My model:
In October last year, I started a self-directed research project into the scale, tractability and neglectedness of child marriage. I read and collected dozens of sources, analyzed data, contacted a top researcher, compared effective interventions, built a mental model of what the charitable space looked like and identified potential interventions for EA support.
I came to the following findings, based on around 80 hours of research:
Scale/importance
Child marriage is a widespread practice that affects around 12 millions girls per year (UNICEF, 2022)
Child marriage is a harmful practice that increases the risk of negative maternal and sexual health outcomes, domestic and sexual violence, and reduces the likelihood that a girl will complete school. This is the consensus position held by global development institutions (see meeting report from leading global institutions on child marriage UNFPA, 2019).
Tractability
There are cost effective interventions that work to prevent child marriage, e.g. the ‘cost per marriage averted’ ranged between US$159 and US$732 in this study (Erulkar, Medhin and Weissman 2017).
The effect of child marriage on quality adjusted or disability adjusted life years has not been quantified so it difficult to compare cost effectiveness with other interventions (EA Forum explainer on these metrics).
Neglectedness
Population Council is a research body focussed on running quasi-experimental programs and creating scalable interventions (Population Council, date unknown).
The lead investigator into child marriage at Population Council informed me that it is not currently running programs to prevent child marriage because of lack of funding.
Conclusion
Effective altruist funding organisations focused on international health and poverty should consider funding effective interventions to prevent child marriage at a large scale.
What broke my model
Earlier this week, I decided it would be useful to try and quantify the harm of child marriage, or at least some of the harms, into commonly used metrics like quality adjusted or disability adjusted life years (QALYs or DALYs). I anticipated that this would be a key piece of information for EA funders, and it had not been done so far (finding 2b). In doing so, I came across a study that fundamentally challenged finding 1b: child marriage is an underlying cause of many harmful outcomes. Without strong evidence that child marriage causes harm, the other findings on its tractability and neglectedness are significantly less consequential.
Fan and Koski, 2022 examined the data from 58 studies on the effects of child marriage on a range of health outcomes (see my summary Table 1). They found that while data clearly shows that child marriage increases the risk of physical violence, results from other metrics, including impacts on contraceptive use, maternal health, nutrition and mental wellbeing are mixed (Fan and Koski, 2022). Some studies even showed that child marriage was associated with more positive outcomes, such as higher contraceptive use (Fan and Koski, 2022). It is unclear why results are so mixed, but worth noting that these studies take place over many different countries and cultural contexts across Sub-Saharan Africa and South Asia.
Additionally, Woden et al., 2017 reviewed a series of studies into decision-making power and found that it is unclear whether child marriage leads to lessened decision-making power.
This is not to say that child marriage is not harmful, it is linked to:
Increased risk of physical violence (Fan and Koski, 2022, citing 8 studies).
Lower attendance in school (Delprato et al., 2015, Lloyd and Mensch, 2008, Field and Ambrus, 2008).
These metrics also do not measure broader societal effects on gender inequality.
Girls married before the age of 15 are a less studied cohort than girls under 18. The few studies that look at girls married before 15 as a separate group indicate negative impacts. While limited, studies that show that the following health risks are greater for girls married before the age of 15 for:
Unwanted pregnancy (Kamal, 2013)
Stillbirth and miscarriage (Paul, 2018)
Lower use of contraception (Habyarimana F, Ramroop, 2018)
Pregnancy complications (Paul, 2018)
Malnutrition (Yimer, 2016).
Table 1 below summarizes the information I have collected on the harms of child marriage so far.
Table 1
Fan and Koski, 2022, citing seven studies: some found increased risk, some found decreased risk, some found no effect and some found variable effects in different countries.
Girls married before 15 studied separately by only one study. This study found there was increased risk for this group (Kamal, 2013).
Unclear
Unclear
Updated positions and next steps
My updated views on child marriage after doing this research are that:
The causal model that child marriage leads to harmful outcomes, except for the metrics of increased risk of physical violence and decreased school completion, is not stable (see Table 1).
I should direct time into measuring:
How much child marriage leads to physical violence and decreased school years completed, and attempt to quantify this harm through the most appropriate metric (e.g. QALYs or DALYs).
The harms to girls married before the age of 15, and attempt to quantify this harm into the most appropriate metric (e.g. QALYs or DALYs).
If the findings of 2 or 3 are significant, I should research the tractability and neglectedness these problems.
It may be better to look outside of the causal model of child marriage at more specific harms and attempt to address these directly. For example, this study found that no more than 20% of girls who dropped out of school in francophone Africa did so because of marriage or pregnancy (Lloyd and Mensch, 2008).
What I know so far about scale and tractability: marriage before 15
UNICEF estimates that around 5% of women aged 20-24 alive today were married before the age of 15 (UNICEF, 2021). The Population Council demonstrated that providing educational materials to girls at a cost of $20 per girl reduced the risk of girls being married. The adjusted risk ratio was 0.09 (91% decreased chance) with a 95% confidence interval of (0.01, 0.71) or 99% to 29% decreased chance and (significance: p<0.05). However, the results of this intervention were not consistent in Burkina Faso and Tanzania (Erulkar et al., 2017). I plan to review further studies that measure interventions to delay marriage targeted at girls under 15.
What I will do next time
I am glad I undertook this project. I learnt lots of things about child marriage, taught myself some statistical concepts, and things that I will do differently next time to save time and more quickly develop my model.
Assumptions
Before committing time to a research project:
List my underlying assumptions (I likely would have listed 1b)
Search for sources that disagree with these assumptions (I would likely have found Fan and Koski, 2022 within the first 5 hours of doing this project, and disrupted 1b)
Consider whether those assumptions still hold.
Consensus and data
It is important not to automatically trust a position made by large institutions:
As soon as I see a claim being made, no matter which authority is making this claim, trace the claim back to its underlying data (I would have traced UNICEF and UNFPA’s claims about 1b back to their source material).
Thoroughly examine the methods and results sections of studies and come to my own conclusions based on the data before reading any discussion or conclusions (I would have found that 1b was weaker than I anticipated, and worked to refine my research questions).
Find someone to report on my projects to
I think the act of reporting my process and findings to someone else, and having them challenge my assumptions, would probably have led me to challenge my assumptions faster, and saved me lots of time. I will be reaching out to fellow EAs to fulfill a supervisor/challenger role for future self-directed projects. Also, if you have a project, I am happy to play this role for you. My email address is catherinefist@gmail.com.
If you found this post interesting, you may also like:
How science works and how best to read scientific papers
I really enjoyed listening to Spencer Greenberg and Christie Aschwanden speak about how slow and hard determining even the most simple things using scientific methods is in this podcast. They also spoke about how they read scientific papers, which I found useful.
EA Forum post on violence against women and girls
If you are interested in the broader topic of reducing gender based violence, I recommend this recent post from Akhil: What you can do to help stop violence against women and girls—EA Forum.
A promising maternal health charity start-up
Sarah Hough and Ben Williamson launched the Maternal Health Initiative (MHI) last year through Charity Entrepreneurship. MHI which aims to increase access to family planning in Sub-Saharan Africa. They have a robust theory of change and I am excited to see the impact they make.
Thanks :)
Thanks to RP and TF who reviewed this post for me and gave me wonderful feedback.