Empowering Numbers: FEM since 2021

hhart23 Aug 2023 11:48 UTC
234 points
23 comments10 min readEA link

∼250,000 new contraceptive users, ∼20 million new listeners and up to ∼60x cost effectiveness

TL;DR:

In 2021, FEM launched our pilot: a 3-month family planning radio campaign, educating listeners about maternal health and effective contraception. Our ads and shows played up to 860 times in Kano state in northern Nigeria, reaching ~5.6 million. An independent survey showed that in 11 months, contraceptive use increased by ~75% among all women in the state. Since then, we launched a 9-month campaign, aired proofs of concept in three new locations and re-aired in 8 more states, reaching an estimated 20 million new listeners. We were recommended by Giving What We Can and Founders Pledge, who assess our campaigns as ~22 times as effective as cash transfers. We developed new technologies that will allow us to customise health advice to listeners and run a randomised controlled trial. We continue to build organisational capacity to scale across Nigeria and new cause areas in the years ahead.

Studio host during show recording in Kano

Promising results from our pilot (Oct—Dec 2021)

In Nigeria, 1 in 22 women dies of pregnancy-related complications.

In 2020, this meant ~1,047 mothers died for every 100,000 births. For contrast, in the United Kingdom, there were 9.8 maternal deaths per 100,000 births.

Across Nigeria, about half of women have an unmet need for effective contraception. Increasing access to contraception reduces unwanted pregnancies and increases space between births, making pregnancies much safer. It can save the lives of thousands of women and save others from devastating health conditions such as obstetric fistula, and postpartum anaemia and depression. When women and girls understand and access family planning services, they receive substantial positive effects: get more education, make more money, and enjoy the ability to take better care of their children.

Launched in September 2020, Family Empowerment Media develops evidenced-based radio campaigns to improve maternal and child health by raising awareness about contraception. In our Proof of Concept phase, we chose Kano, partnered with a Nigerian organisation, and conducted a short radio campaign.

In September 2021, on World Contraception Day, we launched a pilot campaign in collaboration with the Nigerian Ministry of Health. We produced radio programmes, including dramas, storytelling shows, Q&A programmes with health experts, and interviews with prominent religious leaders on the topic of childbirth spacing.

Our programmes reached a total of 5.6 million listeners up to 860 times and it looks like they had a substantial impact.

Data from pre-post survey by PMA, intervention period is indicated by the shaded area. Over 11 months, we saw an increase from 10% to 19% among married women and from 8% to 14% among all women.

An external survey on contraceptive use in Kano saw a 75% increase (among all women) between February 2021 and January 2022, corresponding to approximately 250,000 new contraceptive users. This is twice as many new users as over the last 5 years combined.

A similar trend was not seen in Lagos or other Nigerian states, suggesting the change was specific to Kano. We were the only new large scale family planning demand generation activity in the state at that time. We are working with researchers to better understand how much impact can be attributed to FEM’ campaigns. But these results are promising.

New technology to evaluate our impact (Jan -Dec 2022)

FEM is committed to collecting robust, scientific data about the effects of our work. With funding from Grand Challenges Canada (Stars in Global Health), FEM developed a new technology to evaluate our impact: a transmitter that identifies when FEM’s content is on air, and replaces it with music. This means the region surrounding the transmitter is temporarily not exposed to our programmes.

Transmitter technology being tested in Kano State

With this technology, we can randomise listeners into control and treatment groups, enabling randomised controlled trials of radio interventions in regions where this was previously impossible. We plan to launch an RCT of our intervention in early 2024. We will be assisted by a team of academics with diverse expertise in maternal and child health, mass media, and health systems in Nigeria: Maggie McConnell, John Marshall, Belinda Archibong and Dylan Groves.

Among our other innovations, FEM is testing an AI WhatsApp chatbot to facilitate 1:1 follow-up and receive real-time feedback. Listeners send questions, a large language model generates two potential responses, and a trained human operator picks one or writes a custom response. We are testing this as a part of developing our data pipeline. We hope listener feedback will help us maintain an up-to-date understanding of how our material is received. Further, FEM can address popular and generalisable questions in subsequent programs. The WhatsApp source is but one of many datastreams.

Launching our first 9 month campaign (May 2022 - Feb 2023)

In May 2022, we launched a nine-month campaign in Kano state, to an audience base of 5.6 million over 2600 times. In this campaign, we produced a serial drama for the first time – an educational programme lasting eight weeks, following the stories of three families.

These stories were received favourably by men and women in our target population. For this work, our team, including our partner iDevPro-African Institute, received an Award for Excellence from the National Youth Progressive Council, an umbrella organisation for youth (ages 15-35) NGOs and youth organisations across Nigeria.

Throughout our first two years we conducted research with 2,000 individuals, hundreds of hours of desk research, and experimented with different implementation models, landing on one we believe can scale. We also improved on the systems we use to manage campaign development processes.

Our Kano team receives an Award for Excellence from the Youth Progressive Council on the right

Scaling our work: three new proofs of concept (Oct—Dec 2022)

From October to December, we completed proofs of concept in three new Nigerian states: Ondo, Kogi, and Anambra. Here, our programmes reached an estimated 10 million additional listeners. (Our reach data for these locations is less robust than in Kano; we expect to have better estimates in the future.) For proofs of concept, we collaborated with our Nigerian partners to:

  • Hold a stakeholder workshop with over 40 attendees, including Ministry of Health, religious leaders and civil society organisations

  • Hold two focus groups with target listeners

  • Develop 2 ads, based on desk research and focus groups

  • Work with radio station to broadcast campaign in the state for one week

  • Survey 40 target listeners, to assess if people have heard and understood our content

  • We conducted these proofs of concept to test core region-specific assumptions, including partner reliability and stakeholder buy-in. Testing these early prevents waste as we scale.

Despite a flood earlier in the year in Kogi and Anambra, our partners were able to implement the plan. We moved forward with two of these organisations to our formative research phase this year.

Focus group discussion from Anambra

Cost-effectiveness estimates

Charity evaluator Founders Pledge started recommending FEM and offered us a grant from their Global Health & Development Fund in 2022. In a research report on mass media interventions (summarised here), they estimated FEM’s work in northern Nigeria as 22 times as effective as unconditional cash transfers, when considering the impact on maternal health. Based on Founders Pledge’s assessment, Giving What We Can, a nonprofit devoted to highly effective giving, now recommends FEM as a top-rated charity.

Our work also seems promising according to other methods of evaluation. If we enter our reach and cost into a model developed by Givewell, we find our intervention 27 times as effective as cash transfers in the north of Nigeria. In contrast to the models above, Givewell’s model includes more impacts than those on maternal health. It could be that it underestimates our impact on maternal deaths because it is not adjusted for the difference in maternal mortality rates in northern Nigeria vs. other places on which the model was based; Kano has a nearly 4x higher maternal mortality rate than Burkina Faso, for example. This adjustment (which we will make when GiveWell publishes its broken-down moral weights for increased family planning uptake) would likely reveal that our intervention is more effective.

Rethink Priorities recently published this research project on our work suggesting we may be as much as 60 times as cost-effective in northern Nigeria as unrestricted cash transfers. This is highly encouraging.

Meet the FEM team

Since our last update, FEM registered as a 501(c)3 in the US, an Incorporated Trustee in Nigeria and a CIC in the UK, developed strong financial management systems, and made some amazing hires. Our team now consists of 10 ambitious, impact-driven individuals including the heads of our partner organisations:

  • Anna Christina Thorsheim—Executive Director & Co-Founder—co-founded a health-tech startup developing VR-based phobia treatment and worked as Head of PR at a startup incubator for for-profits aiming to achieve Sustainable Development Goals.

  • Tolulope Oyekanmi—Head of Growth—has 10+ years experience implementing and advising reproductive, maternal, and child health projects and worked with the Clinton Health Access Initiative and Population Services International across northern and southern Nigeria.

  • Lia Boldt—Director of Programmes—worked in Cambodia, Ethiopia, Germany, and South Africa, supporting education and health projects at UNESCO and GIZ, and co-founded the nonprofit Sahakom e.V. to enhance global education.

  • Kabiru Abdulkareem—Research and Process Manager—managed field activities at New Incentives, a GiveWell recommended organisation, and supported eHealth Africa in implementing GIS information technology programmes to monitor polio immunisation campaigns.

  • Dr. Tukur Mansur Muhammad—Implementing Partner, northern Nigeria—founded and directs iDevPro-African Institute, has served as a Regional Manager for Marie Stopes in northern Nigeria and become one of Nigeria’s foremost family planning experts, with a PhD in international health.

  • Dr. Charles Umeh—Implementing Partner, southern Nigeria—director of Parkers Mobile Clinic, public health physician and telemedicine expert, member of the International Medical Informatics association and Healthcare Informatics Society of Nigeria.

  • Dr. Ireti Adesida—Implementing Partner, western Nigeria—founder and director of Rhealyz Global Empowerment Initiative ‘Rhealyz Africa,’ also founded Food Gist Hub, an agricultural communication project, and HeForShe.

  • Dr Usman Isyaku—Head of Research—has 10+ years experience advising research projects and training students in Nigeria; he investigated climate change mitigation and adaptation strategies at the University of Bergen and consulted for the World Bank’s Development Impact Evaluation Research Group.

  • Rowan Lund—Operations Manager—supported the operations, community engagement, and public relations of various charities, including Animals Aotearoa and Stop Ecocide.

  • Harmony Hart—Communications Manager—has 10+ years experience in the nonprofit world as a board president, writer, editor, and movement builder, and has founded and cofounded multiple small for-profit businesses.

Other important partners include UNITRONIC (Pavel Strnad, Zuzana Strnadova, Josef Liska) and Freedom Radio (Ibrahim-Jamilu Dahiru).

Key learnings

  • Partnering with Nigerian organisations is highly valuable—We’ve had the opportunity to collaborate with four Nigerian organisations, which has greatly increased our effectiveness, and been mutually enriching. FEM shares capacity and extensive research, and offers training on how to produce evidence-based social and behavioural change campaigns. Our partners provide a wealth of knowledge about Nigeria, operational skills, and help managing stakeholder relationships. They also identify and engage trusted religious leaders, community heads, celebrities and health professionals whom we can spotlight in our radio content.

  • We can re-air content across states—One of our core objectives for 2023 was to test the capacity of our implementing partners to air content in states similar to those where they currently operate. Our partners are based in Kano, Kogi, Ondo, and Anambra. Over the past 6 months, we have also aired content in 7 new states – Enugu, Abia, Delta, Osun, Ogun, Oyo, and Niger – reaching an estimated total of 20-25 million (incl. partners’ states). This successful project aired our content 630 more times and established that our partners can implement regionally not just in their home states.

  • Adjacent interventions also seem cost-effective—We compared the projected cost-effectiveness of family planning programs in other countries to producing radio programmes on different, adjacent issues in Nigeria (including exclusive breast feeding, intimate partner violence, and kangaroo mother care). We can use the knowledge, connections, and infrastructure we’ve already built to produce campaigns in these new areas. We are now more open to expanding across causes, in addition to scaling up our family planning work.

Our biggest mistakes

  • We hired too slowly—FEM depended on a small number of people for a long time. After hiring in 2022 and 2023, we have more stable capacity. We also kept applicants waiting a long time through the hiring process. Since we have resolved this challenge and have more capacity, we expect all candidates to hear back in 1-2 weeks per stage.

  • We didn’t keep you updated—Our blog was last updated a year and a half ago. We’ve heard the question, “Is FEM still active?” The answer is “Yes! More than ever!” We hope to prioritise keeping you more frequently updated.

  • We had some methodological mistakes in our survey—We selected survey participants based on a criteria that fluctuated between baseline and endline surveys, making it difficult for us to be sure we were comparing similar groups. Though we saw a slight increase in contraceptive use in this data too, we don’t trust it that much. In our next roll-out we will work with an external survey firm to evaluate our programmes. We believe the PMA data above is the most reliable data source, indicating our impact.

Our biggest successes

  • We built a great product and strong processes—We believe our product is evidence-based, context specific, and truly empowering. Our content goes through 4 rounds of reviews, including with the Ministry of Health, and our prototypes have received excellent feedback. Our research suggests our content enables access to informed decision-making about births and birth spacing.

  • We secured buy-in and built partnerships—FEM collaborates with a wide range of stakeholders, from Nigerian state Ministries of Health and local religious leaders to engineering and tech institutions. We spotted impactful opportunities, and maintained these collaborations.

  • We hired great people and developed a supportive, cohesive culture—A fast moving organisation, we work together in an encouraging and appreciative way, and reward when people notice their mistakes.

What we’re doing now

We are enabling more women and men to reach their reproductive goals by closing information gaps and spreading the knowledge needed to make informed choices. As we enter the second half of 2023, here’s how we’re doing that:

  • Preparing for the RCT—We are continuing to prepare for the randomised controlled trial of our intervention in three regions in Nigeria, with which we’ll reach an estimated 10-15 million listeners. We estimate this campaign will lead to ∼147,000 new contraceptive users, save ∼345 lives, and avert ∼5,000 unsafe abortions. It will also generate important, robust data about our intervention. We are conducting formative research and producing content across these new regions this year. We have also signed Memorandums of Understanding, formal collaboration agreements, with the state Ministry of Health in Anambra, Ondo and the Primary Health Care Agency in Ondo.

  • Re-airing—Meanwhile, we will re-air our radio content with a different station partner in Kano, to create additional impact in northern Nigeria. We chose Arewa station, which reaches 4.6 million Nigerians, including over 1 million mostly rural women of reproductive age. We expect to avert ∼70 maternal deaths through this campaign.

Our plan for the years ahead

Our three-year plan is ambitious but achievable and could catalyse major improvements in global health and wellbeing.

We anticipate launching our RCT in March 2024.

While running the RCT, we will prepare for multiple outcomes by conducting one pilot in a new cause area, and one family planning pilot in a new country.

Our selection of a new country and intervention will follow the same rigorous process that led us to choose Nigeria and family planning. This includes evidence reviews, cost effectiveness modelling and expert consultation. At this time, it looks particularly promising to work in campaigns covering topics related to perinatal health (exclusive breastfeeding, Kangaroo Mother Care, ANC visits) and child health (malaria, pneumonia and diarrhoea). With regard to countries, we are optimistic about Chad, Niger and DRC. Decisions will be made by November 2023, after additional research and expert consultation.

Completing these three projects by 2026 will enable an evidence-based decision about how FEM can make the greatest positive impact on lives, health, and empowerment. If RCT findings support our current estimates of impact, we will have already overcome many hurdles associated with working in new countries. We will re-air content from our RCT to all cost-effective Nigerian states, potentially reaching 35 million listeners for 23 months.

Above, you can see a map of Nigeria – the colours indicate the states that we can reach in collaboration with different partners (each colour corresponds with a partner). The states in black are too dangerous for us to work.

Our vision as an organisation is to save and improve the maximum number of lives, cost-effectively, at scale. We aim to reach 50 million people by 2030, with cost-effective campaigns across a range of high-impact areas. Radio programmes are cheap to produce. Once made, they can be aired again and again. For relatively little money, we can transform people’s lives for the better by increasing their access to life-changing information.

And you can help!

Support our work

Donate. We can reach 5 million listeners for a minute with $14, and you can sponsor a week’s airtime costs for ads for $980. Our 2023 gap is $342k. You can reach out to hhart@familyempowermentmedia.org for additional donation options.

Volunteer with us short- and long-term.

Give us feedback. We want to hear from you!

Spread the word. Do you have a friend or a colleague who may be interested in our work? Share this post.

Keep updated: Sign up to receive updates, at the bottom of our homepage.

hhart23 Aug 2023 11:48 UTC
234 points
23 comments10 min readEA link