Requesting feedback: proposal for a new nonprofit with potential to become highly effective
TL;DR
I make the case for funding a new initiative that addresses an important, tractable, and neglected problem with potential to reach GiveWell-level cost-effectiveness. Please review this and share your thoughts!
Millions of people die every year from preventable conditions, especially in low- and middle-income countries (LMICs).
Some of these deaths can be avoided by improving health workers’ skills and knowledge.
Working with a nonprofit, I’ve been piloting a way to train health workers online with a scrappy WordPress-based MVP, and still getting good results.
I’m looking for funding to create and scale a solution that’s 10X better than anything that exists.
Why I’m sharing this
I’m pretty new to EA and I’d like to submit a version of this to EA organizations for funding. I’m seeking feedback to help make this as clear, logical, and succinct as possible.
Importance: what is the scale of problems that can be addressed with health worker training?
Every year, there are tens of millions of deaths that can be averted with simple interventions, including:
2.9 million deaths of newborn babies readily preventable with standard care,
1.5 million deaths caused by diarrheal diseases; many of which can be averted with improved sanitation practices and low-cost treatment,
2.5 million deaths caused by lower respiratory infections; some of which can be prevented with appropriate diagnosis and treatment, and
10.8 million deaths caused by high blood pressure, which is inexpensively controlled with generic medications.
Pandemics are an equally important global problem. COVID-19 has caused an estimated 17.5 million deaths. Unfortunately, experts agree that COVID-19 isn’t a worst-case scenario: future pandemics could pose an existential threat to humanity.
These problems and their causes have an uneven global distribution. Newborn deaths, lower respiratory infections, and diarrheal disease, in particular, are concentrated in LMICs. Moreover, for a variety of reasons, LMICs are particularly susceptible to epidemics. Solutions to these problems must be implemented in LMICs.
Tractability: how can online health worker training solve these problems?
There is ample evidence that health worker (HW) training can have benefits on its own or when combined with other interventions.
My analysis of existing research studies shows that training HWs to properly care for newborn babies is likely to be highly cost-effective, with an average cost of $59 per DALY averted ($100 per DALY averted is sometimes cited as a benchmark for highly effective interventions). Please see the full forum post for the details of this analysis.
Management of childhood illness training, which covers simple interventions (such as appropriately diagnosing and treating lower respiratory infections and diarrhea), measurably improves HW clinical skills.
A recent study estimated that properly training HWs and giving them personal protective equipment would have saved $7.2 billion in just the first six months of the COVID-19 pandemic.
The decisions that HWs make early in a disease outbreak can save countless lives. Notably, a doctor who acted quickly to quarantine a sick traveler is credited with stopping the spread of Ebola into Nigeria in 2014. HWs need to be well trained to be ready to make these critical decisions at any time.
There is also emerging evidence that online learning is an effective way to train HWs.
Online neonatal care training leads to significant knowledge and skills gains among HWs.
The infection control training I’ve been working on has high completion rates, learning gains comparable to those seen in more resource-intensive in-person training, and very positive learner feedback.
Recent research shows that a short online training in blood pressure measurement improves HWs’ clinical skills.
Modeling indicates that online HW training could be highly cost-effective. Even if online training is much less effective than in-person training on a per-HW basis, its potential for massive scale could still make it a “best buy.”
Finally, training can be used to address many different problems—it’s a cross-cutting solution. For example, a single platform could provide high-quality training on management of childhood illness and newborn care, while also rapidly updating HWs with the latest guidance on emerging pathogens with pandemic potential.
Neglectedness: what gaps could this solution fill?
Most aspects of health workforce development require greater attention and investment.
As of 2013, there was a global shortage of 17.4 million HWs, with a shortfall of 4.2 million HWs in Africa alone.
A lack of essential skills and knowledge is a persistent but understudied problem in the health workforce. The available evidence paints a grim picture. For example:
in a referral hospital in Nigeria, over 70% of HWs have poor knowledge of blood pressure measurement,
HWs caring for sick children in four African countries had very low rates of adherence to evidence-based protocols for diagnosing the most life-threatening conditions (though adherence was higher among those who were recently trained), and
HWs who take our infection control course enter with low baseline knowledge in the subject.
Gaps in skills and knowledge are associated with insufficient training. For example, a recent study found that only ~25% of low-income countries surveyed had ongoing infection control training for HWs who are in the workforce.
Global investment in HW training is fragmented, which prevents the most promising interventions (such as digital HW training) from moving past pilot stages and realizing economies of scale.
A lack of high-quality research on HW training, particularly in LMICs, prevents us from even developing a complete understanding of the problem.
More generally, I’ve searched extensively for high-quality online training targeted to HWs in LMICs. There are some great resources, but there is no “Duolingo for health care,” and I believe the world is ready for that. The time is right, given ongoing massive growth in mobile device ownership and internet access in many of the countries with the highest burden of preventable disease.
Progress to date
Since April 2021, working with the nonprofit Resolve to Save Lives, I have been creating and delivering mobile-first, simple, engaging, and effective online HW training. We currently use a WordPress-based MVP to deliver the training.
The learning process is inspired by Duolingo and based on best practices from education research. HWs learn by responding to questions and receiving feedback in short modules that present realistic clinical cases.
We work closely with governments to offer the training in existing health facilities, so we don’t incur high costs of setting up new hospitals and health centers.
We’ve already had thousands of HWs from across Africa enroll in courses.
Results of an initial pilot of an infection control course were published in BMC Public Health.
Completion rates of a follow-up pilot reached 65%, more than 10 times higher than the average completion rates of free, optional, self-paced courses taken in LMICs.
There were substantial and significant knowledge gains among HWs who completed the infection control course.
Funding request and next steps
[Note: this section has been updated based on feedback; we make a small-scale funding request for a pilot, rather than a larger request for full project implementation.]
We seek funding of $250,000 for 12 months. We’ll use this catalytic funding to run a small-scale pilot, reaching >1,000 HWs in 1-2 countries in Africa. In this pilot, we will assess the feasibility of the solution and measure short-term outcomes (learning gains, completion, and learner feedback). Our goal is to use a lean startup approach to learn as much as possible and accelerate toward a larger-scale, rigorous (and more costly) evaluation of the health impacts of the intervention.
We will start with courses in English that align with international (World Health Organization) guidance, and launch in anglophone Africa before expanding to additional regions and languages. For the foreseeable future, we will focus on HWs who work in primary care—including doctors, nurses, midwives, and community health workers—who form the backbone of the health system and most frequently diagnose and treat the preventable diseases discussed in this piece.
Contact me for more detailed information such as a budget and key milestones.
Addressing skepticism and frequently asked questions
This project is clearly not risk-free. HW training is not as well-studied as other interventions such as the GiveWell top charities. Here are some questions I’ve fielded about this project, along with my answers.
There are a lot of other things that contribute to these preventable deaths, such as a lack of access to medications and equipment. Aren’t those more important problems? How can we be sure that HW training is the right solution to invest in?
It’s absolutely true that this problem has many root causes. I’m not making the broad claim that training will put an end to readily-preventable deaths. Instead, I’m making the more narrow claim that it is a cost-effective way to avert some of these deaths. Most health systems strengthening activities are quite expensive and don’t scale particularly well; online HW training merits more attention because it looks like an exception to this rule.
Isn’t it presumptuous to state that a lack of HW knowledge or skills leads to deaths? Don’t we need to prove that before we invest a lot in training interventions?
As discussed above, the best available evidence on this question comes from training HWs in newborn care, where several different studies have shown that interventions cost-effectively save lives. These studies support the claim that inadequate training contributes to unnecessary deaths. If this project is funded, we would be able to begin to develop a better understanding of whether this is true of topic areas other than newborn care.
Shouldn’t you just focus on training HWs in newborn care, since that’s the topic with the best cost-effectiveness data?
This sounds perfectly reasonable but I don’t think it accounts for how educational technology products achieve widespread adoption in the real world. To me, this is a bit like suggesting that Netflix remove all of its other programming to focus on Stranger Things. The most widely used online learning platforms all offer a large amount of content (or let users contribute their own content). While newborn care training is the first course on our roadmap, I think it would be very difficult to maximize impact without a critical mass of content. Creating additional courses is not nearly as expensive as building a great learning platform and acquiring learners. If we can create a trusted, engaging learning resource with valuable content covering many different conditions, it will be much more valuable to the HWs themselves than a solution focused on just one cause of death.
Why not just choose existing training tools and interventions and point HWs to those?
Simply put, because most existing solutions are not designed for HWs in LMICs, and there isn’t much evidence that they work for training this audience. For example, the World Health Organization puts out guidance on topics ranging from newborn care to infection control for COVID-19. This guidance is packaged in lengthy and dense PDFs. It’s not realistic to expect frontline HWs to wade through all of this guidance to find the few paragraphs or pages in each document relevant to their work. Even though we’ve run pilots on a shoestring so far, the user-centered approach we are taking is already yielding better results than those existing solutions.
I don’t think online learning works. In fact, I think it’s terrible. How can you justify more investment in it?
Some hesitation about online learning is certainly warranted because there’s a lot of bad online training out there. If you are skeptical about online learning in general, I’d recommend my previous writing that analyzes the challenges we face and solutions we can leverage to create effective online HW training. To summarize, solutions have to be carefully designed for efficacy and tailored to the learning needs of the target audience. It’s not surprising that badly designed online learning doesn’t deliver!
Shouldn’t we invest in the pipeline of new HWs rather than training existing HWs?
Training new HWs is expensive, resource-intensive, and heavily regulated. I know this from my first-hand experience in helping to launch a medical school in Rwanda. Continuing education has a better landscape for innovation: the barriers to entry are lower, accreditation is simpler, and the feedback cycles are shorter. Although there’s a great need in both areas, there’s no reason that this innovation can’t get started in the continuing education space and subsequently contribute to improving and simplifying the process of training new HWs.
How does your argument account for uncertainty?
For a more in-depth discussion of some limitations and cost-effectiveness modeling that accounts for different levels of efficacy of online training, I would refer readers to an earlier post I wrote on this topic.
Expressing gratitude
I’d like to thank Alex Chalk and an anonymous member of the EA community for providing great feedback on early drafts of this piece. I contacted Holly Kristensen after seeing this post and also received great feedback from her. The ideas (and any errors) in this piece are mine and do not necessarily reflect the opinions of these generous reviewers. I’m extremely grateful to the team at Resolve to Save Lives for supporting this work in a number of critical ways.
Thanks for this! It’s exciting to see people try out new things and experiment with ways to improve the world.
Some thoughts on the information you present in this post, and what it seems to be missing:
Based on the information presented in this post alone, I’m not confident in the theory of change you present, specifically the (crucial) link between “training HW’s” → “reduction in deaths”, or your team’s plans to measure the impact of training on:
The actual change in HW behavior (e.g. they are now giving better advice and treatment in real life / on the field)
Whether the change in HW behavior actually saves lives (e.g. the improvement in advice/treatment leads to X lives saved)
[EDIT, Sep 13 2022: The paragraph after this one is wrong, see the excerpt below where Marshall links to the post about neonatal care, which includes studies of this kind of intervention directly measuring deaths averted.)
I still think my points apply to Marshall’s teams specific implementation of the intervention.
[End Edit]
The closest I found in this post was a discussion on evidence related to skill gains & knowledge retention. None of the following mention either a or b.
The second study you cite doesn’t mention actual skills change, which seems more important than knowledge retention. The third study you cite is the one above in a US context, and I don’t know how much this evidence transfers to an LMIC context.
However, you mention a lack of evidence in a previous post “New cause area: training health workers to prevent newborn deaths” you mention that:
2. Based on this post, it’s not clear to me how you / your team is prioritizing which countries or regions to operate in, which health problems to work on first, and why. Based on what you wrote in the previous post, the next steps you outlined seemed very promising, and plausibly neonatal care could be good, but this is not mentioned in this post.
3. Becaus the funding amount is quite large ($4 million), it would be good to explain why you think this is the right amount to ask for right now. Based on my knowledge of early-stage global health startup NGOs, I’d expect that you could probably create a strong case for impact (e.g. run an RCT) and create a strong case for impact for somewhere in the range of USD $100,000-$500,000 (very very rough estimate, someone please correct me if this is way off!).
Thanks for this feedback! This is exactly why I posted, so before I provide any specific responses to your points, please know that I appreciate all of the questions and suggestions and I’m already thinking of how they could be addressed in a future version of this proposal.
1. I appreciate your point that the key step in the theory of change is not clear—and I think this is not due to a gap in the data itself but instead due to a gap in my presentation of the evidence. The key supporting evidence is linked out from this statement:
The linked post cites six studies that show reductions in mortality due to HW training. While there are remaining reasons for skepticism, I think these six studies support this key step in the theory of change, at least for some types of training. Regarding your sub-points on point (1), I accept the feedback that we can and should provide more detail on the evaluation in a future version of this. The six studies provide pretty clear guidance on the type of data we would collect.
2. I agree that a roadmap of regions / countries / priority courses would be helpful to include and can add this to a future version. Thanks for the suggestion. We’d want to start with topics that have the strongest existing evidence base (such as neonatal care and management of childhood illness).
3. The dollar amount may seem high, but this is a technology development project. I think it will be very difficult to build a truly excellent learning platform that is tailored to this target audience without attracting top engineering talent, and that gets expensive. As I mentioned in the post, we’ve already done substantial piloting on a shoestring and I plan to continue to do that! I’ll think further about whether we can present a tiered approach, with additional pilots done with an MVP.
Thanks for responding!
I missed that link! Thanks for flagging. I think when I read that, it wasn’t clear to me that this study had explicit examples of reduction in mortality. I’ll edit my first comment so that people know it was included.
That makes sense, and I think it could make a big difference to e.g. potential funders reading this to be more clear.
I think the thing I would see as most important is demonstrating that your specific implementation of the solution results in deaths averted, and this could be done for a lower cost. At that point, if there is evidence, it makes sense to scale up / professionalize the platform.
I like the post and your hands-on attitude!
May I suggest to change the title to something more descriptive?
Eg “Requesting feedback: funding proposal for remote training of healthcare workers in LMICs”
Also: it’s generally considered bad form to claim your organisation is highly effective, especially in such an early stage.. better to use the phrase “potentially highly effective”
Many thanks for reading and for your suggestions, which I’ve acted on! The title is now updated :).
I’d also recommend putting “funding” in the title to make potential funders more likely to read this
Thank you for sharing this project. It looks great. A few minor comments and ideas. Wordpress is very flexible but requires lots of plugins to interface with each other for many functions to work. Consider chatting to Aqeel or JJ Hepburn from Sangro/AI Safety Support who recently used wordpress for a learning management system to see how they found it. Consider also using an existing platform with more pre-built features (e.g., Thinkific) where cross-compatibility might be less painful (see our uni EA fellowship site). At least at the start, these help projects like this get off the ground more easily. Most projects add their bells and whistles later.
My PhD student is doing a thesis very close to this project. She’s trying to accelerate knowledge translation in developing countries. Our hypothesis, like yours, is that online learning will rapidly and cost-effectively close the research-practice gap.
The first study in her thesis is a systematic review of randomised trials using online learning in healthcare. We want to know how well online learning teaches professionals, and how well the training helps people translate it into practice. She’s aiming to find what features help the interventions work better. If you’re interested in the review, she’s looking for team members. Being a team member means you learn the results much more quickly and become an author on the paper, which can be good for credibility. If you want to find out more, email me at noetel [at] gmail.com or send me a message on the forum.
Her second study is a cost-effectiveness analysis of an online nursing intervention. Her third study is a series of interviews in LMICs to see how professionals from those countries feel about online learning. It sounds pretty well aligned with the kind of scoping your team is doing. If you’d be interested in the findings of a study like that, and possibly have some contacts from healthcare in LMICs, then again we’d be interested in collaborating (email me). She could run the interviews but you might find the results valuable.
What is the name and do you have a website for your project?
Thanks! I have a few possible names but haven’t picked one (and the associated website domain name) yet. The pilots described here recently wrapped up but I’d be happy to share a demo module hosted on our MVP that’s focused on neonatal / child health. Please DM me if you’re interested.