Wow Max, this is super impressive. I really appreciate how clearly you surfaced the assumptions and made the sensitivity analysis explorable. Making the counterfactuals explicit is enormously valuable.
Having worked in northern Nigeria, one thing that stood out to me is how dynamic those counterfactuals can be in practice. For example, in Sokoto and Zamfara, DHS coverage numbers capture the endpoint, but underneath that you have shifting factors like outreach consistency, staffing, supply reliability, and community trust. I have seen system performance change meaningfully over relatively short periods in ways that would materially affect those parameters.
It also made me think about places like Kenya and Mozambique that are highlighted as “best” countries in your table. Even within the same country, conditions can vary enormously across regions and over time depending on implementation strength and system capacity. Those differences do not always show up immediately in the underlying data, but they can have real implications for how stable those cost effectiveness estimates are.
Curious how you think about parameter stability over time in settings where the system itself is evolving. The model makes the tradeoffs legible, but the inputs themselves can be moving targets.
Thanks Benita, really appreciate the field perspective. You’re right that the parameters are a snapshot — the tool takes GiveWell’s November 2025 spreadsheet values as given and doesn’t attempt to model how they change over time. GiveWell updates their spreadsheets periodically as they get new data, and the tool would need to be re-extracted to reflect that.
On within-country variation, this is a real limitation. The model treats each country as a single unit with one set of parameters, but as you note, conditions in Sokoto vs. other parts of Nigeria can be very different. The sensitivity analysis helps show how much the result depends on any single parameter (like counterfactual coverage), but it doesn’t capture the kind of correlated shifts you’re describing where multiple parameters move together as systems evolve.
I built this as a side project to make GiveWell’s existing estimates more explorable, not to improve on their parameter estimation — that’s solidly their domain expertise. But the tool does make it easy to test “what if coverage in Zamfara drops by 10%” type questions, which I think is part of what you’re getting at.
Wow Max, this is super impressive. I really appreciate how clearly you surfaced the assumptions and made the sensitivity analysis explorable. Making the counterfactuals explicit is enormously valuable.
Having worked in northern Nigeria, one thing that stood out to me is how dynamic those counterfactuals can be in practice. For example, in Sokoto and Zamfara, DHS coverage numbers capture the endpoint, but underneath that you have shifting factors like outreach consistency, staffing, supply reliability, and community trust. I have seen system performance change meaningfully over relatively short periods in ways that would materially affect those parameters.
It also made me think about places like Kenya and Mozambique that are highlighted as “best” countries in your table. Even within the same country, conditions can vary enormously across regions and over time depending on implementation strength and system capacity. Those differences do not always show up immediately in the underlying data, but they can have real implications for how stable those cost effectiveness estimates are.
Curious how you think about parameter stability over time in settings where the system itself is evolving. The model makes the tradeoffs legible, but the inputs themselves can be moving targets.
Really thoughtful contribution.
Thanks Benita, really appreciate the field perspective. You’re right that the parameters are a snapshot — the tool takes GiveWell’s November 2025 spreadsheet values as given and doesn’t attempt to model how they change over time. GiveWell updates their spreadsheets periodically as they get new data, and the tool would need to be re-extracted to reflect that.
On within-country variation, this is a real limitation. The model treats each country as a single unit with one set of parameters, but as you note, conditions in Sokoto vs. other parts of Nigeria can be very different. The sensitivity analysis helps show how much the result depends on any single parameter (like counterfactual coverage), but it doesn’t capture the kind of correlated shifts you’re describing where multiple parameters move together as systems evolve.
I built this as a side project to make GiveWell’s existing estimates more explorable, not to improve on their parameter estimation — that’s solidly their domain expertise. But the tool does make it easy to test “what if coverage in Zamfara drops by 10%” type questions, which I think is part of what you’re getting at.