CEO of Fortify Health, Mulago and Jacobs Fellow, Ex-IDinsight and Management Consulting.
I lead Fortify Health, a GiveWell, Coefficient Giving and Founder’s Pledge supported non-profit dedicated to reducing and preventing iron-deficiency anaemia. I love thinking about how to scale impactful, evidence-based, cost-effective interventions to alleviate poverty.
Hi NOVAH team,
I have a lot of admiration for your transparency and public commitment. In general, I think this kind of public commitment and pre-analysis planning for key evaluation results is laudable and pushes the sector towards evidence-based decision making.
That said, I do have some concerns about setting a pre-commitment on an effect size from an RCT, particularly at such an early stage of an intervention’s maturity. How are you thinking about mitigating the risk of a null result? I understand that in that scenario you’d pause for a few months and conduct exploratory research. But would there not be an opportunity, at earlier stages of your theory of change, to better test the inputs-to-outputs and outputs-to-outcomes links through things like:
simple observational studies
mixed methods
process evaluations, run in parallel with or prior to the RCT?
I’ll put my biases on the table. I think RCTs are incredibly useful tools, but they’re often not the most helpful tools for understanding the complex theory of change behind an intervention. They give you a single output, usually an effect size, that can be hard to disentangle and act on operationally, especially if that effect size turns out to be null.
Hi Sai—Thanks for doing this. I’d love your meta-level perspective on building operations functions for rapidly scaling organizations, given what GiveWell has built. (Disclosure: I lead an organization that receives GiveWell funding.)
A few related questions:
As you’ve built out GiveWell’s operational scaffolding to support scaling grant-making, what heuristics would generalize to other organizations? Are there specific roles or seniority levels you’d recommend prioritize building first and why?
At what stage of growth would you suggest an organization start investing in operations capacity, relative to investing further in client- or beneficiary-facing functions?
Do you have any rules of thumb on operations team size, for example operations headcount as a share of total headcount, either as a current benchmark or a target as you scale further?
Many organizations in the sector, including ours, are thinking hard right now about how to build absorptive capacity for scale. Since so much of this comes down to people and structures rather than just funding, I think there’s a lot to learn from a more mature operations function like GiveWell’s. Any general advice would be great!
I believe that giving to for-profits to drive cost-effective impact makes a huge amount of sense. Within the broader development sector, we’ve seen the preponderance of “venture philanthropy” increasing over recent years. Some canonical examples include LGT Venture Philanthropy, Mulago Foundation, Draper Richard Kaplan Foundation, and more who are increasingly supporting social enterprises that provide poverty alleviation through for-profit social enterprises.
I would be interested to learn more about how these venture philanthropies model out cost-effective impact of their work within a model of providing equity with no expectation of receipt but ownership of the firm, or debt with a certain rates of payback.
It seems imminently feasible that there are a wide range of interventions that could generate revenue and be highly impactful, justifying debt and/or equity investment to a for-profit entity. What seems less clear to me and less clear from this article are how to model out the impacts of such funding in a way that allows for easy comparison with philanthropic giving.
Hi Habiba, It is amazing what you have been able to build with the team at Spiro, and wishing you all the continued success, especially at this exciting moment as you have government interest and engagement.
I would be interested to learn more about how you are thinking about the scale strategy for Spiro. On the one hand, it appears that there is some level of demand from government as a potential doer at scale, and perhaps slightly less so as a payer at scale. On the flip side, it appears that we’re seeking funding to directly cover the costs of serving all of the proposed districts within Sindh.
Are you foreseeing this as a stepping stone to building a model that could scale directly through government engagement and support, where they may be actually willing to pay and do this intervention directly? Or do you feel like the most likely path to scale would be through directly providing this intervention support, both within Sindh and other districts in Pakistan, or perhaps in other locations around the world?
What are some of the key uncertainties or pieces of knowledge you would want to gather as an early-stage organization to make better determinations on what type of scale strategy would be appropriate?
This is terribly thought-provoking, and for me it raises questions about some of the fundamental axioms of macroeconomics and development. One thing that strikes me as paradoxical is that manufacturing-led growth in low- and middle-income countries has always been premised on rising demand from importing countries. I’d be curious to understand the extent to which demand for physical goods (and the ability to pay for them) is a lagging indicator relative to the production of those goods.
I should flag that I’m not a macroeconomist, so I’m reasoning from first principles rather than evidence here. My sense is that there’s a lag, but not a massive one—recent fuel price shocks suggest supply responds to price signals fairly quickly. It feels reasonable that a corollary would hold on the demand side: as the number of people in a market with jobs (and therefore purchasing power) falls, demand for manufactured goods would fall with it, in something approaching a linear relationship. There are only so many textiles, electronics, white goods and houses any one person can consume. Beyond a certain point, equitable growth and wage growth seem necessary to sustain manufacturing output at all.
If the indicator you’ve shared is genuinely as lagging as it appears, that may present a short-term window for countries like Bangladesh and India. If not, we may be on the verge of something truly frightening for countries that see export manufacturing as their path out of poverty and towards social mobility.
I look forward to reading your next posts—or perhaps I don’t. This all feels quite scary. Please do push back if you think I’m misreading the demand-side mechanics here.
Hi Sjir—thanks for laying this out so clearly, and for continuing to push on evaluator infrastructure rather than just funder mobilisation.
I agree with the core argument: the constraint was never going to be whether there’s $50B worth of compelling initiatives, it’s going to be whether we can build a marketplace that channels the money toward genuine impact rather than the vanity metrics both traditional philanthropy and impact investing optimised for. And I think you’re right that the four gaps you flag (standards/governance, transparency, coverage, evaluator accountability) are the right places to focus, precisely because the ecosystem is still early enough that fixing them now is much cheaper than fixing them after $50B has already arrived and calcified the wrong incentives.
Where I’d add a note of caution, and this is something I’ve been chewing on since some conversations at Skoll: the same dynamic you diagnose in impact investing can show up inside the effective giving ecosystem too, just one layer down. As evaluators standardise their criteria and get more transparent about what they reward, cost-effectiveness estimates, RCT-backed evidence, clear theories of change, charities have a growing incentive to professionalise their communications around exactly those criteria. Some of that is genuinely good: it pushes organisations toward better measurement and more honest reporting. But I’m increasingly seeing charities get fluent in “evaluator-speak” in ways that shape how a programme is presented more than how it’s actually run.
So I’d frame the challenge as: as we build out the standards, transparency and accountability mechanisms you list, we should be explicit that their target is the underlying impact, not a charity’s ability to articulate it convincingly. Something that I think is fantastic and can help solve for this is GWWC’s past evaluations of evaluators as an example (just a vote for continuing this, although I know the team is already extremely busy :)
From your experience, are there systematic or structural differences in the effective altruism and evidence-based development communities in Australia compared to other high-income Western countries? If so, what do you think drives those differences, and how has your approach to communications and engagement within Australia adapted as a result?
My hypothesis is that Australia’s relative geographic isolation, and perhaps certain historic patterns of broader insularity, may mean that global development and health topics require a different kind of conversation to build empathy for places that feel more distant. This is speculative, though, so I’d genuinely welcome your perspective.
Thanks Christophe for sharing these thoughts. I would flip this discussion somewhat, as you note (and which I agree with).
“I’ve recently talked with a charity founder: Their cost effectiveness was lower than that of their “competitor” because their competitor paid themselves a lower salary. This is a race to the bottom that is unhealthy for the movement. The problem is, of course, that this can’t be done by one charity alone. This would negatively impact their cost effectiveness when compared to others. … Best thing to try probably is for large donors to start requiring this from their charities and thereby change the standard. For that the donors need to accept that their reported impact will go down.”
The example you raise is a good one, but for me it points to an inefficiency in how donors measure effectiveness rather than in how organisations set salaries. Within a standard cost-effectiveness calculation, there’s a direct incentive for organisations to underpay senior staff, particularly founders. For smaller organisations with budgets in the range of $300-500k, two co-founders paying themselves a living wage of $30-40k instead of a market rate of $80-100k generates savings of $80-140k. That’s a reduction in total costs of roughly 25%, which inflates reported cost-effectiveness by a corresponding amount, entirely artificially.
This creates three real problems. First, it distorts comparisons between organisations at the early stage, potentially directing funding toward interventions that look cost-effective partly because their founders are financially sacrificing. Second, and relatedly, it penalises organisations that pay fair salaries, which is the race to the bottom you’re describing. Thirdly, it can curtail individuals, who may otherwise have a great incentive in donating to other high-impact causes to actually do so.
My view is that researchers conducting cost-effectiveness analyses have a role to play here: benchmarking standardised market rates for senior staff, independent of what individuals actually choose to pay themselves. This would remove the artificial incentive, give organisations flexibility to determine their own compensation, and allow any differential between market rate and actual salary to be treated as what it effectively is: a donation. The linked EA Forum piece makes this case well.
I recognise this is a longer-term structural shift and harder to move than individual donor preferences. But I think a lot of what you’re describing is downstream of this measurement problem, and that’s where the leverage is.
Hey Svetha—I have a lot of admiration for what you and your team have built at New Incentives. A couple of questions:
1. Could you describe how New Incentives has used evidence throughout its history to inform strategic pivots / changes to its intervention, and what you have learned from this process?
2. The KPIs and cost-effectiveness figures that New Incentives has achieves are remarkable. What has it taken (that may not be visible in these numbers) to build an organisation and foster a team to achieve these results? What are some generalisable learnings / frameworks that may assist other organisations in their growth / scaling journeys?
3. I understand that New Incentives is listed as one of GiveWell’s Top Charities which I assume helps with bringing in significant philanthropic support. From my experience at the recent Skoll Conference, there is a lot of discussion right now of engaging with government as the doers and payers at scale—particularly as philanthropy has gone through a recent sea-change. Are you thinking about sustainability and engagement with government as a route to further scale, or is the current model of philanthropic support the major vector that you see for ongoing scaling of your intervention?
Hi Mark — thank you for the detailed response and for sharing the steps GiveWell is already taking on monitoring implementation fidelity. All of this is very encouraging, particularly the moves toward real-time identification and resolution of issues at the grantee level.
One area I’d love to hear more about is the following:
“Ensuring grantees are adequately monitoring critical activities. We’ve made it standard practice to identify the critical activities required for the desired impact and to ensure grantees have plans for monitoring these in real-time and addressing issues quickly.”
I think this is one of the most important levers available. Internal grantee monitoring, when done well, is likely to be the fastest and most cost-effective feedback mechanism. External evaluation processes generate valuable information, but they are almost always slower and more resource-intensive. Beyond improving grantmaking decisions directly, I’d expect this to have an indirect benefit: making M&E expectations explicit and visible tends to strengthen internal M&E culture within organisations over time.
What I’d be curious about is whether GiveWell has considered moving toward some degree of transparent standardisation of what “monitoring critical activities” actually looks like in practice — for instance, a shared framework defining what grantees are expected to track, at what frequency, and how findings should be acted on.
Is this something GiveWell is actively developing, and if so, is there scope for that framework to be shared more publicly?
This is a thoughtful framework, and I broadly find the approach reasonable. One dimension I’d like to see explored further, though, is the risks embedded in using collective user preference as the mechanism for determining what counts as “prosocial.”
The post rightly flags the challenge of identifying uncontroversial prosocial actions, and grounding this in aggregated user preferences is an intuitive starting point. But collective preference carries well-documented risks, including majoritarian bias, and what users collectively want may not align with what is genuinely beneficial for minority groups or for society in the long run. The history of democratic theory gives us good reason to be cautious here.
This raises a question I’d genuinely like to hear views on: to what extent should formal governance structures, including governments and their regulatory capacity, play a role in defining the boundaries of prosocial AI behaviour? I recognise the practical complexity here, particularly given the current fragmented state of AI governance globally. But philosophically, democratically accountable institutions offer something that collective user preference alone cannot: legitimacy derived from deliberative processes, legal accountability, and explicit protections for minority interests.
I’m not arguing that regulation is a clean solution. But it might serve as a useful complementary layer to user preference aggregation, providing a check against the most significant failure modes of purely preference-based approaches.
Hi Max,
Want to flag that we are doing exactly as you suggested, and I hope to provide a new PR shortly. We believe that communicating our cost effectiveness using a tool like this will make the currently quite cumbersome spreadsheet come to life.
Thanks again. We will keep you posted.
Thanks for sharing this post. I appreciated the honest behind-the-scenes look at what is involved in being a grant maker.
As someone not working in the AI safety space, I’m intrigued by your opinions as to what extent grant making within AI safety is similar to and different from grant making within other cause priority areas, for example animal advocacy and global development and health?
My sense from reading the post is that those areas may be relatively less neglected, with fewer opportunities for identifying opportunities with outsized impact returns on investment. Do you think that is a reasonable assumption to be making?
Thanks for synthesizing and sharing this, Dan.
My sense is that the social cost of greenhouse gases carries fairly wide uncertainty bounds, driven by discounting assumptions and long-run projections. I’m underqualified to judge how robust these are, though I’d guess the biases could run either way.
What I’m more interested in is the second-order effect on cause prioritisation. More incoming philanthropic funds are likely to saturate grantmakers in global health and development, pushing the cost-effectiveness bar down, and we’re already seeing that with GiveWell and Coefficient Giving.
But I wonder if part of what we’re seeing there is really an absorptive capacity constraint (a limited set of organisations able to scale up delivery), rather than a genuine shortage of cost-effective interventions. Scaling delivery takes time, people, and systems, and that constraint doesn’t come through clearly in your piece. Whether climate faces the same constraint isn’t clear to me either.
You mention Giving Green’s large 2025 grant will take roughly a year to fully disburse. How should we think about absorptive capacity in climate versus GHD?
If climate absorption is genuinely constrained, then GHD interventions, with likely smaller uncertainty bounds, could see absorptive capacity grow over two to four years as delivery organisations scale. That would push the cost-effectiveness bar back up and soften the piece’s headline conclusion. Keen to hear how you’d think through this dynamic.