The CEAs were built with different levels of involvement from the fellows, our team, and LLM support. Claude being the main supporting tool.
The template and overall structure of the CEAs were developed outside of Claude, based on our methodology (and heavily comparing different templates from different organizations).
As for the content of the models:
Costs: fellows first analyzed the cost structure of the interventions they were drawing evidence from (i.e., understanding the key cost drivers). Based on that structure, they built localized budgets using country-specific prices and costs. All organizational budgets were developed by the fellows within our template for direct implementation, with those benchmarks and certain assumptions, we calculated the intervention cost at scale, so no Claude involvement here.
Key assumptions: (e.g., reach, limiting factors) were fully defined by the teams, based on their contextual knowledge, and we then used scale factors, to project reach at scale, so no Claude involvement here.
Evidence: (RCTs, meta-analyses, etc.) was reviewed as part of the research process (we used other LLMs for this as well, mainly Elicit and Perplexity); Claude mainly helped structure and organize the information within the template.
Calculations: were done with Claude, but step by step, so we reviewed every output generated (making this the most time-consuming part of the process). I can confidently say we reviewed every cell, adjusting assumptions, asking questions, and catching errors. Of course, this is also the part where errors are most likely to occur, and these are only the first versions of the models, so I am also confident that there are still mistakes we did not catch.
Once the models were completed, we held review sessions with each team to revisit key assumptions and refine the models. For example, asking whether certain implied assumptions made sense for the real world, and because the fellows themselves know their context and intervention details, many things were changed. However, the models are far from being final, but the template structure was created for them to be used as a planning tool (not only a fundraising tool), so teams will definitely adjust them with time and with their internal evidence, once available.
In full honesty, building the models was the part of the incubation process that scared me the most, so I’m very glad Claude appeared at the right time, and it also made me more confident in that using templates like this is a way to make CEAs more accessible: once organizations have clear cost structures (and many orgs have detailed budgets), assumptions, and external evidence, building a first model becomes much more feasible. It’s not perfect, but it’s a strong starting point.
Very happy to share the template or walk through it together, we’re very keen to improve it and learn further with feedback.
Thanks for the clarifying comment, Verónica. I strongly upvoted it. I would be happy to have a look into the CEAs of the animal welfare interventions (for free, sometime over the next 14 days or so). I just requested commenter access.
Hi Vasco,
Thanks for the question, happy to answer.
The CEAs were built with different levels of involvement from the fellows, our team, and LLM support. Claude being the main supporting tool.
The template and overall structure of the CEAs were developed outside of Claude, based on our methodology (and heavily comparing different templates from different organizations).
As for the content of the models:
Costs: fellows first analyzed the cost structure of the interventions they were drawing evidence from (i.e., understanding the key cost drivers). Based on that structure, they built localized budgets using country-specific prices and costs. All organizational budgets were developed by the fellows within our template for direct implementation, with those benchmarks and certain assumptions, we calculated the intervention cost at scale, so no Claude involvement here.
Key assumptions: (e.g., reach, limiting factors) were fully defined by the teams, based on their contextual knowledge, and we then used scale factors, to project reach at scale, so no Claude involvement here.
Evidence: (RCTs, meta-analyses, etc.) was reviewed as part of the research process (we used other LLMs for this as well, mainly Elicit and Perplexity); Claude mainly helped structure and organize the information within the template.
Calculations: were done with Claude, but step by step, so we reviewed every output generated (making this the most time-consuming part of the process). I can confidently say we reviewed every cell, adjusting assumptions, asking questions, and catching errors. Of course, this is also the part where errors are most likely to occur, and these are only the first versions of the models, so I am also confident that there are still mistakes we did not catch.
Once the models were completed, we held review sessions with each team to revisit key assumptions and refine the models. For example, asking whether certain implied assumptions made sense for the real world, and because the fellows themselves know their context and intervention details, many things were changed. However, the models are far from being final, but the template structure was created for them to be used as a planning tool (not only a fundraising tool), so teams will definitely adjust them with time and with their internal evidence, once available.
In full honesty, building the models was the part of the incubation process that scared me the most, so I’m very glad Claude appeared at the right time, and it also made me more confident in that using templates like this is a way to make CEAs more accessible: once organizations have clear cost structures (and many orgs have detailed budgets), assumptions, and external evidence, building a first model becomes much more feasible. It’s not perfect, but it’s a strong starting point.
Very happy to share the template or walk through it together, we’re very keen to improve it and learn further with feedback.
Thanks for the clarifying comment, Verónica. I strongly upvoted it. I would be happy to have a look into the CEAs of the animal welfare interventions (for free, sometime over the next 14 days or so). I just requested commenter access.
Thanks Vasco, hugely appreciated!
The CEAs of the animal welfare interventions looked very thorough. I left some comments.
Thanks Vasco!!!
You are welcome. Feel free to get in touch in the future if you want me to have a look at other CEAs of animal welfare interventions.
Awesome! Will do, thank you!!!