Thanks for the reply, and most of this makes sense to me.
I’m not sure I understand how you won’t be involved in the selection of evaluators, who will do that exactly? Or maybe you mean you won’t select the on-the-ground evaluators as in that will be done by the company, which makes sense.
“The monitoring grant will fund the creation of a sampling frame that includes both public and private health facilities, which we think will yield more complete data than contacting hospitals through our partners.” This could work (high risk), but seems like a roundabout and inefficient way to do things. Following up on that data from multiple hospitals in West Africa for example could be a nightmare.
I would have thought with this kind of massive funding and the relatively small number of people who get procedures (in the thousands), MiracleFeet could maintain a database the contact details of every kid who gets help—this wouldn’t be hard and would make M&E so much easier for everyone. Hospitals might well collect substandard information which makes proper follow up impossible, spoiling your M&E efforts.
If I was going to give one piece of advice on M&E, it would be that your evaluators should follow up personally a completely random sample of individuals who had been treated—both to check that both the interventions actually happened, and that the claimed improvement is real. There should be a list of names, home locations and phone numbers of every single patient who received treatment—I think if that’s not there and individuals can’t be followed for this kind of intervention, than meaningful M&E becomes close to impossible.
Yes, to clarify, MiracleFeet is selecting the on-the-ground evaluator that will conduct the monitoring in each location, although GiveWell will lightly vet MiracleFeet’s choices and meet with each evaluator.
The primary purpose of the monitoring grant is to understand how many children are treated for clubfoot both with and without MiracleFeet’s support. So, although MiracleFeet has records of children treated through facilities it’s supported, we also want an assessment of baseline treatment coverage before MiracleFeet launches its program (or expands it, in the case of the Philippines). We do plan to incorporate some form of data audit as part of endline activities; we’ll work out the details of that at a later date.
Thanks again for your interest in this and for taking the time to ask questions!
Thanks for the reply, and most of this makes sense to me.
I’m not sure I understand how you won’t be involved in the selection of evaluators, who will do that exactly? Or maybe you mean you won’t select the on-the-ground evaluators as in that will be done by the company, which makes sense.
“The monitoring grant will fund the creation of a sampling frame that includes both public and private health facilities, which we think will yield more complete data than contacting hospitals through our partners.” This could work (high risk), but seems like a roundabout and inefficient way to do things. Following up on that data from multiple hospitals in West Africa for example could be a nightmare.
I would have thought with this kind of massive funding and the relatively small number of people who get procedures (in the thousands), MiracleFeet could maintain a database the contact details of every kid who gets help—this wouldn’t be hard and would make M&E so much easier for everyone. Hospitals might well collect substandard information which makes proper follow up impossible, spoiling your M&E efforts.
If I was going to give one piece of advice on M&E, it would be that your evaluators should follow up personally a completely random sample of individuals who had been treated—both to check that both the interventions actually happened, and that the claimed improvement is real. There should be a list of names, home locations and phone numbers of every single patient who received treatment—I think if that’s not there and individuals can’t be followed for this kind of intervention, than meaningful M&E becomes close to impossible.
Hi, Nick,
Yes, to clarify, MiracleFeet is selecting the on-the-ground evaluator that will conduct the monitoring in each location, although GiveWell will lightly vet MiracleFeet’s choices and meet with each evaluator.
The primary purpose of the monitoring grant is to understand how many children are treated for clubfoot both with and without MiracleFeet’s support. So, although MiracleFeet has records of children treated through facilities it’s supported, we also want an assessment of baseline treatment coverage before MiracleFeet launches its program (or expands it, in the case of the Philippines). We do plan to incorporate some form of data audit as part of endline activities; we’ll work out the details of that at a later date.
Thanks again for your interest in this and for taking the time to ask questions!
Miranda