Hey, thank you for the work you are doing! Here are my thoughts (I’m an economist at IDinsight and work on this type of research):
If you want to understand the impact of your program, I don’t recommend doing an RCT at this stage. This seems like a very small pilot and you won’t have enough power / sample size to detect an effect (more see below). You should only consider running an RCT if and when you plan to scale this up later to a sufficient scale.
Instead what I advise is trying to understand and improve your impact by doing some small sample survey + qualitative research. E.g. when you go to a village, talk to locals (ideally capture a good representation of different types of people in the community, not just leaders but also relatively marginalized groups; you could do a rigorous sampling but I’m not sure if that’s realistic or worthwhile at this stage given the trouble that involves) to understand their current knowledge, attitudes, and behavior around COVID (what knowledge they lack, what attitude needs changed, what rumors are around etc.) -- to better design your messages; also ask them what kind of information campaign would engage them, and after you do your program ask how they felt—whether they liked it, whether they found it useful, what they learned, what they’d do differently etc. Can also contact them some time later to see if they observe any behavioral change among people in the community (better than asking what they themselves do due to social desirability bias).
More technical details:
Since you’re doing a clustered RCT—treatment is at the village level and the outcomes of people within a village are likely positively correlated—you’ll need a larger sample size than if you were doing an individual-level RCT (for the math, see section 4.2 of this—generally a great resource for RCT design). You can do a power calculation for a clustered randomized controlled trial, e.g. using Stata’s “power twomeans” command. One parameter that’s missing is the intraclass correlation (correlation among individuals within a treatment unit). However, since your cluster size is SO small (3 and 3), when I try to do this calculation in Stata with any reasonable assumption Stata says you cannot have enough power (assuming you want all the standard -- 80% power, 5% significance level etc.). That’s why I recommend not doing an RCT unless you have a program at scale
Thank you so much. This answers my question. Yes, there will be a before and after qualitative survey asking about own and others’ behavior—which may need to be truncated to speak with more different groups. Then, the face covering data can be used to complement the survey information.
Hey, thank you for the work you are doing! Here are my thoughts (I’m an economist at IDinsight and work on this type of research):
If you want to understand the impact of your program, I don’t recommend doing an RCT at this stage. This seems like a very small pilot and you won’t have enough power / sample size to detect an effect (more see below). You should only consider running an RCT if and when you plan to scale this up later to a sufficient scale.
Instead what I advise is trying to understand and improve your impact by doing some small sample survey + qualitative research. E.g. when you go to a village, talk to locals (ideally capture a good representation of different types of people in the community, not just leaders but also relatively marginalized groups; you could do a rigorous sampling but I’m not sure if that’s realistic or worthwhile at this stage given the trouble that involves) to understand their current knowledge, attitudes, and behavior around COVID (what knowledge they lack, what attitude needs changed, what rumors are around etc.) -- to better design your messages; also ask them what kind of information campaign would engage them, and after you do your program ask how they felt—whether they liked it, whether they found it useful, what they learned, what they’d do differently etc. Can also contact them some time later to see if they observe any behavioral change among people in the community (better than asking what they themselves do due to social desirability bias).
More technical details:
Since you’re doing a clustered RCT—treatment is at the village level and the outcomes of people within a village are likely positively correlated—you’ll need a larger sample size than if you were doing an individual-level RCT (for the math, see section 4.2 of this—generally a great resource for RCT design). You can do a power calculation for a clustered randomized controlled trial, e.g. using Stata’s “power twomeans” command. One parameter that’s missing is the intraclass correlation (correlation among individuals within a treatment unit). However, since your cluster size is SO small (3 and 3), when I try to do this calculation in Stata with any reasonable assumption Stata says you cannot have enough power (assuming you want all the standard -- 80% power, 5% significance level etc.). That’s why I recommend not doing an RCT unless you have a program at scale
Hello Sindy,
Thank you so much. This answers my question. Yes, there will be a before and after qualitative survey asking about own and others’ behavior—which may need to be truncated to speak with more different groups. Then, the face covering data can be used to complement the survey information.