Thank you. I was not able to get (a pdf of) Field Experiments, but downloaded the “Field Experimental Designs for the Study of Media Effects,” also co-authored by Green. They point out “robust cluster standard errors” to estimate “individual-level average treatment effect” (172).
To answer your points:
The smallest effect size you would hope to observe
20%. From 5⁄10 to 6⁄10 or equivalent % increase
Your available resources
Researchers in all of the campaign clusters and some of the non-campaign ones. They can count whether e. g. few hundreds of individuals wear face covering
The population within each cluster
Different, average of 180,000⁄6 = 30,000.
The total population
Since we are just looking to estimate the impact of the 180,000-person campaign and not to generalize it, this should be 180,000x2 (180,000 participating and an equal number of non-participants who are the nearest geographically and in characteristics).
Your analysis methodology
Probit, logit or simple linear regression, but open to suggestions
I meant 6 groups in the intervention area, and some number of groups (e. g. 3 or 6) in the non-intervention area.
OK. So 3 intervention clusters and 3 non-intervention clusters are better than 6 intervention clusters and 3 non-intervention clusters but 6+6 may be necessary? Would the answer depend on the intra-cluster correlation coefficient (ρ)? Perhaps, the texts that generally talk about clustering assume relatively significant between cluster variability and low within cluster variability (so high ρ). However, in this study, how people respond to the messaging may not depend much on their ‘cluster assignment,’ but much more on their individual characteristics that, on average, may be comparable across the clusters and the studied population.
I should ask EA Cameroon about the possibility of different average responses in different villages.
Do you know of any online sample size calculator that includes clusters?
I refer you to Sindy’s comment (she is actually an expert) but I want to note and verify that it sounds as if you may not actually be thinking of collecting individual-level data, and that you’re thinking of making observations at the village level (e.g. what % of people in this village wear masks?). So it’s not just the case that you wouldn’t have enough clusters to make a statistical claim, but you may actually be talking about doing an experiment in which the units are villages… so n = 6 to 12. Then of course you’d have considerable error in the village-level estimate, and uncertainty about the representativeness about the sample within each village. I agree with Sindy that you probably don’t want an RCT here.
Thank you. I was not able to get (a pdf of) Field Experiments, but downloaded the “Field Experimental Designs for the Study of Media Effects,” also co-authored by Green. They point out “robust cluster standard errors” to estimate “individual-level average treatment effect” (172).
To answer your points:
The smallest effect size you would hope to observe
20%. From 5⁄10 to 6⁄10 or equivalent % increase
Your available resources
Researchers in all of the campaign clusters and some of the non-campaign ones. They can count whether e. g. few hundreds of individuals wear face covering
The population within each cluster
Different, average of 180,000⁄6 = 30,000.
The total population
Since we are just looking to estimate the impact of the 180,000-person campaign and not to generalize it, this should be 180,000x2 (180,000 participating and an equal number of non-participants who are the nearest geographically and in characteristics).
Your analysis methodology
Probit, logit or simple linear regression, but open to suggestions
I meant 6 groups in the intervention area, and some number of groups (e. g. 3 or 6) in the non-intervention area.
OK. So 3 intervention clusters and 3 non-intervention clusters are better than 6 intervention clusters and 3 non-intervention clusters but 6+6 may be necessary? Would the answer depend on the intra-cluster correlation coefficient (ρ)? Perhaps, the texts that generally talk about clustering assume relatively significant between cluster variability and low within cluster variability (so high ρ). However, in this study, how people respond to the messaging may not depend much on their ‘cluster assignment,’ but much more on their individual characteristics that, on average, may be comparable across the clusters and the studied population.
I should ask EA Cameroon about the possibility of different average responses in different villages.
Do you know of any online sample size calculator that includes clusters?
I refer you to Sindy’s comment (she is actually an expert) but I want to note and verify that it sounds as if you may not actually be thinking of collecting individual-level data, and that you’re thinking of making observations at the village level (e.g. what % of people in this village wear masks?). So it’s not just the case that you wouldn’t have enough clusters to make a statistical claim, but you may actually be talking about doing an experiment in which the units are villages… so n = 6 to 12. Then of course you’d have considerable error in the village-level estimate, and uncertainty about the representativeness about the sample within each village. I agree with Sindy that you probably don’t want an RCT here.
OK, thank you.