Ultimately, we’d like to come out of the pilot program with a metric that falls somewhere close to “cost per TLYCS pledger via pamphlets”
I noticed what might be a significant confounder in getting this estimate: you are likely to be particularly enthusiastic/eloquent about the whole thing, which is an extra input which will help the effectiveness of the pamphlets, but is very hard to budget properly on the ‘costs’ side.
To account for this, you would probably be better hiring someone else to distribute the pamphlets; probably someone without a deep existing commitment to TLYCS (but some contact should be fine—the question is whether you could get similarly good people when trying to scale it up).
But if you’re going to scale this, you’ll probably get LYCS members or EAs to hand out the pamphlets anyway, right? I mean, we kind-of do need more concrete volunteery tasks that we can give to student groups anyway.
So it’s best to get random EAs or LYCS members to do it in the study, right?
I agree that if the people running the study are also distributing the pamphlets then you end up with bias.
This is a really good point. Yeah, the scaling model is to have local TLYCS chapters organizing volunteers to do this as a regular, rolling semester activity. I hadn’t really considered myself a confounding variable in this sense, because I’m definitely not a master pamphleteer. I’m an engineer by trade, and if this program takes off, I’ll eventually just be another volunteer in the LA area that helps hand out leaflets occasionally. We’re also thinking about splitting crews on Friday distribution days—so I would have a crew that hits up two universities, and there would be another volunteer crew hitting up two different campuses. Any thoughts on this?
I noticed what might be a significant confounder in getting this estimate: you are likely to be particularly enthusiastic/eloquent about the whole thing, which is an extra input which will help the effectiveness of the pamphlets, but is very hard to budget properly on the ‘costs’ side.
To account for this, you would probably be better hiring someone else to distribute the pamphlets; probably someone without a deep existing commitment to TLYCS (but some contact should be fine—the question is whether you could get similarly good people when trying to scale it up).
But if you’re going to scale this, you’ll probably get LYCS members or EAs to hand out the pamphlets anyway, right? I mean, we kind-of do need more concrete volunteery tasks that we can give to student groups anyway.
So it’s best to get random EAs or LYCS members to do it in the study, right?
I agree that if the people running the study are also distributing the pamphlets then you end up with bias.
I wasn’t sure what the scaling model was, but if there are enough plausible volunteers then this sounds right.
The general point is that you want to try and produce a typical case, not a special case.
This is a really good point. Yeah, the scaling model is to have local TLYCS chapters organizing volunteers to do this as a regular, rolling semester activity. I hadn’t really considered myself a confounding variable in this sense, because I’m definitely not a master pamphleteer. I’m an engineer by trade, and if this program takes off, I’ll eventually just be another volunteer in the LA area that helps hand out leaflets occasionally. We’re also thinking about splitting crews on Friday distribution days—so I would have a crew that hits up two universities, and there would be another volunteer crew hitting up two different campuses. Any thoughts on this?