Super nice post. Impressed by the ‘translation into EA language’ from the original post. It’s also a skill I’m working on.
I’d like to ask a little more about your point in “2. Fund rigorous evaluations of participatory allocation” How does this process respond to participatory grantmaking inherently having a lower statistical verifiability, given that participatory grantmaking is generally much less uniform and traceable using quantitative techniques? There’s no good counterfactual for Extinction Rebellion (funded by Guerilla Foundation), and FundAction’s grantees are tiny and extremely diverse. I’ve seen granters have to give up on statistics in these cases and lean into other ways of knowing e.g. relationships, trust and feelings dreaded by all EAs;) I think that a vast amount of time and resources will be needed to get a large enough sample size, and I’m not sure that social movement groups will be too happy about that.
Hey Sophie! Thanks again for the nudge to post here.
You’re right that the most movement-driven work resists clean counterfactuals. There’s no control group for Extinction Rebellion, and forcing FundAction’s grantees into a sample-size study could distort the thing and likely annoy the groups themselves. An RCT doesn’t fix that.
But I’d push us on the binary. The choice isn’t statistics or “feelings dreaded by all EAs.” It looks that way because the field keeps reaching for bednet-style tools, watching them fail to fit, and concluding the thing itself can’t be measured. Trust can be studied rigorously. Social network analysis, mixed methods, racialized-trust frameworks.
I’m working on this right now with Dr. Amber Banks. Her PhD mapped cross-cultural trust networks using network analysis, and she spent years as a program officer inside Gates. She doesn’t experience rigor and trust-based knowing as opposites.
“Fund rigorous evaluations” doesn’t mean RCTs everywhere. It means stop treating proximity as the place evidence ends. Often the better question isn’t “did it beat a counterfactual” but “did this process surface and fund what the legible process couldn’t see?” You can actually measure that.
And you’re right that underneath all of it sits the real question: whether EA will count relationship-based knowing as evidence at all. That discomfort your wink points at is the content, not a side issue. Amber and I are betting you can honor that knowing and still study it well. We’re taking a crack at it.
Super nice post. Impressed by the ‘translation into EA language’ from the original post. It’s also a skill I’m working on.
I’d like to ask a little more about your point in “2. Fund rigorous evaluations of participatory allocation”
How does this process respond to participatory grantmaking inherently having a lower statistical verifiability, given that participatory grantmaking is generally much less uniform and traceable using quantitative techniques? There’s no good counterfactual for Extinction Rebellion (funded by Guerilla Foundation), and FundAction’s grantees are tiny and extremely diverse. I’ve seen granters have to give up on statistics in these cases and lean into other ways of knowing e.g. relationships, trust and feelings dreaded by all EAs;)
I think that a vast amount of time and resources will be needed to get a large enough sample size, and I’m not sure that social movement groups will be too happy about that.
In this case perhaps RCT isn’t the gold standard? What about qualitative evidence?
p.s. I’m Sophie Davison on LinkedIn
Hey Sophie! Thanks again for the nudge to post here.
You’re right that the most movement-driven work resists clean counterfactuals. There’s no control group for Extinction Rebellion, and forcing FundAction’s grantees into a sample-size study could distort the thing and likely annoy the groups themselves. An RCT doesn’t fix that.
But I’d push us on the binary. The choice isn’t statistics or “feelings dreaded by all EAs.” It looks that way because the field keeps reaching for bednet-style tools, watching them fail to fit, and concluding the thing itself can’t be measured. Trust can be studied rigorously. Social network analysis, mixed methods, racialized-trust frameworks.
I’m working on this right now with Dr. Amber Banks. Her PhD mapped cross-cultural trust networks using network analysis, and she spent years as a program officer inside Gates. She doesn’t experience rigor and trust-based knowing as opposites.
“Fund rigorous evaluations” doesn’t mean RCTs everywhere. It means stop treating proximity as the place evidence ends. Often the better question isn’t “did it beat a counterfactual” but “did this process surface and fund what the legible process couldn’t see?” You can actually measure that.
And you’re right that underneath all of it sits the real question: whether EA will count relationship-based knowing as evidence at all. That discomfort your wink points at is the content, not a side issue. Amber and I are betting you can honor that knowing and still study it well. We’re taking a crack at it.