I am a charity founder and M&E professional focused on farmed animal welfare, especially within Low- and Middle-Income Countries.
Tom Billington
Karma: 1,101
Hey Aidan :)
Yes! I think we agree here, but we’re framing things differently.
We would say that mass investment, even in speculative gains, is not necessarily a bad thing (though we’re also not saying it’s necessarily a good thing).
However, we would say that such a mass investment should be done with our eyes open. We define a bubble specifically as “a situation where expected impact in a given timeframe substantially exceeds realistic impact”. It is the mismatch of expectations (in both the upswing and the downswing) that we mean to point to as causing harm to the movement.
Thanks for the clarification!
I added the word “potentially” to the relevant section to make it more accurate.
Hey Caleb!
Going through each point in turn:
1. “Isn’t most of the focus [for cultivated meat] already on development?”
Yes, it is. However, our experience was that during the peak hype period, clean meat was communicated as almost ready to be on store shelves worldwide.
So we are arguing that there is pressure for ideas to be presented as close to last-mile delivery, even when this is not the case.
2. “I don’t see why [leafletting] was named as a ‘bubble’”
For us, a bubble just refers to “a situation where expected impact in a given timeframe substantially exceeds realistic impact”, so it does not necessitate that the downward adjustment is wrong.
Though maybe your point is more “why include it if the downward adjustment was justified?”.
To that, we would say there is damage caused on the upswing and the downswing. In this instance, the damage was in the detraction from other strategies and the unnecessary disillusionment.
Also, we meant for the historical examples to be more illustrative of the pattern, and definitely don’t mean to be disparaging to those efforts or decisions.
3. “I think one needs to make an actual argument at the object level for why there could be an overcorrection towards welfare tech”
Interesting point for clarification!
In part, we mean for this post to be a flag, giving a name to a pattern and signalling that welfare tech shares many of the hallmarks. Similarly, we don’t mean to say that any given effort is necessarily a mistake.
That said, we do think that there is a significant amount of resources currently being directed towards welfare tech (for example here, here, here) in ways that are oriented towards a belief that welfare tech is either ready for implementation and scaling or close to it. That could totally be right in some instances! But it could also be wrong. Even the fact that we are talking about welfare tech as a monolith is, for us, a signal that it is being simplified in the way that we describe as part of the cycle of a bubble.
Beware Silver Bullets: Are we making “welfare tech” into the next animal advocacy bubble?
Learning from Agricultural Development to Reshape Animal Welfare Work in Low- and Middle-Income Countries
Fish Welfare Initiative Hacked
Aaron!
Thanks for posting :) I’m coming at this as someone who spent a lot of time running on-farm research at Fish Welfare Initiative (and planning to do more through my new charity). I broadly agree, but I’d add a few caveats:
The core issue with farms as “welfare labs” is heterogeneity (variability). Especially in LMICs, farms are messy, uncontrolled environments where confounding variables easily creep in. That creates a lot of statistical noise. If you’re aiming for high certainty, farms can make that difficult.
Relatedly, on-farm research makes it harder to isolate specific effects. You mention the benefit of insights collected “under real commercial conditions”, and I agree that ultimately effectiveness in the real world is what matters. But there’s also strong value in isolating variables to understand mechanisms. If we’re testing whether pigs prefer straw or wood shavings, we may not want to simultaneously capture differences in how farmers manage those materials. Otherwise, when results don’t support our hypothesis, we don’t know whether we’re observing animal preference or management differences. That’s why the difference between efficacy (does it work in controlled conditions?) and effectiveness (does it work in real-world conditions?) is useful.
So in sum: on-farm research is a valuable tool, but it can’t replace controlled research, and I’d hesitate to frame it as more valuable.
I also think two ideas may be getting conflated: monitoring existing farm conditions vs running experiments on farms. Both can be useful, but both need a clear use-case for the data. I agree, though, that there’s strong potential on both fronts.
Cool! Would be keen to sign on to a mailing list :)
Strong agree.
And I understand why this is a problem. It can be hard to independently create contacts in these spaces from scratch, and there is an aspect of not knowing what you don’t know at play. I’m almost certain I am committing the same mistake in multiple places in my work.Would be interested to think about solutions here. Like perhaps a group such as Consultants For Impact could take on a role of knowledge dispersal, doing things like getting project management experts to give a talks at EAGs?
For sure!
I would say that EAs are missing large parts of M&E, including:
- The formal setting of key questions / assumptions that form the basis of what you will focus on trying to answer
- Creating formal monitoring frameworks (e.g. a log frame) that takes these questions / assumptions and identifies practical indicators and a method of measuring them
- I think EAs don’t use the full diversity of M&E tools. In my experience we tend to over-index on surveys (vs., say, interviews, focus group discussion, or observational data)
- I think considering the frequency of use of surveys, we could generally up-skill in high-quality survey design
- Using a diverse set of evaluation types (EAs generally know about RCTs, but these are such a narrow slice of the available evaluation types)
In general I think we care about M&E but lack experience in the formal processes of it, especially monitoring. So application is patchy and not generally in line with best practices.
I should perhaps clarify that I am mostly talking about the non-global development side of EA. I think their norms for M&E are significantly better.
Intrac’s M&E universe is one place to see an overview of what M&E entails. I think also The Mission Motor intends to create more resources on these topics in the future :)
Congrats on launching!
Excited to follow your work.
My advice for EAs who want to skill up in a neglected area:
Get a good mentor who broadly aligns with EA values but already has the skill. I’ve had three of these relationships (with experts in fish welfare, M&E, and agricultural development). In each case, it was symbiotic: I could offer EA knowledge and connections in return for them teaching me their skillset.
Book calls with as many relevant people as possible in the field. Shamelessly ask for help, even from people only tangentially related to what you’re trying to learn. The more you snowball contacts, the better your chances of finding a mentor. Also, having lots of people explain the fundamentals to you repeatedly is very useful, as it helps you learn the core ideas of the field.
Read a lot. Try to consume a lot of media from the chosen field. You know you’re doing well when you start to notice consistent underlying patterns and premises (again, the core ideas).
Start applying it before you’re ready. Look for small projects you can use as practice for the new skill.
In general, when learning a new skill, Andrea Gunn’s talk on training leaders offers a lot of good insight. I also made a one-page summary of her talk.
I have historically been able to do this upskilling as a side project to my existing job.
Why I Steal (Ideas) and You Should Too
Glad it’s helpful!
Monitoring, Evaluation, and Learning: A Simple Introduction
Have some level of scepticism that the total “util count” from life on earth will be net-positive. I’m also in general wary of impact that is too long-term speculative.
Can Monitoring and Evaluation Drive Organizational Change? Lessons from The Mission Motor
That’s the one!
Weird that I remember it being half the size.
Anecdotal evidence, but I would attribute at least part of the reason I ended up in an EA career to a 2 day event of ~12 people hosted by EA London.
Yeah, I think there is a thin line sometimes between not overwhelming newcomers and not misleading them.
Honestly, if you’d told me how hard Fish Welfare Initiative would be, I’m not sure that past Tom would have chosen to found it (though I am extremely glad I did!). Now I struggle with how to communicate with the new founders.
Though I would say it is definitely a mistake for bought-in advocates like yourself not to be clear on the realistic timelines/likelihoods of success. And I would also say that it is possible for people to get excited about longer-term pushes where milestones are smaller but realistic.
And also, there is a place for optimism and pushing for great things that others don’t think are possible. So there’s a balance to all of this.