Neil Buddy Shah: What Animal Advocates Can Learn from Medicine and Global Development

De­spite re­cent ad­vance­ments, re­search on an­i­mal welfare is still un­der­de­vel­oped com­pared to re­search in fields like medicine or de­vel­op­ment eco­nomics. In this talk, Neil Buddy Shah, the co-founder and CEO of IDin­sight, draws on les­sons from those fields to pro­pose that a co­or­di­nated, multi-year re­search agenda can ac­cel­er­ate im­prove­ments in an­i­mal welfare.

A tran­script of Neil’s talk, which we have lightly ed­ited for clar­ity, is be­low. You may also watch it on YouTube or read it on effec­tivealtru­ism.org.

The Talk

I want to start with one big caveat: I am a doc­tor and de­vel­op­ment economist by train­ing who works full-time in global de­vel­op­ment — and I am cat­e­gor­i­cally not an an­i­mal welfare ex­pert. How­ever, I do think there’s a tremen­dous amount that the an­i­mal welfare move­ment can learn from ad­vance­ments in medicine and in so­cial policy, both do­mes­ti­cally and in in­ter­na­tional de­vel­op­ment.

Slide02

What I’m hop­ing to do in this talk is:

1. Opine on a set of hy­pothe­ses about what those learn­ings might be and their im­pli­ca­tions for the an­i­mal welfare move­ment.

2. Per­haps more im­por­tantly, get ideas and crit­i­cal feed­back from those of you who do work in the an­i­mal welfare move­ment about which of these hy­pothe­ses may have legs and should be ex­plored fur­ther — and which of them are the re­sult of my own naivete.

I think we all start from a very similar propo­si­tion, which is that we’re con­cerned with how we can most effec­tively im­prove the welfare of billions of an­i­mals.

Slide03

There are a huge num­ber of po­ten­tial in­ter­ven­tions we have at our dis­posal in or­der to effect change — ev­ery­thing from vol­un­tary cor­po­rate cam­paigns, to reg­u­la­tory and policy re­forms, to in­di­vi­d­ual be­hav­ior-change in­ter­ven­tions.

My propo­si­tion is that, just as in medicine and global de­vel­op­ment, data and ev­i­dence have an ab­solutely cru­cial role for effec­tive al­tru­ists and oth­ers who wish to de­ter­mine how to do the most good pos­si­ble.

Slide04

I think that there are some very tac­ti­cal things that we can take from those other fields as the field of an­i­mal welfare ma­tures.

Slide05

First, I think it’s ob­vi­ous to most peo­ple that an­i­mal welfare shares a huge num­ber of com­mon challenges with medicine, pub­lic health, and de­vel­op­ment eco­nomics. We’re equally con­cerned with ques­tions around how we can most effec­tively im­prove lives — whether those are the lives of hu­mans or non-hu­man an­i­mals — and which in­ter­ven­tions are most cost-effec­tive. And how do we know whether some­thing works or is cost-effec­tive? How do we de­sign in­ter­ven­tions and mea­sure their effec­tive­ness, both for in­di­vi­d­ual be­hav­ior-change in­ter­ven­tions and broader, sys­temic re­forms?

A lot of other peo­ple have rec­og­nized these similar­i­ties. In fact, there’s been a blos­som­ing of a num­ber of ex­cel­lent an­i­mal welfare re­search or­ga­ni­za­tions, such as An­i­mal Char­ity Eval­u­a­tors, the Sen­tience In­sti­tute, Fau­n­a­lyt­ics, Hu­mane League Labs, and, of course, the Open Philan­thropy Pro­ject.

Slide06

But de­spite all of this progress in re­cent years, one of the things I want to ar­gue — and I’m happy to get push­back on this — is that the an­i­mal welfare re­search in­fras­truc­ture and ecosys­tem is in­cred­ibly im­ma­ture com­pared to what has de­vel­oped over decades in so­cial policy, medicine, and pub­lic health.

This re­ally com­plex di­a­gram [see fol­low­ing slide] shows just how so­phis­ti­cated the med­i­cal space is when it comes to the pro­duc­tion of ev­i­dence, the syn­the­sis of en­tire bod­ies of ev­i­dence, trans­lat­ing and dis­sem­i­nat­ing that syn­the­sis for prac­ti­tion­ers, and then en­forc­ing the use of that ev­i­dence by ev­ery­day doc­tors who are not nec­es­sar­ily well-versed in how to pro­duce or use it.

Slide07

You have a ro­bust ecosys­tem of or­ga­ni­za­tions that are spe­cial­ized in en­sur­ing in­cred­ibly high qual­ity at each of the nodes rep­re­sent­ing the pro­duc­tion and use of ev­i­dence. In the mid­dle, there’s a shared set of val­ues and norms around what con­sti­tutes good ev­i­dence — and what doesn’t.

I want to ar­gue that there’s an ana­log to an­i­mal welfare. Some of the core com­po­nents of that are:

1. There are long-term, co­her­ent, and co­or­di­nated re­search agen­das in medicine and in global de­vel­op­ment, such that in­di­vi­d­ual pa­pers and re­search ob­jec­tives cre­ate a lot more im­pact to­gether than they would oth­er­wise have on their own. One of the risks that we run within the an­i­mal welfare move­ment is we have a bunch of sep­a­rate ac­tors run­ning in­di­vi­d­ual tri­als that peo­ple aren’t ag­gre­gat­ing into a larger pic­ture.

2. There are clear ev­i­den­tiary stan­dards. Peo­ple across the sec­tor can re­fer to a com­mon vo­cab­u­lary in or­der to ar­gue about what works and what doesn’t, and what the qual­ity of ev­i­dence is. I think that is still more im­ma­ture in the an­i­mal welfare move­ment than it is in these other fields.

3. There are pro­tec­tive mea­sures to pre­vent data min­ing and other pub­li­ca­tion bi­ases through pre-reg­is­tra­tion and pre-anal­y­sis plans in these other fields, which could be worth adopt­ing in the an­i­mal welfare space.

4. Fi­nally, even if you have ex­cep­tional ev­i­dence cre­ation, you need mechanisms to en­sure that fund­ing flows based on that ev­i­dence. Both medicine and global de­vel­op­ment have cre­ated re­sults-based fi­nanc­ing mechanisms to en­sure that this hap­pens. And while those mechanisms aren’t yet wide­spread, they can still teach us im­por­tant les­sons.

Based on this re­view, I want to pro­pose two ini­tial ideas for the an­i­mal welfare move­ment. And again, I am very open to push­back from those of you who work full-time in this space.

Slide08

The first idea is to cre­ate the equiv­a­lent of the Na­tional In­sti­tutes of Health (NIH) for an­i­mal welfare. And the sec­ond is to start ac­tively ex­plor­ing re­sults-based fi­nanc­ing for an­i­mal welfare non­prof­its.

An over­ar­ch­ing na­tional re­search body

We’ll take each one in turn. The first idea, which pro­poses an over­ar­ch­ing na­tional body to or­ga­nize re­search and fund­ing for an­i­mal welfare, has two sub-pro­pos­als.

Slide09

The first is that we need to start cre­at­ing — as an en­tire ecosys­tem, not in­di­vi­d­ual or­ga­ni­za­tions — a long-term, co­or­di­nated re­search agenda that is based on what’s called the “value-of-in­for­ma­tion (VOI) ap­proach” in medicine and pub­lic health. And the sec­ond is to draft and adopt sec­tor-wide ev­i­dence-grad­ing stan­dards. Both of these are go­ing to be very difficult to im­ple­ment be­cause they re­quire ecosys­tem-wide buy-in and col­lab­o­ra­tion be­tween a num­ber of differ­ent or­ga­ni­za­tions. But if we’re able to do that the same way that medicine has, I think you’ll see dis­pro­por­tionate and non­lin­ear re­turns.

Why do we need a co­or­di­nated re­search agenda? Aren’t we already do­ing pretty well with all of these in­di­vi­d­ual or­ga­ni­za­tions, which I men­tioned pre­vi­ously? I think the short an­swer is that there is an in­cred­ibly com­plex the­ory of change to get to the end out­comes of fewer an­i­mals farmed and farmed an­i­mals suffer­ing less. And there’s a whole host of in­ter­ven­tions that can get us to those ul­ti­mate goals — for ex­am­ple, dietary change in­ter­ven­tions and fund­ing for meat al­ter­na­tives. And then there are the in­di­vi­d­ual be­hav­ior-change pro­cesses that have to hap­pen once those tech­nolo­gies ac­tu­ally ex­ist. Other in­ter­ven­tions in­clude pub­lic ad­vo­cacy, reg­u­la­tory and leg­is­la­tive re­form, vol­un­tary cor­po­rate cam­paigns, and hu­mane farm­ing ad­vo­cacy.

Slide11

At ev­ery sin­gle one of these nodes, in this com­plex the­ory of change, there are huge unan­swered ques­tions about which is ac­tu­ally the most cost-effec­tive. And de­spite the blos­som­ing of fund­ing in this space (the Open Philan­thropy Pro­ject has given over $90 mil­lion to date for an­i­mal welfare, which is in­cred­ible), there are still finite re­sources.

Which of these nodes we fo­cus on should be in­formed by some hy­poth­e­sis of where we’re go­ing to get the most bang for our buck.

Slide10

If you look at what ex­ists to date, there’s a lot of nascent but high-qual­ity ev­i­dence gen­er­a­tion in the an­i­mal welfare move­ment, from work done by An­i­mal Char­ity Eval­u­a­tors, the Sen­tience In­sti­tute, the Open Philan­thropy Pro­ject and a host of other or­ga­ni­za­tions. But what’s lack­ing is a sys­tem­atic look at an over­ar­ch­ing the­ory of change that’s tied to the liter­a­ture and ev­i­dence. Also, there is nei­ther a stan­dard and ac­cepted re­view of how to eval­u­ate the effec­tive­ness of in­di­vi­d­ual pa­pers, nor a way to ag­gre­gate in­di­vi­d­ual pa­pers to form a holis­tic view (e.g., on the effect of leaflet­ing on be­hav­ior change).

Then, if you look across the slide, what’s miss­ing is the benefit from an­swer­ing one ques­tion in that the­ory of change: What is the cost? Is it $10 mil­lion to an­swer one of those nodes, or is it some­thing cheaper? We’re also miss­ing a pri­ori­ti­za­tion sys­tem that flows from that cost-benefit calcu­la­tion — i.e., the value of get­ting more data on each of those par­tic­u­lar nodes. And fi­nally, is there a user-friendly pre­sen­ta­tion for NGOs that are fund­ing or­ga­ni­za­tions? It would be helpful to have some­thing for them to eas­ily refer­ence, with­out hav­ing to be statis­tics ex­perts, so that they can de­ter­mine the top five things to fund based on the ev­i­dence, as well as see whether there is room for more fund­ing.

This is no knock against the ex­cep­tional or­ga­ni­za­tions that have done this work, but what we see is that no or­ga­ni­za­tion in the an­i­mal welfare move­ment has set out to cre­ate a long-term, co­he­sive, and co­or­di­nated re­search agenda. That’s what will be re­quired in or­der to achieve our long-term goals.

So, how do we get to that VOI anal­y­sis in or­der to de­ter­mine where we should be putting our finite re­sources?

Slide12

In medicine and global de­vel­op­ment, you es­sen­tially move from a the­ory of change, to in­di­vi­d­ual re­search top­ics, to calcu­lat­ing the costs and benefits of an­swer­ing in­di­vi­d­ual re­search top­ics. Then, you ar­rive at a pri­ori­tized re­search agenda. This is a highly quan­ti­ta­tive ex­er­cise in medicine and in global de­vel­op­ment.

Slide13

You quan­tify the amount that a de­ci­sion-maker — whether that’s the Good Food In­sti­tute or the Open Philan­thropy Pro­ject — would pay for a piece of ev­i­dence to make a de­ci­sion.

Un­for­tu­nately, in the an­i­mal welfare move­ment, we’re at such an early stage in the gen­er­a­tion of ev­i­dence that we can’t quan­tify it. How­ever, we can use the same pro­cess in or­der to pri­ori­tize re­search top­ics. And as the liter­a­ture gets more ma­ture, we can start to quan­tify the re­turns we would ex­pect from in­vest­ing in differ­ent re­search top­ics.

There are five key com­po­nents to the VOI ap­proach:

Slide14


1. What are your pri­ors? What ev­i­dence ex­ists now about the im­pact of an in­ter­ven­tion — say, the effect of leaflet­ing on be­hav­ior change and re­duc­ing meat con­sump­tion?

2. What is the var­i­ance on your prior? Do you have high con­fi­dence in your es­ti­mate or low con­fi­dence?

3. How much do you ex­pect to spend on this? Would some­thing like leaflet­ing con­sume a huge amount of re­sources for the move­ment — yes or no?

4. How much would it cost to an­swer that ques­tion and up­date those pri­ors that you have?

5. Is there ac­tu­ally room for more fund­ing?

Based on these five com­po­nents, we can ar­rive at a set of re­search pri­ori­ties.

Slide15

Again, this isn’t quan­tified in the same way that it would be in medicine or health. And this is purely illus­tra­tive. There are still go­ing to be differ­ences of opinion among peo­ple in the com­mu­nity. But it makes ex­plicit the as­sump­tions be­hind why some of these ini­ti­a­tives should re­ceive more re­search fund­ing than oth­ers. It al­lows for some pri­ori­ti­za­tion.

That’s the first part of the pro­posal: that we need to cre­ate a long-term re­search agenda. I think if you look at the the­ory of change, a lot of peo­ple will say, “Hey, we’re get­ting a huge bang for our buck from vol­un­tary cor­po­rate cam­paigns, so let’s dou­ble down on that.” But the point is that, over time, if we want to get to a dra­matic re­duc­tion in the num­ber of an­i­mals farmed, we need to start at­tack­ing other nodes in that the­ory of change. And to at­tack those nodes cost-effec­tively in five or 10 years, you need to start gen­er­at­ing the re­search to­day about what’s go­ing to be most effec­tive at those nodes.

The sec­ond part of the “NIH for an­i­mal welfare” pro­posal is that we need to in­sti­tute ev­i­dence-grad­ing stan­dards to avoid the mis­takes made in medicine and global de­vel­op­ment.

Slide16

Some of you may be fa­mil­iar with the fact that there’s been a huge repli­ca­tion crisis in psy­chol­ogy and other so­cial sci­ences. Peo­ple have ar­gued that most pub­lished re­search find­ings are false, and that there is an in­cred­ible waste of re­sources around poor med­i­cal re­search. We want to cre­ate and adopt the stan­dards that have been used in medicine and global de­vel­op­ment to curb some of those ills in the an­i­mal welfare move­ment while the move­ment is still young. But we need to adapt them to the par­tic­u­lar­i­ties of the an­i­mal welfare move­ment.

Slide17

First, as with medicine and global de­vel­op­ment, stan­dards of ev­i­dence qual­ity should with­stand aca­demic scrutiny from other fields. Se­cond, they have to be prac­ti­tioner-fo­cused, so that some­one lead­ing an an­i­mal welfare NGO or fund­ing or­ga­ni­za­tion doesn’t need to be an econo­me­tri­cian to be able to see the grad­ing of a par­tic­u­lar in­ter­ven­tion and de­ter­mine whether it is a good use of their money. Third, this ev­i­dence-rat­ing sys­tem, as with medicine, needs to con­vey both the size of the im­pact of an in­ter­ven­tion and its episte­molog­i­cal un­cer­tainty. In other words, how cer­tain are you that your es­ti­mate is cor­rect?

I think the an­i­mal welfare move­ment needs to adopt a ver­sion of ev­i­dence-based medicine, which has a re­ally sim­ple rubric for all clini­ci­ans in or­der to de­ter­mine whether to provide as­pirin to a heart at­tack pa­tient, or­der an x-ray for some­one who has a pre­sumed sprained an­kle, or make some other de­ci­sion. It’s bro­ken down into just two crite­ria.

Slide18

First, us­ing a rat­ing of strong or weak, what is the po­ten­tial mag­ni­tude of the effect? And sec­ond, how con­fi­dent are you in your es­ti­mate of the ev­i­dence’s qual­ity — do you have a low, medium, or high level of con­fi­dence in it? All in­ter­ven­tions are based on these two crite­ria. It’s a very sim­ple rubric. And while a lot of this rubric from medicine wouldn’t ap­ply to the an­i­mal welfare move­ment, you could adapt some­thing similar to fit its par­tic­u­lar­i­ties.

Slide19

For ex­am­ple, in­stead of fo­cus­ing on the in­con­sis­tency-in-pub­li­ca­tion bias, which might be less of a con­cern in the an­i­mal welfare move­ment, you might want to rate the move­ment’s liter­a­ture on whether there is a ro­bust mea­sure of the de­crease in peo­ple’s con­sump­tion of meat. A lot of stud­ies rely on self-re­ported con­sump­tion of meat, which is very un­re­li­able, ver­sus gro­cery stores’ data on meat pur­chases. There are definitely sig­nifi­cant al­ter­a­tions you would need to make to these crite­ria. But again, with a holis­tic, col­lab­o­ra­tive effort among differ­ent re­searchers in the an­i­mal welfare move­ment, you could ar­rive at some­thing similar to what the NIH and the field of medicine have cre­ated.

This will not give you crys­tal clar­ity on what to do. Sub­jec­tive judg­ment will always be in­volved. But it does al­low for judg­ments that are made ex­plic­itly and with trans­parency around the un­der­ly­ing fac­tors that led to a cer­tain de­ci­sion. There still might be de­bates on whether you should do leaflet­ing, ver­sus on­line ads, ver­sus cor­po­rate cam­paigns. But hav­ing sec­tor-wide, ev­i­dence-grad­ing stan­dards al­lows peo­ple to de­bate spe­cific and im­por­tant un­der­ly­ing fac­tors and rea­sons.

Re­sults-based fi­nanc­ing mechanism­s

The sec­ond pro­posal I’ve taken from medicine and global de­vel­op­ment is this: High-qual­ity ev­i­dence about what works is sim­ply not enough [to spur ac­tion]. There are a ton of great stud­ies high­light­ing cost-effec­tive in­ter­ven­tions and or­ga­ni­za­tions in the effec­tive al­tru­ism com­mu­nity. GiveWell, J-PAL, and IDin­sight can all at­test to the fact that there is not enough up­take and scal­ing of pro­grams that have proven to be effec­tive.

Slide20

Part of the rea­son for that is there are not suffi­cient in­cen­tives for or­ga­ni­za­tions to de­liver against re­sults. In­tro­duc­ing re­sults-based fi­nanc­ing in the an­i­mal welfare move­ment has the po­ten­tial to in­cen­tivize the cre­ation of new solu­tions to in­tractable prob­lems. For in­stance, a lot of peo­ple in the move­ment have said, “It’s too hard to change in­di­vi­d­ual be­hav­ior. We just need to wait un­til clean meat gets so good and so cheap that ev­ery­one’s go­ing to tran­si­tion by them­selves.” But the re­al­ity is if you in­cen­tivize it with fi­nan­cial re­turns and pay­ment based on re­sults, you might get more in­no­va­tive solu­tions. We’ve seen analogs within global de­vel­op­ment. Se­cond, it en­sures that philan­thropic dol­lars go to­ward pro­grams that work.

There are a lot of differ­ent ways to adapt re­sults-based fi­nanc­ing mechanisms to the an­i­mal welfare move­ment. Some of our col­leagues in the global de­vel­op­ment space at In­stiglio have cat­e­go­rized those mechanisms into three broad buck­ets:

Slide22


1. Perfor­mance-based con­tracts be­tween a foun­da­tion or a gov­ern­ment and an im­ple­ment­ing or­ga­ni­za­tion. For these, a foun­da­tion or a gov­ern­ment pairs off with an im­ple­ment­ing or­ga­ni­za­tion, and the im­ple­ment­ing or­ga­ni­za­tion gets paid based on the ex­tent to which they achieve pre­de­ter­mined re­sults, as judged by an in­de­pen­dent eval­u­a­tor.

2. Pri­vate-sec­tor in­vestors that shoulder the risk. There’s a more com­plex mechanism that de-risks in­ter­ven­tions for philan­thropists. Pri­vate-sec­tor in­vestors put up risk cap­i­tal. If the pro­gram works, they get a pos­i­tive re­turn on their in­vest­ment. If it fails, then they lose that money. This al­lows a foun­da­tion to only pay for re­sults. [Here’s a real-world ex­am­ple of this form of fi­nanc­ing.]

3. Prizes to in­cen­tivize com­pe­ti­tion. There are prizes like the XPRIZE for adult liter­acy. Ba­si­cally, a huge pot of money in­cen­tivizes peo­ple to solve a par­tic­u­lar prob­lem — and then re­wards who­ever ul­ti­mately suc­ceeds with a pay­out.

Th­ese are early-stage hy­pothe­ses, but my propo­si­tion is that the an­i­mal welfare move­ment can learn a lot from more ma­ture so­cial jus­tice causes like medicine, pub­lic health, and global de­vel­op­ment. I think it makes sense for us to come to­gether as a com­mu­nity to start eval­u­at­ing which of those can be adapted to po­ten­tially ac­cel­er­ate an­i­mal welfare im­prove­ments for billions of non-hu­man an­i­mals.

Thank you.

Nathan Labenz [Moder­a­tor]: One ques­tion oc­curs to me right off the bat. Since we’re all hu­mans, it seems like there’s a more stan­dard unit of ac­count that can be put into place in medicine, whether it’s qual­ity-ad­justed life years or some­thing else like that. The con­cep­tual prob­lem I see is in try­ing to do some­thing similar for differ­ent an­i­mals, would they count the same, or would they count differ­ently? And do you think that this prob­lem is amenable to the frame­work that you’re propos­ing?

Neil: Great ques­tion. I think it’s a fun­da­men­tal one, but not some­thing that I talked about ex­plic­itly here. I do imag­ine some kind of “NIH for the an­i­mal welfare move­ment” do­ing some ba­sic re­search along those lines. So, there are large unan­swered or semi-an­swered ques­tions around how to value the lives of differ­ent types of an­i­mals. And there’s a lot of ba­sic sci­ence re­search that needs to be done around bet­ter un­der­stand­ing differ­ent types of an­i­mals’ abil­ity to feel pain and plea­sure.

I imag­ine that if you es­tab­lish an in­sti­tute with clear ev­i­den­tiary stan­dards, you could start to lay the foun­da­tion for an equiv­a­lent of dis­abil­ity-ad­justed life years (DALYS) in the an­i­mal welfare space. I know there are some re­searchers try­ing to do some­thing like that. But the big benefit of a pro­gram like this is it doesn’t rely on in­di­vi­d­ual re­searchers’ sub­jec­tive judg­ment. It en­forces across-the-board qual­ity stan­dards.

Nathan: Here’s a ques­tion from the au­di­ence. An­i­mal ad­vo­cacy re­searchers are study­ing how to in­fluence the eco­nomic be­hav­ior of peo­ple in com­mu­ni­ties. By con­trast, med­i­cal re­searchers are study­ing the biolog­i­cal effects of med­i­cal treat­ments in in­di­vi­d­u­als. How rele­vant do you think that differ­ence is?

Neil: I think that is a differ­ence, but med­i­cal re­searchers are also study­ing the effects of in­ter­ven­tions on pop­u­la­tions in pub­lic health. And a lot of those are re­ally analo­gous to in­ter­ven­tions pro­posed by an­i­mal welfare move­ment. So much of pub­lic health and global de­vel­op­ment is about be­hav­ior change.

For ex­am­ple, how do you get peo­ple to test them­selves for malaria or HIV/​AIDS? And once they’ve tested pos­i­tive, how do you get them to ad­here to the pro­gram and the treat­ment in the same way you want to be able to test?

[In the an­i­mal welfare move­ment], you want to be able to test the effec­tive­ness of in­ter­ven­tions to get peo­ple who have ex­pressed the de­sire to re­duce their meat con­sump­tion to ac­tu­ally do it. So much of that is about in­di­vi­d­ual be­hav­ior change and policy or reg­u­la­tory re­form. And those two things are quite analo­gous to large parts of both the med­i­cal and the de­vel­op­ment eco­nomics liter­a­ture, even if there are cer­tain seg­ments of the med­i­cal liter­a­ture that are more phys­iolog­i­cal.

I’d en­courage any­one who works in an­i­mal welfare full-time to please con­tact me. We’ve been work­ing with Lewis Bol­lard at the Open Philan­thropy Pro­ject to see if any of these ideas have legs and are salient for the an­i­mal welfare move­ment. I’m re­ally keen to get feed­back from folks who are work­ing on this 247.

No comments.