Apples and oranges? Some initial thoughts on comparing diverse benefits

An im­por­tant part of effec­tive al­tru­ism is com­par­ing the value of differ­ent al­tru­is­tic en­deav­ors. Many al­tru­is­tic en­deav­ors bring about differ­ent kinds of good things, for in­stance pro­tec­tion of chil­dren from in­ca­pac­i­tat­ing dis­eases, and ex­tra years of qual­ity ed­u­ca­tion. To find the best causes, we need some way of eval­u­at­ing these next to one an­other. How much ex­tra ed­u­ca­tion in the de­vel­op­ing world is worth the same as an ex­tra year of healthy life?

An­swer­ing such ques­tions is no­to­ri­ously tricky, and GWWC faces an even harder prob­lem of an­swer­ing them in such a way that other peo­ple are happy to use our judge­ments, and thus our recom­men­da­tions. One can’t just opt out of an­swer­ing them, for the same rea­son one shouldn’t choose one’s part­ner by their so­cial se­cu­rity num­ber just be­cause it’s hard to weigh up a good sense of hu­mor against kind eyes.

So how can we make such eval­u­a­tions? GWWC re­search has been look­ing into it a bit, and this blog post will tell you some of what we think. We’ll look at var­i­ous meth­ods from eco­nomics and the so­cial sci­ences, and dis­cuss the ad­van­tages and dis­ad­van­tages of differ­ent ap­proaches.

Any at­tempt to com­pare the value of two things has two ba­sic parts. The first step is of­ten to par­cel out all of the main ways each item might have benefits. For in­stance, if a per­son in Malawi doesn’t con­tract malaria next year, this will trans­late to var­i­ous good things: less suffer­ing and more joy for that per­son, less stress for their fam­ily, less con­ges­tion at the lo­cal hos­pi­tal, a few thou­sand dol­lars of pro­duc­tive work, slightly differ­ent ex­pec­ta­tions among peo­ple nearby about how well their lives are likely to go.

This can go on for more steps—a few thou­sand dol­lars of pro­duc­tive work might be fac­tored out in terms of in­creased pros­per­ity for the fam­ily, and more pros­per­ity spread among the wider world. Pros­per­ity for the fam­ily might in turn be fac­tored out as bet­ter nu­tri­tion for them, more ed­u­ca­tion for their chil­dren, some in­vest­ments that will bring them fur­ther pros­per­ity, etc.

Another thing that of­ten hap­pens at this stage is that items are made more ab­stract. For in­stance one par­tic­u­lar girl’s re­cov­ery from schis­to­so­mi­a­sis might be equated to some num­ber of ‘qual­ity ad­justed life years’ or ‘QALYs’. This makes eval­u­a­tion more tractable. Now we can eval­u­ate many similar things at once, a lit­tle bit in­ac­cu­rately, in­stead of do­ing thou­sands of differ­ent eval­u­a­tions. For in­stance if we con­vert many differ­ent health in­ter­ven­tions into QALYs saved, then we can com­pare a QALY to a year of school, and we au­to­mat­i­cally have a com­par­i­son of lots of differ­ent health in­ter­ven­tions.

The sec­ond step is to ac­tu­ally com­pare the value of the items you end up with. This might in­volve for in­stance see­ing how money much a per­son is will­ing to pay to avoid some suffer­ing. You could do this by ob­serv­ing how much money and effort they in­vest in avoid­ing the flu, in flu sea­son. Or you might ask them di­rectly, ‘if you could pay $1000 to avoid this surgery, would you take that offer?’. Or in­stead of look­ing at money they will sac­ri­fice, you might just ask about how they feel. For in­stance you might ask a blind per­son re­peat­edly over a pe­riod how satis­fied they are with their life, and do the same with a per­son who is not blind.

Com­par­ing value can be done im­me­di­ately with­out the pre­vi­ous step of con­vert­ing things, or af­ter so many iter­a­tions of the pre­vi­ous step as to turn health and ed­u­ca­tion into un­rec­og­niz­able buck­ets of value. Many ways of com­par­ing the value of A against B have been de­vel­oped, es­pe­cially in the so­cial sci­ences. I have col­lected some of them into a menu, which de­scribes their up­sides and down­sides, and sug­gests when we might ex­pect them to be most ap­pro­pri­ate.

Note that the meth­ods in the menu are gen­er­ally com­par­ing benefits to iden­ti­fi­able groups of peo­ple. We may care about peo­ple for whom it’s hard to make these mea­sure­ments, such as fu­ture gen­er­a­tions, but we’ll need differ­ent or fur­ther method­ol­ogy to know how to com­pare these effects to the di­rect benefits we can mea­sure. Nonethe­less it’s valuable to un­der­stand how we should com­pare di­verse benefits to­day, and this may be a key com­po­nent in a more gen­eral anal­y­sis.

Also, we won’t usu­ally be able to choose freely from the list of meth­ods. GWWC will prob­a­bly not have the re­sources to go and find out how much an ad­di­tional par­ent im­proves the life out­comes for a per­son in ru­ral China. The hope is rather that this list might help guide choices of com­par­i­son when a few differ­ent meth­ods hap­pen to be available. I’ll give you a few ex­am­ples of eval­u­a­tion meth­ods on the list, then in the rest of this post I’ll de­scribe some of key choices it offers and men­tion some of the fac­tors that might sug­gest one choice over an­other.


To give you a con­crete idea of what we are talk­ing about, here are some ex­am­ples of how we might use differ­ent meth­ods from the list to figure out how good some­thing is for a per­son:

Willing­ness to pay: if a per­son is will­ing to pay $100 for a text­book, you can in­fer that they pre­fer hav­ing the text­book to hav­ing $100, and there­fore to other things they could buy for $100.

In­stan­ta­neous re­ports of sub­jec­tive wellbe­ing: a per­son’s phone pings them through­out the day and and asks some­thing like, ‘zero to ten, how good do you feel right now?’ Us­ing this method, we can build a he­do­nic pro­file for differ­ent in­di­vi­d­u­als and (hope­fully) dis­cover the cor­re­lates and causes of hap­piness, as well as their rel­a­tive im­por­tance.

The stan­dard gam­ble: A per­son is asked what prob­a­bil­ity of dy­ing they would be will­ing to ac­cept for an in­ter­ven­tion that would im­prove their health in some way.

Avert­ing be­hav­ior method: Use the to­tal costs peo­ple pay to avoid ex­tra sun­burn as a lower bound for how much a healthy ozone layer is worth to them.

What to elicit: prefer­ences, hap­piness, or an­other proxy of good?

Whether you ul­ti­mately care to op­ti­mize for good feel­ings, peo­ple get­ting what they want, or some­thing else is a con­tro­ver­sial moral ques­tion. I won’t ad­dress this ques­tion here ex­cept to note that differ­ent an­swers can lead us to pre­fer differ­ent mea­sure­ment tech­niques. For in­stance, it is rel­a­tively easy to ob­serve a per­son’s prefer­ence by look­ing at which things they choose. Their hap­piness in differ­ent situ­a­tions is harder to ob­serve, so you might do bet­ter just ask­ing them di­rectly if that’s what you care about. On the other hand, there are many rea­sons to sus­pect peo­ple’s an­swers to ‘how happy are you’ across time and be­tween peo­ple don’t ex­actly cor­re­spond to the true land­scape of hap­piness, so even if you cared about hap­piness you might take a per­son’s prefer­ences se­ri­ously, if you thought they liked to be happy.

When the situ­a­tion you want to eval­u­ate will have ram­ifi­ca­tions for other peo­ple than the one(s) di­rectly af­fected, the prefer­ences and feel­ings of the peo­ple in­volved will be a poor proxy for the over­all good or bad done. For in­stance, sup­pose you are con­sid­er­ing ed­u­cat­ing a young woman in a de­vel­op­ing coun­try. This should help her, but it is thought to also have large effects on her chil­dren and oth­ers in her com­mu­nity. If you are hop­ing to even­tu­ally lift her re­gion out of poverty through this type of ac­tion, you are rely­ing mostly on good from such in­di­rect effects. In this case, dis­cov­er­ing how much the woman would like to be ed­u­cated is not very use­ful. Even if she wouldn’t like to be ed­u­cated at all it may be a good in­ter­ven­tion. Similarly, if a per­son does not un­der­stand or care very much about their own fu­ture, their views on the value of im­prov­ing it are not very use­ful. Ask­ing a child how much they would like to be de­wormed is not very use­ful.

One might deal with this in the ear­lier step, by equat­ing ‘one child de­wormed’ into a small amount of health, ed­u­ca­tion, and phys­i­cal de­vel­op­ment. A child might know bet­ter how much they dis­like be­ing sick, or adults may be able to shed light on the long term costs of mal­nour­ish­ment. Or you might just ab­stract a child’s ill­ness to be generic ill­ness, and ask how much bet­ter in­formed peo­ple wish to avoid such ill­ness.

How to elicit: ask or ob­serve?

In many situ­a­tions what peo­ple think or say they value differs from what they choose given the op­por­tu­nity. There are many pos­si­ble rea­sons for this, from hypocrisy to difficulty en­visag­ing the situ­a­tion of in­ter­est ac­cu­rately and with­out a bi­ased fo­cus. To the ex­tent one be­lieves a per­son’s be­hav­ior bet­ter in­di­cates their ‘real’ prefer­ences, ob­serv­ing choices will be more ac­cu­rate than ask­ing. How­ever a per­son’s speech may bet­ter re­flect their prefer­ences on re­flec­tion than their be­hav­ior, which usu­ally com­prises a com­bi­na­tion of con­sid­ered prefer­ences and un­en­dorsed urges. If you wish to re­spect only the former, this is some rea­son to ask in­stead of ob­serv­ing.

Ask­ing also has the benefit of mak­ing it rel­a­tively easy to iso­late the is­sue of in­ter­est; nat­u­ral situ­a­tions rarely al­low one to pin a prefer­ence to a spe­cific fac­tor, as so many fac­tors change. For in­stance, if you see that par­ents choose to send their chil­dren to school at sub­stan­tial cost, you might like to in­fer that they ex­pect the ed­u­ca­tion to benefit their chil­dren sub­stan­tially. But it could also be that they ex­pect their chil­dren to look poor if they don’t go to school, and be treated worse. Then if the school didn’t ex­ist their chil­dren would not be worse off—they are just worse off if the school does ex­ist, and they don’t go to it.

Who to elicit it from?

The best peo­ple to eval­u­ate A and B would seem to be peo­ple who have ex­pe­rienced both, and are cur­rently ei­ther ex­pe­rienc­ing both or nei­ther. This way they will hope­fully be fa­mil­iar with both items, and not too bi­ased by the de­tails of one be­ing more salient at the mo­ment.

Often it will be hard to find peo­ple in ex­actly that situ­a­tion though. You will of­ten have a trade off be­tween peo­ple who are fa­mil­iar with one item and not the other (so know more but might be bi­ased) and peo­ple who are not fa­mil­iar with ei­ther (so know lit­tle, but prob­a­bly less bi­ased).

This is all only an is­sue if you are in­ter­ested in eval­u­at­ing A and B for the per­son di­rectly in­volved. If you want to know about the over­all so­cial benefits of fewer peo­ple be­ing blind for in­stance, com­pared to fewer peo­ple be­ing crip­pled, you may be bet­ter off ask­ing some­one who is nei­ther blind nor crip­pled.

Com­par­i­son or sep­a­rate eval­u­a­tion?

Sup­pose you want to know whether it is bet­ter to get a five year old to go to school for an ex­tra year, or to avoid them get­ting malaria for one year, and you in­tend to figure it out by ask­ing their par­ents. One way you could do it is to ask each par­ent ‘do you think your child will be bet­ter off if she goes to school this year and gets malaria, or if she doesn’t go to school or get malaria this year?’. Another is to ask some par­ents how valuable they think go­ing to school is, and other par­ents how valuable they think avoid­ing malaria is, then com­par­ing those val­ues.

If you ask a per­son to com­pare two things at once, you will of­ten get differ­ent an­swers to if you get them to just com­pare one or the other. One rea­son is that when they can see things side by side, they com­pare on the char­ac­ter­is­tics that seem salient. When they can only see one thing, they tend to ig­nore char­ac­ter­is­tics if they don’t feel like they can get any grasp on how good the ac­tual num­ber for the char­ac­ter­is­tic is, even if it seems im­por­tant. In our ear­lier ex­am­ple, sup­pose that for both the school and the health in­ter­ven­tions, the par­ents are told how much ex­tra in­come their child is likely to have in the fu­ture as a re­sult of the pro­ject, and zero-to-ten how much their child will like it at the time. Sup­pose ed­u­ca­tion will bring about much more fu­ture in­come than the health in­ter­ven­tion on offer, but makes the child 210 happy in­stead of 810 happy. Then when the par­ents look at both in­ter­ven­tions to­gether, they might weigh up the pre­sent costs and fu­ture gains and choose the ed­u­ca­tion in­ter­ven­tion. When they look at the pro­jects sep­a­rately though, they might have fairly similar men­tal images of two differ­ent lar­gish gains in fu­ture in­come, whereas hap­piness looks quite differ­ent on the given scale. So im­plic­itly they fo­cus more on the differ­ence in hap­piness, and so the com­par­i­son could come out the other way.


Th­ese have been sev­eral of the con­sid­er­a­tions which make some eval­u­a­tion meth­ods more ap­pro­pri­ate than oth­ers, at differ­ent times. The so­cial sci­ence liter­a­ture has a lot more to say. Hope­fully our menu nonethe­less sum­ma­rizes enough im­por­tant points to strengthen our re­search com­par­ing in­ter­ven­tions, and can be built upon in the fu­ture as we carry out such com­par­i­sons.

Cross­posted from the Giv­ing What We Can blog

No comments.