[Question] Why does EA use QALYs instead of experience sampling?

Back­ground: Ear­lier this year, I at­tended a great pre­sen­ta­tion by Natália Men­doça about ex­pe­rience sam­pling. Here’s the deck from her pre­sen­ta­tion.

A take­away from the pre­sen­ta­tion was that QALYs are con­structed in a way that skews cause pri­ori­ti­za­tion to­wards par­tic­u­lar causes. Alter­na­tive met­rics have differ­ent skews, so us­ing an al­ter­na­tive met­ric could lead to very differ­ent cause pri­ori­ti­za­tion.

For ex­am­ple, un­der the QALY frame­work, one year with “some prob­lems walk­ing about” is con­sid­ered to be about as bad as one year with “mod­er­ate anx­iety or de­pres­sion.”

For any­one who’s had some ex­pe­rience with de­pres­sion or anx­iety, as well as with “some prob­lems walk­ing about,” it should be ob­vi­ous that mod­er­ate de­pres­sion or anx­iety are (much) worse than mod­er­ate mo­bil­ity prob­lems, pound for pound. (Please reach out if you dis­agree with this, I want to pick your brain if you do.)

An al­ter­na­tive met­ric to QALYs is called ex­pe­rience sam­pling. Last month, Natália posted about ex­pe­rience sam­pling on the Fo­rum. The post was mod­er­ately up­voted, though no one com­mented on it.

A take­away from that post is that rol­ling out an ex­pe­rience-sam­pling frame­work seems very tractable.

This re­search di­rec­tion seems like plau­si­bly a high pri­or­ity for EA, as bas­ing cause pri­ori­ti­za­tion on a differ­ent met­ric could lead to no­tably differ­ent pri­or­ity causes.

In par­tic­u­lar, ex­pe­rience sam­pling ap­pears to give a higher weight to men­tal health di­s­or­ders than QALYs does, so it’s plau­si­ble that un­der an ex­pe­rience-sam­pling frame­work, men­tal health in­ter­ven­tions would be higher pri­or­ity than global health in­ter­ven­tions.

Given the po­ten­tial mag­ni­tude of this delta in pri­ori­ti­za­tion (be­tween the ex­pe­rience-sam­pling & QALY frame­works), it’s sur­pris­ing to me that there’s not been more in­ter­est in in­ves­ti­gat­ing al­ter­na­tives to the QALY in the EA com­mu­nity.

To be clear, I’m not claiming that the ex­pe­rience-sam­pling method is su­pe­rior to QALYs. I’m claiming that it is con­structed in an equally plau­si­bly way to the QALY, and that it prob­a­bly re­sults in dras­ti­cally differ­ent cause pri­ori­ti­za­tion. One po­ten­tially ro­bust path for­ward could be to split the differ­ence be­tween pri­ori­ti­za­tion im­plied by QALYs and pri­ori­ti­za­tion im­plied by ex­pe­rience sam­pling.

[Dis­clo­sure: In Fe­bru­ary 2019, I cor­re­sponded about the ex­pe­rience-sam­pling idea with Alex Foster of the EA Meta Fund. He said my points were “cer­tainly quite com­pel­ling,” but the cor­re­spon­dence fell off.

I heard later from an­other source that the EA Meta Fund didn’t end up get­ting ex­cited about the idea, though they didn’t say why not.]