Prospecting for Gold—EAGxOxford 2016 - edited transcript

Novem­ber 18, 2016

In this 2016 talk, Oxford Univer­sity’s Owen Cot­ton-Bar­ratt dis­cusses how effec­tive al­tru­ists can im­prove the world, us­ing the metaphor of some­one look­ing for gold. He dis­cusses a se­ries of key effec­tive al­tru­ist con­cepts, such as heavy-tailed dis­tri­bu­tions, diminish­ing marginal re­turns, and com­par­a­tive ad­van­tage.

A write-up of Owen’s talk is be­low, which we have mod­er­ately ed­ited for clar­ity.

Effec­tive al­tru­ism as min­ing for gold

The cen­tral metaphor run­ning through this ar­ti­cle is think­ing about Effec­tive Altru­ism as ‘min­ing for gold’. I’m go­ing to use this metaphor to illus­trate differ­ent points. Gold, here, is stand­ing in for what­ever it is that we truly value. This could in­clude mak­ing more peo­ple happy and well-ed­u­cated, try­ing to avert a lot of suffer­ing, or try­ing to in­crease the prob­a­bil­ity that hu­man­ity makes it out to the stars. When you read “gold”, take a mo­ment to think about what you value (it may not be just one par­tic­u­lar thing) and put that in place of the gold.

Figure 2 is a photo of Vik­tor Zh­danov who I learned about in Will MacAskill’s book ‘Do­ing Good Bet­ter’. He was a Ukrainian biol­o­gist, who was in­stru­men­tal in get­ting an erad­i­ca­tion pro­gram for smal­l­pox to oc­cur. As a re­sult, he was prob­a­bly coun­ter­fac­tu­ally re­spon­si­ble for sav­ing tens of mil­lions of lives.

Ob­vi­ously, we don’t all achieve as much as this. But by look­ing at ex­am­ples like Zh­danov we can no­tice that some peo­ple man­age to get a lot more gold, or what­ever it is we al­tru­is­ti­cally value, than oth­ers. This is rea­son enough to make us ask ques­tions like:

What is it that gives some peo­ple bet­ter op­por­tu­ni­ties than oth­ers? How can we go and find op­por­tu­ni­ties like that?

Others have dis­cussed where the gold is and trea­sure maps to find it. That is not the fo­cus of this ar­ti­cle. In­stead I’m go­ing to fo­cus on the tools and tech­niques that we can use for lo­cat­ing gold, rather than try­ing to give my view of where it is di­rectly.

I want to be­gin by elab­o­rat­ing on why I am even us­ing a metaphor. The things we care about are big, com­pli­cated, and valuable; so why would I try and re­duce that down to gold? It is be­cause I want this ar­ti­cle to fo­cus on tech­niques, tools, and ap­proaches that we can use. If you have com­plex val­ues, these would just keep pul­ling your at­ten­tion. But a lot of the things that we might do to iden­tify where valuable things are, and how to go and achieve them, are con­stant, re­gard­less of what the valuable thing is. In re­plac­ing them with a sim­ple stand-in for value, I think it helps to put the fo­cus on a level above dis­cus­sions of what is valuable.

Gold is un­evenly spread

My first point is that our gold, like literal gold, is pretty un­evenly spread around the world. There are lots of places with al­most no gold at all, and then there are a few places where there is a big seam of gold run­ning into the ground. This has a num­ber of im­pli­ca­tions. One is that we would re­ally like to find those seams.

Heavy-tailed distributions

The next point is about sam­pling. For ex­am­ple, if I want to know roughly how tall peo­ple are, sam­pling five peo­ple and tak­ing their av­er­age height would not be a bad method­ol­ogy. How­ever, if I want to know on av­er­age how much gold there is in the world, sam­pling five ran­dom places and mea­sur­ing that is not a good method­ol­ogy. It’s quite likely I will find five places where there is no gold and sig­nifi­cantly un­der­es­ti­mate. Or pos­si­bly, one of them will have a load of gold and I will have an in­flated sense of how much gold there is in the world.

This statis­ti­cal prop­erty is of­ten referred to as hav­ing a heavy-tail on the dis­tri­bu­tion. On the left in Figure 6 we have a dis­tri­bu­tion with­out a heavy-tail. This cor­re­sponds to a range of differ­ent amounts of gold in differ­ent places, but none of them has sub­stan­tially more or less than typ­i­cal.

In con­trast, on the right, we have a heavy-tail dis­tri­bu­tion. It looks similar to the dis­tri­bu­tion on the left, apart from this long tail, reach­ing vast amounts of gold with prob­a­bil­ities that are not dy­ing off very fast. This has im­pli­ca­tions.

Figure 7 is an­other way of look­ing at these dis­tri­bu­tions. In this case, I have or­dered, from left to right, places in in­creas­ing amounts of how much gold they have. The per­centiles are on the hori­zon­tal axis and the amount of gold on the ver­ti­cal axis. In this case, the coloured in area be­neath the graph cor­re­sponds to the to­tal amount of gold. On the left, for the dis­tri­bu­tion that is not heavy-tailed, we see the gold is evenly spread across differ­ent places. If we want to get most of the gold, what is im­por­tant is get­ting to as many differ­ent places as pos­si­ble.

So­lar power is like the graph on the left. Some places get more sun­light than oth­ers, but the amount of so­lar power you gen­er­ate de­pends more on the to­tal num­ber of so­lar pan­els, rather than on where you place them.

On the right-hand graph, we have a dis­tri­bu­tion where a lot of the area is in the spike on the right-hand side. This means a lot of the gold, or what­ever it is we find valuable, comes from the top per­centiles of the dis­tri­bu­tion, which are un­usu­ally good.

Although I am not a ge­ol­o­gist, it is my un­der­stand­ing that literal gold is dis­tributed like the heavy-tailed graph on the right. We might then ask, is this also true of op­por­tu­ni­ties to do good in the world? Here is some sup­port for this.

Heavy-tailed prop­erty in the wild

When we look at the work in all its com­plex­ity, we see dis­tri­bu­tions with this heavy-tail prop­erty ap­pear­ing in a num­ber of differ­ent places. There are some the­o­ret­i­cal rea­sons to ex­pect cer­tain types of dis­tri­bu­tions to arise. Em­piri­cally, look­ing at things like in­come dis­tri­bu­tions around the world, we see the heavy-tail prop­erty. [Figure 8]

Ob­vi­ously, there are lots of things that don’t have this prop­erty. But the more we look at things that are com­plex sys­tems with lots of in­ter­ac­tions, the de­gree to which we see this prop­erty in­creases. This is a big fea­ture of lots of ways that we try and in­ter­act to im­prove the world.

Sim­ply look­ing ex­plic­itly at op­por­tu­ni­ties to do good, I can find a cou­ple of rea­sons why I am con­vinced we get some of this prop­erty.

One rea­son is just con­vinc­ing ar­gu­ments. If I care about stop­ping peo­ple starv­ing, which I do, I could ask: Should I be in­ter­ested in di­rect famine re­lief and try­ing to get food to peo­ple who are starv­ing to­day com­pared to some­thing more spec­u­la­tive? Some ar­gue that it would be more effec­tive to fo­cus on re­search­ing solu­tions for feed­ing vast num­bers of peo­ple if agri­cul­ture col­lapses, which I per­son­ally have found con­vinc­ing. This is an ex­treme ex­am­ple and not some­thing we usu­ally think about. But, limit­ing my­self to just try­ing to feed peo­ple, one of the mechanisms looks much more effec­tive than the other.

Heavy-tailed prop­erty in op­por­tu­ni­ties for good

This data from DCP2 [Figure 9], has tried to es­ti­mate the cost-effec­tive­ness of lots of differ­ent de­vel­op­ing world health in­ter­ven­tions. The x-axis is on a log scale, so these have been put into buck­ets and each column is on av­er­age ten times more effec­tive to the ones on its left. Thus, the right­most column is about 10,000 times more effec­tive than the left­most column. Just within this one area of global health, we have man­aged to get good enough data that we can es­ti­mate these things, show­ing there is just a very wide range of cost-effec­tive­ness.

The im­pli­ca­tions of this are that if we want to get gold, we should fo­cus on find­ing seams which con­tain huge amounts of gold. This might give us sur­pris­ing con­clu­sions. We may be less ex­cited to dis­cover some­thing is at the 90th per­centile be­cause be­fore we knew any­thing, it might have been any­where on the dis­tri­bu­tion. Since most of the pos­si­ble value comes from the 99th per­centile, then dis­cov­er­ing that some­thing is only at the 90th per­centile would be good to know but would make us think less well of it. This is the case if you have got a fairly ex­treme dis­tri­bu­tion, but it’s in­ter­est­ing to see how you can get these coun­ter­in­tu­itive prop­er­ties with heavy-tailed dis­tri­bu­tions.

Another im­pli­ca­tion is that a naïve em­piri­cism of, “we’ll just do a load of stuff and see what comes out best”, is not go­ing to be enough, be­cause of the sam­pling is­sue. It’s not pos­si­ble to sam­ple enough times and mea­sure the out­comes well enough to judge how effec­tive things are re­ally go­ing to be.

To max­imise gold, want…

If we want to get as much gold as pos­si­ble, we want to go to a place where there is lots of gold. We want to have the right tools for get­ting the gold out and we want to have a great team that is go­ing to be us­ing those tools. We can port this anal­ogy over to op­por­tu­ni­ties to do­ing good as well. We can roughly mea­sure the effec­tive­ness of the area or type of thing that we are do­ing. We can mea­sure the effec­tive­ness of the in­ter­ven­tions we are im­ple­ment­ing to cre­ate value in an area, rel­a­tive to other in­ter­ven­tions in the area. We can also mea­sure the effec­tive­ness of the team or or­gani­sa­tion which is run­ning the im­ple­men­ta­tion, rel­a­tive to how well other teams might im­ple­ment such an in­ter­ven­tion.

Value is roughly multiplicative

If we have these differ­ent di­men­sions then the to­tal value of the work done is equal to the product of these di­men­sions. In Figure 11 this is rep­re­sented by vol­ume, which we want to be max­imis­ing. This means we want to be do­ing rea­son­ably well on each of the differ­ent di­men­sions, or at least not ter­ribly on any of the di­men­sions. Some im­pli­ca­tions here might be that if we have an area and an in­ter­ven­tion that we are ex­cited about, but we can only find a mediocre team work­ing on it, it may be bet­ter not to sup­port them, but to try and get some­body else work­ing on it. Alter­na­tively, we could do some­thing to re­ally im­prove that team. Similarly, we might not want to sup­port even a great team if they are work­ing in an area that doesn’t seem im­por­tant.

Recog­nis­ing gold

In this sec­tion, I am go­ing to dis­cuss the tools and tech­niques for iden­ti­fy­ing where the gold is. A nice prop­erty of literal gold is that when you dig it up, it’s easy to recog­nise. In our al­tru­is­tic efforts, we of­ten have to deal with cases where we don’t have this. We don’t have the gold, so we have to try to in­fer its ex­is­tence by us­ing differ­ent tools. This fact is like the dark mat­ter of value.

This fact in­creases the im­por­tance of ap­ply­ing those tools dili­gently. Ac­tu­ally, the pic­ture in Figure 12 is iron pyrite, not gold. So just be­cause some­one says, “Hey, this is gold”, it doesn’t mean we should always take their word for it, al­though it may provide some ev­i­dence. We want to have great tools for iden­ti­fy­ing par­tic­u­larly valuable op­por­tu­ni­ties and be­ing able to differ­en­ti­ate and say, “Okay, ac­tu­ally this thing, al­though it has some as­pects of value, may not be what we want to pur­sue.”

Run­ning out of easy gold

If you first go to an area where no­body has been be­fore, then the seams of gold of­ten have lit­tle nuggets of gold just ly­ing on the ground. It is ex­tremely easy to get gold. So you have some peo­ple go in, they do this for a bit and they run out of all the gold on the ground.

Now, if they want to get more gold, maybe more peo­ple come along bring more shov­els. Its a bit more work, but you can still get gold out [Figure 15].

Then you dig deeper, un­til you can’t get in with shov­els any­more, so you need big­ger teams and heav­ier ma­chin­ery to get the gold [Figure 16]. You can still get gold, but it is more work for each lit­tle bit that you are get­ting. This is the gen­eral phe­nomenon of diminish­ing re­turns on work that you are putting in. This con­cept comes up in a lot of differ­ent places, so it is worth hav­ing an un­der­stand­ing of it.

This, like sev­eral of the things I am go­ing to be talk­ing about is a con­cept na­tive to eco­nomics. In some cases I am merely just pul­ling this from eco­nomics, but in oth­ers there is a lit­tle bit more mod­ifi­ca­tion on the con­cept.

For in­stance, I think this is par­tic­u­larly the case in global health. I un­der­stand that 15-20 years ago, mass vac­ci­na­tion were ex­tremely cost-effec­tive and prob­a­bly the best thing to be do­ing. Then the Gates Foun­da­tion came in and funded a lot of the mass vac­ci­na­tion in­ter­ven­tions. Now, the most cost-effec­tive in­ter­ven­tion is less cost-effec­tive than mass vac­ci­na­tions, be­cause we have picked the low hang­ing fruit. Similarly, if in AI safety, writ­ing the first book on su­per­in­tel­li­gence is a pretty big deal com­pared to the 101st book on the topic.

Pre­vi­ously, I dis­cussed how we could fac­tor the effec­tive­ness of or­gani­sa­tions into: the ar­eas which it was work­ing on, the in­ter­ven­tion it is pur­su­ing, and the team work­ing on it. Now, I am go­ing fo­cus on the first fac­tor—how to as­sess the effec­tive­ness of a cause area. In try­ing to as­sess the area, I am go­ing to give a fur­ther fac­tori­sa­tion, split­ting this into three differ­ent di­men­sions.


The first of these di­men­sions is scale. All else be­ing equal, we would pre­fer to go some­where where there is a lot of gold, rather than a lit­tle bit of gold. Most likely, per unit effort, we are go­ing to get more gold if we do that.


The sec­ond di­men­sion is tractabil­ity. We would like to go some­where where we make more progress per unit work. Ideally, where it is easy to dig that ground, rather than try­ing to get your gold out of a swamp.


The third di­men­sion is un­crowd­ed­ness. This has some­times been called ne­glect­ed­ness. Per­son­ally, I find that term am­bigu­ous be­cause some­times peo­ple use ‘ne­glect­ed­ness’ to mean that this is an area which we should al­lo­cate more re­sources to. What I mean here that there are not many peo­ple look­ing at it. All else be­ing equal, we would rather go to an area where peo­ple have not already picked up the nuggets of gold on the ground, than one where they have (where the re­main­ing gold is quite hard to ex­tract).

Ideally, we would like to be in the world where there is lots of gold, where it’s easy to get out, and no­body has taken any of it. But, we are rarely go­ing to be in that ex­act ideal cir­cum­stance.

How can we trade these off these di­men­sions against each other?

I am go­ing to at­tempt to make the ques­tion of ‘how can we trade these di­men­sions against each other?’ pre­cise [Figure 20].

If you are un­ac­cus­tomed to think­ing in terms of deriva­tives, just ig­nore the ‘ds’ here. On the left in Figure 20, is the value of a lit­tle bit of ex­tra work. This is what we care about if we are try­ing to as­sess which of these differ­ent ar­eas we should do more work on.

On the right is a fac­tori­sa­tion which is math­e­mat­i­cally triv­ial and looks like it just makes things more com­pli­cated. I’ve taken the ex­pres­sion on the left and added in a load of things which can­cel each other out. But I hope I can jus­tify this de­com­po­si­tion by virtue of it be­ing eas­ier to in­ter­pret and mea­sure. So I’m go­ing to pre­sent the case for why I think it is.

The first term is mea­sur­ing the amount of value you get for solv­ing an ex­tra one per­cent of a solu­tion. This roughly tracks how much of a big deal the whole prob­lem area that you are look­ing at is. I think that is a pretty pre­cise ver­sion of the no­tion of scale.

The sec­ond term is a bit more com­pli­cated. It is an elas­tic­ity that is mea­sur­ing, for a pro­por­tional in­crease in the amount of work be­ing done, what pro­por­tion of a solu­tion that gives you.

The fi­nal term just can­cels to one over the to­tal amount of work be­ing done, so it is very nat­u­rally a mea­sure of un­crowd­ed­ness.

By mak­ing a pre­cise ver­sion of this kind of scale, tractabil­ity, un­crowd­ed­ness frame­work we can avoid peo­ple hav­ing differ­ent char­ac­ter­i­sa­tions for differ­ent terms. Although, there have been some no­tions of tractabil­ity that don’t all line up with this, the idea of mea­sur­ing how much more work gets you to­wards a solu­tion, is fairly well cap­tured here.

All three di­men­sions matter

I think all of these di­men­sions mat­ter. This means, we prob­a­bly don’t want to work on some­thing that does ter­ribly on any of the di­men­sions. I am not go­ing to spend an hour helping a bee, even if no­body else is helping it and it would be pretty easy to help, be­cause the scale of it is pretty small. I also don’t think we should work on per­pet­ual mo­tion ma­chines, even though ba­si­cally no­body is work­ing on it and it would be fan­tas­tic is we suc­ceeded, be­cause it seems like it is not tractable.

This might give us a warn­ing against work­ing on cli­mate change, be­cause at a global scale it gets a lot of at­ten­tion as a prob­lem. I am go­ing to add caveats to this point. One is that this is go­ing to be true while we think that there are other prob­lems which are just sig­nifi­cantly more un­der-re­sourced. Another is that you think you might have an ex­cep­tion if you have a much bet­ter way of mak­ing progress of the prob­lem of cli­mate change than the typ­i­cal work that is done on it.

Even so, we should find it sur­pris­ing that I am mak­ing a state­ment like “cli­mate change is not a high pri­or­ity area”. This sounds con­tro­ver­sial and we should be scep­ti­cal of this. But the term ‘high-pri­or­ity’ is a bit over­loaded, so I want to dis­t­in­guish that more.

Ab­solute and marginal priority

If we have two places where there is gold in the ground and we ask: “Where should we send peo­ple if we want to get gold?” The an­swer is go­ing to de­pend. Per­haps we send the first per­son to the place on the right in Figure 22; there is only a lit­tle bit of gold but it is re­ally easy to get out. Then we send the next ten peo­ple to the place on the left, be­cause there is more to­tal gold there. The first per­son will already have re­trieved most of the gold on the right and we want more peo­ple in to­tal work­ing on the place on the left. Which of these two places is the higher pri­or­ity? It de­pends on which ques­tion you’re ask­ing.

Th­ese num­ber are made up, but we might have some dis­tri­bu­tion like this on the left in Figure 23. When ask­ing the ques­tion “How much should the world spend on an area in to­tal?” we get a dis­tri­bu­tion where per­haps cli­mate change looks very big.

If we ask in­stead, “How valuable is marginal spend­ing?” The graph might look quite differ­ent be­cause it sig­nifi­cantly de­pends on how much is already be­ing spent. The dot­ted lines in Figure 23 might rep­re­sent how much is already be­ing spend. Then the graph of the right is a func­tion of how much should be spent in to­tal, how much is already be­ing spent and what the marginal re­turns are—what the curve looks like there.

I think both of these are im­por­tant no­tions, and which one we should use should de­pend on what we are talk­ing about. If we are hav­ing a con­ver­sa­tion about what we as in­di­vi­d­u­als or as small groups should do, it is ap­pro­pri­ate to use this no­tion of marginal pri­or­ity and how much ex­tra re­sources help. If we are talk­ing about what we col­lec­tively as a so­ciety or the world should do, it is of­ten cor­rect to talk about ab­solute pri­or­ity and how much re­sources ought to be in­vested, to­tal.

For most things here I have been ex­tremely ag­nos­tic about what out view of value is. How­ever, for this point, I am go­ing to start mak­ing more as­sump­tions. Many peo­ple have the view that we want to try and make as much value over the long term as we can. If you don’t have that view, you can just treat this as a hy­po­thet­i­cal. If you have not thought about it, it is a pretty in­ter­est­ing and im­por­tant ques­tion, and worth spend­ing some time on.

Sup­pose we do care about cre­at­ing as much value in the long term as pos­si­ble. In our gold metaphor, that might mean want­ing to get as much gold as pos­si­ble even­tu­ally, rather than just try­ing to get as much gold out of that ground this year.

Long-term gold

Some tech­nolo­gies may be de­struc­tive; we can use dy­na­mite, which get us lots of gold now, but it also blows up some gold which we never can get later. That could be good if the fo­cus is on try­ing to get gold in the short term, but it could be bad from this even­tual gold per­spec­tive.

We could de­velop some tech­nolo­gies that are similarly effi­cient but less de­struc­tive. There are go­ing to be some peo­ple in the world who do care about cre­at­ing as much gold as pos­si­ble in the short term. They are go­ing to use whichever tech­nol­ogy is the most effi­cient for that. So one of the ma­jor drivers of how much gold is even­tu­ally ex­tracted is the or­der and se­quence in which the tech­nolo­gies are de­vel­oped. If we dis­cover dy­na­mite first, peo­ple are go­ing to use dy­na­mite and de­stroy a lot of the gold. On the other hand, if we dis­cover the drill first, then by the time dy­na­mite comes along the at­ti­tude will be “Well, why would be use that? We have this fan­tas­tic drill.”

Philoso­phers like Nick Bostrom have used this to ar­gue for try­ing to de­velop so­cietal wis­dom and good in­sti­tu­tions for de­ci­sion mak­ing, be­fore de­vel­op­ing tech­nolo­gies or progress which might threaten the long-run tra­jec­tory of civ­i­liza­tion. As well as try­ing to fo­cus on differ­en­tially aiming to de­velop tech­nolo­gies which en­hance the safety of new de­vel­op­ments be­fore any­thing that is driv­ing risk.

Work­ing together

In this sec­tion I am go­ing to dis­cuss how all this is a col­lab­o­ra­tive en­deav­our. The idea is that we are not all in­di­vi­d­u­ally say­ing: “I need to work out where the most gold is, that’s the most ne­glected and tractable. I per­son­ally am go­ing to do just that.” There are many peo­ple who are think­ing like this and there are more ev­ery year. I am re­ally ex­cited about this, but we need to learn how to co­op­er­ate.

Largely, we have the same or similar views on what to value. Maybe some think that silver mat­ters too, not just gold, but we all agree that gold mat­ters. We want to be able to co­or­di­nate and make sure that we are get­ting peo­ple work­ing on the things which make the most sense for them.

Com­par­a­tive advantage

In Figure 25, I have Harry, Hermione, and Ron and they have three tasks they need to do to get some gold. They need to do some re­search, mix some po­tions, and do some wand work. Hermione is the best at ev­ery­thing, but she doesn’t have a time-turner so she can’t do ev­ery­thing. So we need to have some way of dis­tribut­ing the work.

This is the idea of com­par­a­tive ad­van­tage. Hermione has an ab­solute ad­van­tage on all of these tasks, but it would be a waste for her to go and work on po­tions be­cause Harry is not so bad at po­tions. No­body else is good at do­ing re­search, so we should prob­a­bly put her on this.

This is a tool that we can use to help guide our think­ing about what we should do as in­di­vi­d­u­als. For ex­am­ple, if I think some tech­ni­cal work is the most valuable thing to be do­ing but I would be pretty mediocre at it, and in­stead I am a great com­mu­ni­ca­tor, then maybe I should go into try­ing to help tech­ni­cal re­searchers in that do­main com­mu­ni­cate their work to get more peo­ple en­gage with it and bring in more fan­tas­tic peo­ple.

Com­par­a­tive ad­van­tage at differ­ent levels

Now we have ap­plied this at an in­di­vi­d­ual level, we can also ap­ply it at the group level. We can no­tice that differ­ent or­gani­sa­tions or groups may be bet­ter placed to take differ­ent op­por­tu­ni­ties.

On an even more spec­u­la­tive level, we can ap­ply this idea to time. We can ask our­selves: “What are we, to­day, par­tic­u­larly well suited to do, com­pared to peo­ple in the past and peo­ple in the fu­ture?” We can­not change what those in the past did, but we can make a com­par­i­son of what our com­par­a­tive ad­van­tage is rel­a­tive to peo­ple in the fu­ture. If there are go­ing to be differ­ent pos­si­ble challenges in the fu­ture that we need to meet, it makes sense that we should be work­ing on the early ones. Be­cause if challenges are com­ing next year, the peo­ple five year later just do not have a chance to work on that.

Another con­sid­er­a­tion, is that we have a po­si­tion to in­fluence how many fu­ture peo­ple there will be who are in­ter­ested in and work on these challenges. We have more in­fluence over that than peo­ple in the fu­ture do, so it may make sense as a thing for us to fo­cus on.

Build­ing a map together

Another par­tic­u­larly im­por­tant ques­tion is how to work stuff out. The world is big, com­pli­cated, and messy. We can­not ex­pect all of us, in­di­vi­d­u­ally, to work out perfect mod­els of it, in fact, it is too com­pli­cated for us to ex­pect any­body to do this. Maybe, we are all walk­ing around with the lit­tle ideas, which, in my metaphor, are puz­zle pieces for a map to find the gold. We want in­sti­tu­tions for as­sem­bling these into a map. It is com­pli­cated be­cause some peo­ple have puz­zle pieces which are from the wrong puz­zle and they don’t track where the gold is. Ideally, we want out in­sti­tu­tions to filter these out and only as­sem­ble the cor­rect pieces to guide us where we want to go.

As a so­ciety, we have to deal with this prob­lem in a num­ber of differ­ent do­mains and we have de­vel­oped differ­ent in­sti­tu­tions for do­ing this. There is the peer re­view pro­cess in sci­ence, Wikipe­dia for ag­gre­gat­ing knowl­edge, Ama­zon re­views ag­gre­gate knowl­edge on which prod­ucts are good, and democ­racy lets us ag­gre­gate prefer­ences over many differ­ent peo­ple to try and choose what is go­ing to be good.

Of course, none of these in­sti­tu­tions is perfect and this is a challenge. This is like a wrong puz­zle piece that made it into the di­alogue. This come up in many cases, for ex­am­ple, the repli­ca­tion crisis in part of psy­chol­ogy, van­dal­ism in Wikipe­dia, and fake re­views on Ama­zon that make some prod­ucts looks good and oth­ers bad.

It may be the case that we can adapt an ex­ist­ing in­sti­tu­tion for our pur­pose of try­ing to ag­gre­gate knowl­edge about what are the ways to go and do the most good. But we may want some­thing a bit differ­ent and maybe some­one read­ing this is go­ing to do some work on com­ing up with valuable in­sti­tu­tions for this. I think this is a re­ally im­por­tant prob­lem that is go­ing to be­come more im­por­tant for us to deal with as the effec­tive al­tru­ism com­mu­nity grows.

Good lo­cal norms

Another thing which can help con­struct a bet­ter pic­ture, is try­ing to have good lo­cal norms. We tell peo­ple the ideas we have and other peo­ple may start listen­ing. They may just listen based on the charisma of the per­son talk­ing, rather than the truth­ful­ness of the puz­zle piece. But we would like to pro­mote the spread of good ideas, in­hibit­ing the spread of bad ones, and en­courag­ing origi­nal con­tri­bu­tions.

To achieve the first two aims, we can rely on au­thor­ity. We can take the at­ti­tude: “Well, we’ve worked out this stuff. We’re to­tally con­fi­dent about this and now we won’t ac­cept any­thing else.” But this isn’t helpful in get­ting new ideas.

Pay at­ten­tion to why we be­lieve things

Some­thing we can do is here is pay at­ten­tion to why you be­lieve some­thing. Do you be­lieve it be­cause some­body else told you? Do you be­lieve it be­cause you have re­ally thought this through care­fully and worked it out for your­self? There is of­ten a blur be­tween these two cases and we of­ten ac­cept rea­sons given to us with­out deeply ex­am­in­ing the ar­gu­ment.

It is use­ful to be hon­est with your­self about this and then also com­mu­ni­cate it to other peo­ple. If it is the case that you be­lieve this be­cause Joe Bloggs has told you, but Joe is a pretty care­ful guy and he is dili­gent about check­ing his ar­gu­ments, you can com­mu­ni­cate this. Or it could be the case that you cut this puz­zle piece your­self.

Cut­ting it your­self doesn’t mean we should nec­es­sar­ily have higher cre­dence in it. Pre­vi­ously, I have worked things out and thought I have proved things, be­fore find­ing a mis­take in my proof. So you can sep­a­rately keep track of the level of cre­dence you have in some­thing and why you be­lieve it.

More­over, our in­di­vi­d­ual and col­lec­tive rea­sons for be­liev­ing things can differ. Con­sider the state­ment: “It costs about $3,500 to save a life from malaria”. This is broadly be­lieved across the effec­tive al­tru­ism com­mu­nity. I think, col­lec­tively, the rea­son we be­lieve this is due to a num­ber of ran­dom­ized con­trol tri­als. Then some pretty smart, rea­son­able an­a­lysts at GiveWell have looked care­fully at this, dived into the coun­ter­fac­tu­als, pro­duced their anal­y­sis and come to the con­clu­sion: “On net, it is about $3,500″.

Short­en­ing the chain

But that is not why I be­lieved it. I be­lieved it be­cause peo­ple have told me that GiveWell have done this anal­y­sis and they say it is $3,500 on their web­site. In writ­ing this ar­ti­cle, I went and read it my­self on the web­site. Although it is more work for me, it is of value for the com­mu­nity be­cause I am short­en­ing the chain of Chi­nese whispers. As mes­sages get passed along, it is pos­si­ble that mis­takes en­ter and then it gets re­peated. By go­ing back and check­ing ear­lier sources in the chain, we can try to re­duce that and make our­selves more ro­bustly con­fi­dent in these state­ments.

Disagree­ment is an op­por­tu­nity to learn

When you no­tice that you dis­agree with some­body, you can see that per­haps parts of their jig­saw puz­zle are wrong. You might dis­miss what they have to say, but I want to ar­gue that this is of­ten not the most pro­duc­tive thing to do. Although, part of what they have to say is wrong, they may have some other as­pect of their think­ing pro­cess that would fill a gap in your per­spec­tive and help you get a bet­ter pic­ture of what is go­ing on.

I of­ten do this when I find that some­one has a per­spec­tive that I think is un­likely to be cor­rect. I am in­ter­ested in this pro­cess of how they get there and how they think about it, in part, be­cause peo­ple and they way they think is fas­ci­nat­ing. But also be­cause it is po­lite and use­ful. It helps me to build a deeper pic­ture of all the differ­ent bits of ev­i­dence we col­lec­tively have.

Ret­ro­spec­tive—What I be­lieve and why

In this sec­tion, I am go­ing to ap­ply the ideas I have just men­tioned. Through­out this ar­ti­cle, I’ve dis­cussed a range of differ­ent things with­out touch­ing on my level of com­pe­tence in these ar­eas, or why I be­lieve them. So I am go­ing to do that here.

Heavy-tailed dis­tri­bu­tions: I think it is pretty ro­bust that the baseline dis­tri­bu­tion of op­por­tu­ni­ties in the world does fol­low a dis­tri­bu­tion with a heavy-tailed prop­erty. See­ing it in many differ­ent do­mains and un­der­stand­ing some of the the­ory be­hind why it should arise makes it ex­tremely likely. How­ever, there’s an open em­piri­cal ques­tion to ex­actly how far that tail goes out as the prop­erty is not bi­nary but rather a con­tinuum.

Di­gres­sion: Altru­is­tic mar­ket efficiency

There is an im­por­tant caveat here: there is a mechanism which might push against this. If peo­ple are good at seek­ing out and iden­ti­fy­ing the best op­por­tu­ni­ties and they are uniformly seek­ing out and tak­ing them, then the best things that are left might not be so much bet­ter.

This oc­curs in reg­u­lar mar­kets; ways to make money start out dis­tributed across a wide range. In Figure 36 we use a log scale on the x-axis to rep­re­sent a heavy-tailed dis­tri­bu­tion. Those that are los­ing money stop do­ing that, and ob­serve oth­ers do­ing ac­tivi­ties which are mak­ing lots of money and pur­sue those in­stead. More peo­ple move into this area and by diminish­ing re­turns, you ac­tu­ally make less money than you used to in that area. Even­tu­ally, you end up with a nar­rower dis­tri­bu­tion of the value that is be­ing pro­duced by peo­ple do­ing these differ­ent things than we started with.

We might get a push in that di­rec­tion among op­por­tu­ni­ties to cre­ate al­tru­is­tic value. Cer­tainly, I don’t think we are in a prop­erly effi­cient mar­ket, but I am not sure how effi­cient it is. I hope that as this com­mu­nity grows and more peo­ple are ac­tively try­ing to choose very valuable things, that will mean the dis­tri­bu­tion does get less heavy-tailed.

One of the mechanisms that leads to effi­ciency in reg­u­lar mar­kets is feed­back loops; peo­ple no­tice if they are get­ting rich or los­ing money. Another mechanism is peo­ple do­ing anal­y­sis as a re­sult of feed­back loops. They try to work out whether ad­di­tional re­sources in a par­tic­u­lar op­por­tu­nity will make us richer or not. I think that do­ing this sort of anal­y­sis is an im­por­tant part of the pro­ject we are col­lec­tively em­bark­ing on.

Over­all, I don’t think we do have an effi­cient mar­ket for this, I do be­lieve we have heavy-tailed dis­tri­bu­tions; and I am not sure how ex­treme, but that is be­cause it re­sponds to the ac­tions peo­ple are tak­ing.

Fac­tor­ing cost-effectiveness

Fac­tor­ing cost-effec­tive­ness: I think this is a sim­ple point with­out room for er­ror. How­ever, there is an em­piri­cal ques­tion as to how much these differ­ent di­men­sions mat­ter. We may have more vari­a­tion in one of the di­men­sions than oth­ers. I don’t have a clear view on the mat­ter. We saw that the in­ter­ven­tion effec­tive­ness within global health varies by 3-4 or­ders of mag­ni­tude. I think area effec­tive­ness could be more than that, mn and as for or­gani­sa­tional effec­tive­ness I’m not an ex­pert and can­not claim much of a view on that.

Diminish­ing returns

Diminish­ing re­turns: I think this is a ro­bust point. In some do­mains, there are in­creas­ing re­turns to scale, where you get effi­cien­cies of scale and that helps you. This more of­ten ap­plies to the or­ga­ni­za­tion scale or or­ga­ni­za­tion within a do­main, whereas diminish­ing re­turns of­ten ap­ply at the do­main scale. De­spite this, I would add a note of cau­tion as I do know some smart peo­ple who think I am over­stat­ing the case for diminish­ing re­turns.

Scale, tractabil­ity, uncrowdedness

Scale, tractabil­ity, ne­glect­ed­ness: I think it is ob­vi­ous that they all mat­ter and that this fac­tori­sa­tion is cor­rect as a fac­tori­sa­tion. What is less clear is whether this breaks it up into things that are eas­ier to mea­sure and whether this is a helpful way of do­ing it. I would ar­gue that is is, since is loosely matches up with an in­for­mal frame­work that peo­ple have been us­ing and seem to find helpful.

Ab­solute and marginal priority

Ab­solute and marginal pri­or­ity: Again, this point is some­what triv­ial, but I made this point be­cause I think not ev­ery­body has these sep­a­rate no­tions and we can con­fuse each other if we blur them.

Differ­en­tial progress

Differ­en­tial progress: This ar­gu­ment ap­pears in a few aca­demic pa­pers and is be­lieved by some of the smartest and most rea­son­able peo­ple I know, which gives me some ev­i­dence that it might be true, out­side of my per­sonal in­tro­spec­tion. It is a bit coun­ter­in­tu­itive and has not had much scrutiny, so we may want to ex­pose it to more.

Com­par­a­tive advantage

Com­par­a­tive ad­van­tage is a stan­dard idea from eco­nomics. Nor­mally mar­kets try to work to push peo­ple into work­ing in the way that uti­lizes their com­par­a­tive ad­van­tage. We don’t nec­es­sar­ily have that when we are aiming for al­tru­is­tic value.

The ap­pli­ca­tion across time is more spec­u­la­tive. I am one of the main peo­ple who have been try­ing to rea­son this way and as of yet I haven’t had any­body push back on it. But it should be taken with a bit more of a pinch of salt as it has faced less scrutiny.

Ag­gre­gat­ing knowledge

Ag­gre­gat­ing knowl­edge: I think ev­ery­one tends to think we want in­sti­tu­tions for this. There is also a broad con­sen­sus that ex­ist­ing in­sti­tu­tions are not perfect, but whether we can build bet­ter in­sti­tu­tions, I am less cer­tain about.

Shar­ing rea­sons for beliefs

Shar­ing rea­sons for be­liefs: This is some­thing that I think is com­mon sense and all else be­ing equal, this is a good thing. There are costs to do­ing it; it slows down our com­mu­ni­ca­tion and it may not sound glamorous and be harder to get peo­ple on board with. I think we want to nudge peo­ple in the di­rec­tion of shar­ing rea­sons for be­liefs, but I am not sure how far, and do not want to by over­whelm­ingly de­mand­ing on this. To some ex­tent, I be­lieve this be­cause some smart, rea­son­able peo­ple I know think we should head in this di­rec­tion and I weigh in the opinion of other when I don’t see a rea­son that I have a no­tably bet­ter per­spec­tive on it than them.


Fi­nally, why have I been shar­ing this with you?

Peo­ple can mine gold with­out un­der­stand­ing all these the­o­ret­i­cal ar­gu­ments about the dis­tri­bu­tion of gold in the world.

But, be­cause what we ac­tu­ally value is in­visi­ble, we need to be more care­ful about aiming at the right things.

I think it is im­por­tant for our com­mu­nity to have this knowl­edge broadly spread; and as we are still in the early days of the com­mu­nity, it is par­tic­u­larly im­por­tant to try and get this knowl­edge into the foun­da­tions and work out bet­ter ver­sions.

We don’t want to have the kind of gold rush phe­nomenon where peo­ple charge of af­ter some­thing and it turns out there wasn’t ac­tu­ally that much value there.