Good policy ideas that won’t happen (yet)

Key points:

  • Over the last year the Cen­tre for Effec­tive Altru­ism has had over a dozen meet­ings with UK poli­cy­mak­ers and tested the wa­ters on a range of poli­cies we thought might have sig­nifi­cant pos­i­tive benefits for the world.

  • In most cases we quickly found that they could not cur­rently at­tract poli­ti­cal sup­port, for var­i­ous rea­sons.

  • Here we dis­cuss two more we in­ves­ti­gated in greater de­tail. Th­ese were both aimed at im­prov­ing the depth of knowl­edge of poli­cy­mak­ers about risks from new and up­com­ing tech­nolo­gies.

  • Th­ese turned out to be pre­ma­ture, as they were poli­cies to deal with is­sues about which there is no aca­demic con­sen­sus on the na­ture of the prob­lem, let alone the ap­pro­pri­ate re­sponse. More ground­work needs to be done be­fore there will be ma­jor­ity sup­port for sig­nifi­cant policy changes in these ar­eas.

  • Nonethe­less we had suc­cesses in rais­ing the pro­file of un­prece­dented tech­nolog­i­cal risks in Govern­ment via a re­port we wrote which was widely cir­cu­lated through sev­eral de­part­ments and read by se­nior civil ser­vants, ad­vi­sors, and poli­ti­ci­ans. We were also in­vited to con­tribute a chap­ter on ex­is­ten­tial risk to the Govern­ment Chief Scien­tist’s An­nual Re­port.

Cross-posted from the Global Pri­ori­ties Project


Dur­ing the past year the Cen­tre for Effec­tive Altru­ism has been en­gaged in on­go­ing policy dis­cus­sions with poli­cy­mak­ers in the UK Govern­ment. We gen­er­ated hun­dreds of policy op­tions and had many meet­ings with poli­cy­mak­ers within the UK Govern­ment, both party poli­ti­cal ad­vi­sors, poli­ti­ci­ans and civil ser­vants. This post de­scribes some of our learn­ings from this pro­cess. Th­ese learn­ings will prob­a­bly be ob­vi­ous to any­one with ex­pe­rience of the policy-mak­ing pro­cess, but I hope that dis­cussing them in more de­tail will be use­ful for peo­ple less fa­mil­iar with policy mak­ing who at­tempt to do policy work go­ing for­wards. This work was pri­mar­ily car­ried out by Toby Ord, Nick Beck­stead, William MacAskill, Owen Cot­ton-Bar­ratt, Robert Wiblin, Haydn Belfield, and my­self.

One of our mo­ti­va­tions for en­gag­ing in this work is that de­vel­oped coun­tries’ gov­ern­ments typ­i­cally con­trol bud­gets of the or­der of a trillion $US ($10^12) per year, and so mov­ing a small por­tion of this bud­get could re­sult in large amounts of money moved to effec­tive causes. Paul Chris­ti­ano dis­cusses the po­ten­tial value of en­gag­ing in poli­tics in more de­tail here.

Over the past year we have de­vised many out­lines for pos­si­ble poli­cies which aimed to, among other things, re­duce poverty, re­duce an­i­mal suffer­ing, im­prove de­ci­sion mak­ing, im­prove sci­en­tific progress, or re­duce ex­tinc­tion risk. The poli­cy­mak­ers thought that many of these poli­cies were promis­ing or in­ter­est­ing, but weren’t ex­cited enough about any spe­cific ini­ti­a­tives to re­ally push for them. The over­whelming ma­jor­ity of the pro­pos­als that we pre­sented were set aside by poli­cy­mak­ers with­out fur­ther in­ves­ti­ga­tion. Most of those ini­ti­a­tives that were in­ves­ti­gated fur­ther had prob­lems as­so­ci­ated with them, did not provide the benefits that we were ini­tially hop­ing for, or sim­ply weren’t promis­ing enough for the poli­cy­mak­ers to push for them pub­li­cly. While we will not be mak­ing the full list pub­lic at this time, there are a num­ber of themes that emerged from the poli­cies that were not taken up.

Poli­ti­cal feasibility

The most com­mon rea­son for poli­cies to be re­jected was be­cause they were poli­ti­cally in­fea­si­ble. We re­al­ised that the ini­ti­a­tives that were most likely to progress were those that would in­cur lit­tle to no fi­nan­cial or poli­ti­cal cost to the poli­ti­ci­ans. It may be pos­si­ble to over­come the prefer­ence for zero or low fi­nan­cial and poli­ti­cal cost by build­ing sup­port­ing coal­i­tions, but I will save the dis­cus­sion for later is this post.

Fi­nan­cial cost

Many of the poli­cies and ini­ti­a­tives we sug­gested were re­jected be­cause they would cost Govern­ment money. Poli­cies ruled out for this rea­son in­cluded:

Others were re­jected be­cause they would cost some vot­ers money, even if they would save more money for oth­ers. Poli­cies ruled out for this rea­son in­cluded:

  • Tax­ing all non-free-range meat products

  • Stop­ping sub­sidies of cer­tain agri­cul­tural products

Poli­ti­cal cost

Many poli­cies were ruled out be­cause of the poli­ti­cal, fi­nan­cial and time op­por­tu­nity cost. One way of con­ceiv­ing of this ob­jec­tion is that the poli­cy­mak­ers were un­will­ing to spend poli­ti­cal cap­i­tal be­cause they had more promis­ing ways in which to spend their time and money. This could be be­cause the policy was weird, un­pop­u­lar, or could be blocked by spe­cial in­ter­est groups. Poli­cies that were ruled out for these rea­sons in­clude:

  • Creat­ing more dis­aster shelters to pro­tect against global catas­trophic risks (too weird)

  • In­creas­ing im­mi­gra­tion via var­i­ous mechanisms (too un­pop­u­lar)

  • Set­ting min­i­mum sizes for ges­ta­tion crates for pigs (could be blocked by spe­cial in­ter­ests)

Box 1: Over-sim­plis­tic policy anal­y­sis (or ‘why some sci­en­tists are bad at policy’)

Dur­ing my years as a cli­mate sci­en­tist and cam­paigner, I had many op­por­tu­ni­ties to see sci­en­tists at­tempt­ing to en­gage in policy poorly. I would of­ten see sci­en­tists:

  1. Iden­tify a prob­lem, e.g. cli­mate change due to in­creas­ing EU road trans­port CO2 emissions

  2. Iden­tify a pos­si­ble re­sponse, e.g. re­duce EU road trans­port emissions

  3. Iden­tify a pos­si­ble policy, e.g. re­quire some frac­tion of EU petrol to come from EU biofuels

  4. Pro­mote this policy

One of the prob­lems with this ap­proach is that the sci­en­tists have not en­gaged in policy anal­y­sis, for ex­am­ple by:

  • Iden­ti­fy­ing all of the other pos­si­ble policy re­sponses,

  • Defin­ing our aims, not just in the spe­cific area the policy is ad­dress­ing, but also in terms of limit­ing other po­ten­tially nega­tive con­se­quences of the policy,

  • Es­ti­mat­ing how well these pos­si­ble poli­cies achieve our aims,

  • And as­sess­ing the trade-offs be­tween the poli­cies to iden­tify the best policy.

Another prob­lem is that they have not analysed the poli­ti­cal land­scape or ini­ti­ated stake­holder en­gage­ment, for ex­am­ple by:

  • Speak­ing with differ­ent stake­hold­ers to de­ter­mine how much sup­port and op­po­si­tion there is for these differ­ent policies

  • Con­sid­ered tweak­ing the policy in a num­ber of ways to bring po­ten­tial al­lies on board and neu­tral­ise po­ten­tial opponents

  • Used tools such as power ma­tri­ces to es­ti­mate whether it would be pos­si­ble to build a coal­i­tion in­fluen­tial enough to push any of these poli­cies through to im­ple­men­ta­tion.

By mov­ing straight from step three to four above we ig­nore key as­pects of the policy-mak­ing pro­cess and end up pro­mot­ing poli­cies that are usu­ally sub-op­ti­mal from an effi­ciency per­spec­tive, and are also poli­ti­cally in­fea­si­ble.

One way in which sci­en­tists can en­gage with the poli­ti­cal pro­cess suc­cess­fully is by propos­ing their policy and ‘pass­ing the ba­ton’ on to policy an­a­lysts who can then com­pare it to the other poli­cies on the table. But the sci­en­tist should not be sur­prised if their policy turns out to be sub-op­ti­mal or poli­ti­cally in­fea­si­ble. In­deed this is prob­a­bly the most likely out­come for a policy that has been pro­posed by some­one who is not an ex­pert in this policy area. How­ever if the policy an­a­lyst is in favour of the idea, a pow­er­ful part­ner­ship can de­velop in which the sci­en­tist acts as a sci­en­tific ad­vi­sor to an on­go­ing policy de­vel­op­ment pro­cess. One ex­am­ple of this oc­cur­ring is the role that Dr. Drew Shin­dell played in the de­vel­op­ment of the Cli­mate and Clean Air Coal­i­tion, which was largely de­vel­oped by the US State Depart­ment with Dr. Shin­dell act­ing as a sci­en­tific ad­vi­sor to the pro­cess.

Note that King­don’s ‘mul­ti­ple streams’ model of policy change sug­gests that ‘policy win­dows’ caused by the shift­ing poli­ti­cal land­scape could al­low pre­vi­ously-re­jected poli­cies to be im­ple­mented. Un­der this model a sci­en­tist who is un­able to tell when a policy win­dow has opened could have suc­cess in con­tin­u­ously pro­mot­ing policy over a long pe­riod of time and hop­ing that at some point a policy win­dow may open. An ex­am­ple of where this may have oc­curred is in the story that is told about the cre­ation of an ob­ser­va­tional pro­gram to mon­i­tor ocean cur­rents in the North At­lantic. Th­ese mea­sure­ments had been de­sired by the sci­en­tific com­mu­nity for some time, but ac­cord­ing to this story it was not un­til a sci­en­tist men­tioned the prob­lem of a po­ten­tial slow-down in the ocean cur­rents and the need to mon­i­tor them to then UK Prime Minister Tony Blair that fund­ing be­came available for the pro­gramme.

A po­ten­tially more effec­tive op­tion for the sci­en­tists push­ing for their policy con­tin­u­ously would be for them to ally with a policy an­a­lyst with more ex­per­tise in analysing the poli­ti­cal land­scape, who could mon­i­tor the land­scape and re­port back to the sci­en­tist if a policy win­dow opens or has a chance of open­ing soon. This may have been the case with the Cli­mate and Clean Air Coal­i­tion, which some com­men­ta­tors said was Obama’s at­tempt to claim a suc­cess in tack­ling cli­mate change in the run up to the 2012 gen­eral elec­tion in or­der to woo back en­vi­ron­men­tal vot­ers who had been unim­pressed by his record on cli­mate change to date. This policy win­dow may have caused policy an­a­lysts at the State Depart­ment to im­ple­ment Dr. Shin­dell, and oth­ers’ long-stand­ing policy recom­men­da­tions.

There is plenty more work that needs do­ing in iden­ti­fy­ing pos­si­ble re­sponses to prob­lems that effec­tive al­tru­ism com­mu­nity has iden­ti­fied (i.e. in step two above), be­fore the com­mu­nity can fo­cus the ma­jor­ity of its efforts on propos­ing poli­cies (see step three). How­ever, if the effec­tive al­tru­ism com­mu­nity would like to do some of this policy anal­y­sis, poli­ti­cal land­scape anal­y­sis and stake­holder en­gage­ment it­self rather than rely­ing on busy and po­ten­tially dis­in­ter­ested policy an­a­lysts and cam­paign strate­gists, we will need to de­velop ex­per­tise in these ar­eas.

More ini­ti­a­tives that won’t work (yet)

Once the ma­jor­ity of our pro­pos­als had been ruled out due to not be­ing clearly poli­ti­cally fea­si­ble, we had a much smaller num­ber of op­tions to in­ves­ti­gate fur­ther. Of these, sev­eral failed be­cause they were too ad­vanced rel­a­tive to our cur­rent un­der­stand­ing, and re­quired a ma­jor­ity to sup­port them. I dis­cuss two ex­am­ples in more de­tail be­low.

An In­ter­gov­ern­men­tal Panel on Biotech­nolog­i­cal Risk

The proposal

Set up a body, mod­el­led on to the In­ter­gov­ern­men­tal Panel on Cli­mate Change (IPCC), to syn­the­sise knowl­edge on biotech­nolog­i­cal risks.

The appeal

The IPCC had a huge im­pact on the cli­mate change de­bate, and even won a No­bel peace prize “for their efforts to build up and dis­sem­i­nate greater knowl­edge about man-made cli­mate change, and to lay the foun­da­tions for the mea­sures that are needed to coun­ter­act such change”. Pre­sum­ably an In­ter­gov­ern­men­tal Panel on Biotech­nolog­i­cal Risk could do the same for biotech­nolog­i­cal risk?

The stum­bling blocks

The IPCC was set up by the sci­en­tific com­mu­nity to illus­trate to the world that there was a sci­en­tific con­sen­sus on cli­mate change. The IPCC does not do origi­nal re­search, and does not progress the field other than by record­ing the state of cli­mate sci­ence re­search ev­ery six or seven years. The con­sen­sus in the field on what to do about the risks aris­ing from biotech­nol­ogy is not as uniform as the state of cli­mate sci­ence was when the IPCC was set up. There are still a wide range of views on ques­tions such as whether cer­tain types of po­ten­tially dan­ger­ous re­search should be un­der­taken, and whether cre­at­ing novel dan­ger­ous pathogens in the lab is over­all a net harm.

My un­der­stand­ing from Prof. Steve St­ed­man, former As­sis­tant Sec­re­tary Gen­eral to the United Na­tions, is that he spoke with over fifty ex­perts about the ques­tion of set­ting up an IPCC-equiv­a­lent body in this area and con­cluded that there were nei­ther high-level ex­perts will­ing to cham­pion the ini­ti­a­tive, nor was there a gen­eral feel­ing that the ini­ti­a­tive would suc­ceed, in part due to the lack of con­sen­sus.

IPCC au­thors are not paid for their time, and so they ei­ther vol­un­teer their time, or use fund­ing from other grants to cover their costs. This as­pect also made the ini­ti­a­tive un­ap­peal­ing to po­ten­tial cham­pi­ons of the ini­ti­a­tive.

Another prob­lem is that the in­cen­tives are al­igned rather differ­ently. When cli­mate sci­en­tists raised the alarm about cli­mate change, it did not cur­tail their re­search free­dom, and in­creased the fu­ture fund­ing to cli­mate sci­ence. If biotech­nol­o­gists raise the alarm about biotech risk, their pro­pos­als will largely be im­pos­ing re­stric­tions or large costs on their own and their peers’ re­search. Ad­di­tion­ally, the over­all effect may well be to re­duce the to­tal amount of fund­ing to their area, though this is un­cer­tain.

Prof. St­ed­man con­ducted the most in depth in­ves­ti­ga­tion to date of the pos­si­bil­ity of set­ting up an IPCC for biotech­nol­ogy risk. He was con­vinced enough that it was un­likely to go ahead that af­ter his in­ves­ti­ga­tion he re­turned the re­main­der of the fund­ing that he had been given to set up the ini­ti­a­tive and moved on to other things.

House of Com­mons/​Lords Science and Tech­nol­ogy Com­mit­tee in­quiry into un­prece­dented tech­nolog­i­cal risk

The proposal

The House of Com­mons/​Lords Science and Tech­nol­ogy Com­mit­tee would run an in­quiry into un­prece­dented tech­nolog­i­cal risk, par­tic­u­larly fo­cus­ing on its im­pact on ex­is­ten­tial risk.

The appeal

The Govern­ment must re­spond in writ­ing to the com­mit­tee’s re­ports, and this could al­low the com­mit­tee to in­fluence gov­ern­ment policy on this is­sue. We also hoped that we could use this as a way to cause peo­ple to in­ves­ti­gate this is­sue and de­velop pos­si­ble re­sponses.

The stum­bling blocks

House of Com­mons/​Lords com­mit­tees such as this one can­not do origi­nal re­search, and largely syn­the­sise the views of the wit­nesses that they call on. As there has been lit­tle re­search done into policy re­sponses to ex­is­ten­tial risk, and there is no con­sen­sus in the area, it is un­likely that the com­mit­tee would make strong recom­men­da­tions to Govern­ment.

Ad­di­tion­ally, in or­der to launch the in­quiry we would need mul­ti­ple sym­pa­thetic MPs or Lords on the com­mit­tee. The in­quiry would need to be led by a par­tic­u­larly sym­pa­thetic mem­ber, and it would take up a large amount of their time. This is time that could oth­er­wise be put to po­ten­tially more use­ful ini­ti­a­tives.

This may be a use­ful ini­ti­a­tive to take up in fu­ture when there has been more re­search done into re­sponses to un­prece­dented tech­nolog­i­cal risks and their im­pact on ex­is­ten­tial risk, par­tic­u­larly once those re­sponses have been de­vel­oped into pri­ori­tised policy pro­pos­als.

Com­mu­ni­cat­ing ex­pert opinion

Both of these failed pro­pos­als are about com­mu­ni­cat­ing the ma­jor­ity view of ex­perts to de­ci­sion-mak­ers and in­ter­ested par­ties. The rea­son they failed is be­cause the field of ex­is­ten­tial risk re­duc­tion is so new that the ma­jor­ity of ex­perts do not yet hold similar views on the is­sue. For ex­am­ple, even within the Fu­ture of Hu­man­ity In­sti­tute there is sig­nifi­cant in­ter­nal dis­agree­ment on key el­e­ments of what we should be do­ing to re­duce ex­is­ten­tial risk. This sug­gests there is much more re­search to be done in this area be­fore we can start im­ple­ment­ing many of our pos­si­ble re­sponses.

The ra­tio­nal model of policy analysis

There are many mod­els of the policy-mak­ing pro­cess. Schattschnei­der’s ‘ex­pan­sion of con­flict’ model sug­gests that the policy-mak­ing pro­cess is largely de­ter­mined by the ex­tent to which par­ties are able to ex­pand the pro­cess to in­clude ad­di­tional groups that are sup­port­ive of their point of view. One of the more pop­u­lar mod­els is King­don’s “mul­ti­ple streams” model, which sug­gests that poli­cies are only im­ple­mented when ‘prob­lems, poli­cies and poli­tics’ al­ign to cre­ate a win­dow of op­por­tu­nity. King­don also in­tro­duces the idea of a ‘policy en­trepreneur’ as some­one who seeks to gain benefits in ex­change for im­ple­ment­ing poli­cies and ini­ti­a­tives that are pop­u­lar. Baum­gart­ner and Jones fo­cus on is­sue-defi­ni­tion and venue-shop­ping to sug­gest that is­sues usu­ally fol­low a sta­ble policy di­rec­tion but that this sta­bil­ity is in­ter­jected with pe­ri­ods of rapid, un­pre­dictable change. Sch­nei­der, In­gram, Stone and oth­ers fo­cus on is­sue fram­ing as the fac­tor which de­cides which poli­cies are im­ple­mented and which are not. Th­ese are just a small sub­set of the more pop­u­lar mod­els of the poli­cy­mak­ing pro­cess which have been pro­posed. Th­ese mod­els cover differ­ent as­pects of the de­ci­sion-mak­ing pro­cess, and do not always agree where they over­lap, as we might ex­pect in a real-world pro­cess as com­plex as this one.

Si­mon’s ra­tio­nal model of policy anal­y­sis is an­other model that we can use to frame our dis­cus­sion of the poli­cy­mak­ing pro­cess. It is a sim­ple model that I will dis­cuss in more de­tail is I be­lieve it can be used to illus­trate the broader pic­ture of why many of our poli­cies were not taken up. It was origi­nally put for­ward by Her­bert A. Si­mon to de­scribe a pro­cess one could take to de­velop pub­lic policy. Below is an adap­ta­tion of Ian Thomas’s de­scrip­tion of Si­mon’s model of policy de­vel­op­ment:

  1. In­tel­li­gence gath­er­ing— data and po­ten­tial prob­lems and op­por­tu­ni­ties are iden­ti­fied, col­lected and an­a­lyzed.

  2. Iden­ti­fy­ing a problem

  3. Iden­ti­fy­ing op­tions to solve this problem

  4. Assess­ing the con­se­quences of all options

  5. Re­lat­ing con­se­quences to val­ues— with all de­ci­sions and poli­cies there will be a set of val­ues which will be more rele­vant (for ex­am­ple, eco­nomic fea­si­bil­ity and en­vi­ron­men­tal pro­tec­tion) and which can be ex­pressed as a set of crite­ria, against which perfor­mance (or con­se­quences) of each op­tion can be judged.

  6. Choos­ing the preferred op­tion— given the full un­der­stand­ing of all the prob­lems and op­por­tu­ni­ties, all the con­se­quences and the crite­ria for judg­ing op­tions.

Stages in the model can oc­cur con­cur­rently and stages can be skipped.

Crit­i­cism of the ra­tio­nal model

Like any sim­ple model, the ‘ra­tio­nal model’ ig­nores some im­por­tant fac­tors. For ex­am­ple, it ig­nores the poli­ti­cal land­scape and vested in­ter­ests, and as­sumes the Govern­ment is a uni­tary ra­tio­nal ac­tor in a static sys­tem. There is a liter­a­ture of case stud­ies both show­ing benefits and draw­backs of the ra­tio­nal model in differ­ent cir­cum­stances. Nonethe­less it can be helpful to us in pro­vid­ing a hy­poth­e­sis as to why our poli­cies were un­work­able, and where we might fo­cus our at­ten­tion in the fu­ture.

An adapted ra­tio­nal model

The ra­tio­nal model has been adapted in a num­ber of ways through­out the liter­a­ture. Here I sug­gest an ex­ten­sion that may help ex­plain some of our ex­pe­rience. As with all mod­els based on the ra­tio­nal model, it will have a num­ber of short­falls, but is hope­fully still use­ful. It pro­poses that policy de­vel­op­ment largely goes through the fol­low­ing stages:

Like in the origi­nal ra­tio­nal model, stages can oc­cur con­cur­rently and stages can be skipped. I make a dis­tinc­tion be­tween re­sponses (de­sired out­comes, e.g. re­duce emis­sions), and poli­cies (meth­ods of achiev­ing the de­sired out­comes, e.g. a cap and trade regime) as I did in the box above.

We can rep­re­sent our progress to­wards a policy-based solu­tion for a given prob­lem in this model. For ex­am­ple, on ex­is­ten­tial risk due to ar­tifi­cial in­tel­li­gence, the com­mu­nity is per­haps work­ing on gath­er­ing in­tel­li­gence and iden­ti­fy­ing the prob­lem (stages 1 and 2A), with limited work on iden­ti­fy­ing re­sponses and build­ing coal­i­tions around the prob­lem (stages 3A and 2B). Thus it would not be as use­ful for the com­mu­nity to shift its at­ten­tion to work­ing on stages 4A, 3B, 4B or 5 at this point. There may be space for in­di­vi­d­u­als to do pi­o­neer­ing work in these ar­eas, but in gen­eral the model would sug­gest that they will find progress eas­ier once more progress has been made in the pre­vi­ous stages.

On cli­mate change miti­ga­tion (re­duc­ing green­house gas emis­sions) in the UK, most of the work that re­mains to be done is on de­vel­op­ing poli­cies, build­ing coal­i­tions around var­i­ous re­sponses and poli­cies and at­tempt­ing get those poli­cies im­ple­mented (stages 4A, 3B, 4B and 5). This is one of the rea­sons I moved away from be­ing a cli­mate sci­en­tist, where I was pre­dom­i­nantly work­ing on stage one. This is not to say that there is no more work to be done in this area; fur­ther high­light­ing the dan­gers from cli­mate change is still use­ful, just per­haps not as use­ful as well-di­rected work in the policy arena. In the US, it is plau­si­ble that the coal­i­tions around the prob­lem (stage 2B) are not yet strong enough, and so there is po­ten­tially more work to be done there.

We can view the in­ter­gov­ern­men­tal panel on biotech­nolog­i­cal risk as an at­tempt to build a coal­i­tion around re­sponses to biotech risk once we had a coal­i­tion in place around the prob­lem of biotech risk (mov­ing from stage 2B to stage 3B). The prob­lem in this case was that there was not a pre-ex­ist­ing coal­i­tion de­vel­oped around the prob­lem large enough to rep­re­sent the ma­jor­ity view.

We in­ves­ti­gated us­ing the House of Com­mons/​Lords Select Com­mit­tee to do re­sponse iden­ti­fi­ca­tion, policy iden­ti­fi­ca­tion, and ad­vo­cacy (stages 3A on­wards), how­ever the com­mit­tees are un­able to carry out re­sponse and policy iden­ti­fi­ca­tion (stages 3A and 4A) as these are not within the Com­mit­tees’ man­dates. In­deed I can­not think of any ex­am­ples of Govern­ment com­mit­tees that do this form of re­sponse and policy re­search, ex­cept per­haps in ex­treme cir­cum­stances such as dur­ing times of crisis. We should per­haps view the Com­mons/​Lords Com­mit­tees as part of mov­ing from a coal­i­tion around a spe­cific policy to get­ting that policy im­ple­mented (mov­ing from stage 4B to stage 5). We are not yet ready for this stage as poli­cies and coal­i­tions around re­sponses (stages 4A and 3B) have not been ad­e­quately de­vel­oped, let alone coal­i­tions around poli­cies (stage 4B).


There are many ways that this model may mis­lead us, par­tic­u­larly if it is fol­lowed in too for­mu­laic a fash­ion. I will list just some of these crit­i­cisms be­low.

For ex­am­ple, by build­ing visi­ble coal­i­tions we can also ig­nite op­po­si­tion, and so need to be wary of the poli­ti­cal land­scape in or­der to im­prove the chances of our ac­tivi­ties have an over­all pos­i­tive effect.

This model could also cause us to miss easy wins, in which the size of the coal­i­tion re­quired to cause the policy to be im­ple­mented is sur­pris­ingly small, if we were fo­cus­ing at­ten­tion on ear­lier stages in the model.

This model doesn’t prop­erly in­cor­po­rate the pos­si­bil­ity of larger-scale poli­ti­cal change (such as a change of party af­ter an elec­tion), nor does it ac­count for ex­ter­nal events (such as a war or var­i­ous types of dis­aster) that can dra­mat­i­cally al­ter coal­i­tions and the thresh­olds re­quired for policy im­ple­men­ta­tion.

Ad­di­tion­ally, by sim­ply mod­el­ling ‘coal­i­tions’ the model ig­nores the differ­ent roles played by differ­ent ac­tors such as the me­dia, the pub­lic, in­ter­est groups, poli­ti­cal elites, and oth­ers in the pro­cess. A more fine grained model of the poli­ti­cal land­scape and stake­holder en­gage­ment will be use­ful in the later stages of the model.

So this is a sim­plis­tic model, yet it al­lows us to make hy­pothe­ses about where fur­ther work may be most use­ful to help move to­wards policy solu­tions to prob­lems. Th­ese hy­pothe­ses can then be probed fur­ther us­ing other meth­ods.

Out­comes and next steps

Over the course of our policy en­gage­ment to date we have learned much, and also had some sig­nifi­cant suc­cesses. Our re­port on un­prece­dented tech­nolog­i­cal risks was widely dis­tributed and read by poli­ti­ci­ans and civil ser­vants across a num­ber of de­part­ments. We were also in­vited to con­tribute a chap­ter on ex­is­ten­tial risk to the UK Govern­ment Chief Scien­tist’s An­nual Re­port on risk.

Go­ing for­wards, we have de­cided to fo­cus more of our effort on build­ing coal­i­tions around the prob­lem and iden­ti­fy­ing solu­tions (stages 2B and 3A). One of the rea­sons for this is that we found it eas­ier than ex­pected to gain ac­cess to poli­cy­mak­ers, and thus when we have more poli­ti­cally fea­si­ble re­sponses to prob­lems and more sup­port for them, we think it will be pos­si­ble for us to take these to poli­cy­mak­ers. Nonethe­less, we will con­tinue to re­spond to policy op­por­tu­ni­ties in a more re­ac­tive fash­ion, and will con­tinue to meet with se­nior poli­cy­mak­ers and poli­ti­ci­ans as op­por­tu­ni­ties arise.

This work was car­ried out by the Global Pri­ori­ties Pro­ject, which is cur­rently fundrais­ing to hire an ad­di­tional re­searcher for the pro­ject and a full-time pro­ject man­ager who will also lead our out­reach to Govern­ments and foun­da­tions. If you would be in­ter­ested in con­tribut­ing to this effort please con­tact me or Robert Wiblin. We have more in­for­ma­tion on our cur­rent plans which you can read here.


Some of the key points I feel I have learned from our in­ter­ac­tions with poli­cy­mak­ers are:

  • We need more re­search into po­ten­tial re­sponses to un­prece­dented tech­nolog­i­cal risks.

  • Once we have a bet­ter idea of the re­sponses to un­prece­dented tech­nolog­i­cal risks we would like to see im­ple­mented, we can start re­search­ing policy im­ple­men­ta­tions of those re­sponses.

  • If we want to im­ple­ment poli­cies that in­cur fi­nan­cial cost or poli­ti­cal cap­i­tal, then we will need to start build­ing poli­ti­cal coal­i­tions around the prob­lems, re­sponses and poli­cies that we are cham­pi­oning.

  • Coal­i­tion-build­ing is some­thing that we do not have much ex­per­tise in among the peo­ple I spoke to within the effec­tive al­tru­ism com­mu­nity.

  • Ini­ti­a­tives that build upon a ma­jor­ity view among ex­perts are not use­ful un­less there is already a ma­jor­ity view among ex­perts.

  • It was eas­ier than ex­pected to dis­cuss our ideas with se­nior civil ser­vants and poli­ti­ci­ans.

Ad­di­tional points not men­tioned above, but which I felt I learned through this pro­cess:

  • It would have been use­ful if we had been even more fa­mil­iar with what the Govern­ment was already do­ing on un­prece­dented tech­nolog­i­cal risks.

  • Poli­cy­mak­ers were more will­ing to ac­cept that there was a prob­lem in the area of un­prece­dented tech­nolog­i­cal risk than I was ex­pect­ing. This shifted the con­ver­sa­tion over to spe­cific policy re­sponses, which is where I felt we had the least to say.

  • Poli­cy­mak­ers’ be­liev­ing that there was a prob­lem in the area of un­prece­dented tech­nolog­i­cal risks was not suffi­cient to cause them to put en­ergy into de­vel­op­ing re­sponses. It was ex­pected that we were the peo­ple who would do this work.

Thanks to Nick Beck­stead, Owen Cot­ton-Bar­ratt, Steve St­ed­man, Toby Ord, Haydn Belfield, Robert Wiblin, William MacAskill and David Frame for the work, com­ments and con­ver­sa­tions that went into this post.

Niel Bow­er­man is a co-founder and Direc­tor of Spe­cial Pro­jects at the Cen­tre for Effec­tive Altru­ism. He was a mem­ber of Obama’s 2008 pres­i­den­tial elec­tion En­ergy and En­vi­ron­ment Policy Team, and was Cli­mate Science Ad­vi­sor to the Pres­i­dent of the Mal­dives. He has a PhD (DPhil) in physics from the Univer­sity of Oxford.