Deliberation May Improve Decision-Making


In this es­say from Re­think Pri­ori­ties, we dis­cuss the op­por­tu­ni­ties that de­liber­a­tive re­forms offer for im­prov­ing in­sti­tu­tional de­ci­sion-mak­ing. We be­gin by de­scribing de­liber­a­tion and its links to demo­cratic the­ory, and then sketch out ex­am­ples of de­liber­a­tive de­signs. Fol­low­ing this, we ex­plore the ev­i­dence that de­liber­a­tion can en­gen­der fact-based rea­son­ing, opinion change, and un­der cer­tain con­di­tions can mo­ti­vate longterm think­ing. So far, most de­liber­a­tive ini­ti­a­tives have not been in­vested with a di­rect role in the de­ci­sion-mak­ing pro­cess and so the ma­jor­ity of policy effects we see are in­di­rect. Pro­vid­ing de­liber­a­tive bod­ies with a bind­ing and di­rect role in de­ci­sion-mak­ing could im­prove this state of af­fairs. We end by high­light­ing some limi­ta­tions and ar­eas of un­cer­tainty be­fore not­ing who is already work­ing in this area and av­enues for fur­ther re­search.


Many in the effec­tive al­tru­ism com­mu­nity are con­cerned about im­prov­ing in­sti­tu­tional de­ci­sion-mak­ing,[1] and even more are con­cerned with the lives of fu­ture gen­er­a­tions and non-hu­man an­i­mals who are cur­rently not rep­re­sented in our de­ci­sion-mak­ing sys­tems.[2] One ap­proach to im­prov­ing de­ci­sions is to set up in­sti­tu­tional struc­tures that are con­ducive to good de­ci­sion-mak­ing ( Thaler & Sun­stein, 2008; Whit­tle­stone, 2017a ). A range of re­cent pro­pos­als have been made to re­form poli­ti­cal in­sti­tu­tions to­wards bet­ter de­ci­sion-mak­ing pro­cesses e.g. ap­proval vot­ing, age-weighted vot­ing, fu­ture gen­er­a­tion ori­en­tated offices and com­mit­tees, poli­cies ad­dress­ing Trans­for­ma­tive AI and many more. Re­forms to in­crease de­liber­a­tion in the de­ci­sion-mak­ing sys­tem is an­other po­ten­tial op­tion to con­sider. De­liber­a­tion, the pro­cess of weigh­ing the mer­its of com­pet­ing ar­gu­ments with at least the aim to ar­rive at a con­sen­sus, can lead to more an­a­lyt­i­cal rea­son­ing, and given rele­vant prompts leads to fu­ture-ori­ented think­ing. Such im­prove­ments to de­ci­sion-mak­ing may be es­pe­cially use­ful in com­plex high-stakes de­ci­sions where it’s difficult to even know what the best de­ci­sion is, let alone ac­tu­ally mak­ing that de­ci­sion (Whit­tle­stone, 2017b).

Demo­cratic deliberation

De­liber­a­tion is a pro­cess dis­tinct from open-ended dis­cus­sion or zero-sum ar­gu­men­ta­tion be­cause it in­volves a col­lec­tive ex­change of rea­sons among equals as if they aim to reach a shared prac­ti­cal judg­ment. This is quite un­like ne­go­ti­a­tion or ad­ver­sar­ial de­bate where one side is try­ing to “win”. In con­trast to com­pro­mise, a gen­uine con­sen­sus is not sim­ply a ne­go­ti­ated agree­ment of the par­ti­ci­pants, but in­cludes a trans­for­ma­tion of prefer­ences. Even if agree­ment is not reached par­ti­ci­pants are more likely to leave a de­liber­a­tion as more so­phis­ti­cated, tol­er­ant, and par­ti­ci­pa­tive cit­i­zens who are bet­ter able to pro­cess more in­for­ma­tion more ob­jec­tively ( Gastil, Deess, & Weiser, 2002; Gastil & Dillard, 1999; Tet­lock, 1983, 1985 ). This is due to the en­vi­ron­ment in which par­ti­ci­pants are ex­posed to in­for­ma­tion. As all ar­gu­ments are given a fair hear­ing, par­ti­ci­pants are more likely to care­fully con­sider com­pet­ing ar­gu­ments and gain ex­pe­rience in bet­ter de­ci­sion-mak­ing habits such as un­der­stand­ing ex­pert opinions. It can ex­pand the in­for­ma­tion ba­sis of de­ci­sion-mak­ing and en­hance the level of re­flec­tion among the par­ti­ci­pants.

Many of the de­liber­a­tive re­forms dis­cussed be­low have been pro­posed by demo­cratic the­o­rists. Demo­cratic the­ory took a “de­liber­a­tive turn” in the 1990s, shift­ing fo­cus away from ter­ri­to­rial rep­re­sen­ta­tion and max­i­miz­ing cit­i­zens’ ex­pres­sion of poli­ti­cal prefer­ences and to­wards ran­dom­ized rep­re­sen­ta­tion, akin to jury se­lec­tion, and to max­i­miz­ing cit­i­zens’ ca­pac­ity to form prefer­ences and judg­ments on pub­lic af­fairs. The out­come of this shift is a de­sire for dis­cur­sive or de­liber­a­tive democ­racy. De­liber­a­tive democ­racy is “a sys­tem of self-gov­ern­ment that con­cerns it­self as much with the qual­ity of its in­ter­nal de­liber­a­tion as it does with the dis­tri­bu­tion of for­mal power” (Gastil & Richards, 2013, p. 255). Here we are less con­cerned with the over­all struc­ture of democ­racy but rather the act of demo­cratic de­liber­a­tion ,which ties to­gether the egal­i­tar­ian, open char­ac­ter of the fo­rum and rigor­ous an­a­lytic rea­son­ing. Such de­liber­a­tion can oc­cur both in demo­cratic and non-demo­cratic regimes, be­ing no­tably pop­u­lar in China (He & War­ren, 2011; Ma & Hsu, 2018).

From the per­spec­tive of de­liber­a­tive democrats, de­liber­a­tion may offer a way to fill the gap be­tween elites who ap­pear un­re­spon­sive to the pub­lic’s con­cerns and pop­ulists who use emo­tive non-de­liber­a­tive ap­peals. Ideally, it fills this gap with a rea­soned weigh­ing of ar­gu­ments that rep­re­sents the in­ter­ests of the wider pub­lic. Of course, al­though there are demo­cratic rea­sons to be con­cerned about ap­pro­pri­ate de­liber­a­tion, the broader idea that many EAs may be con­cerned with is that it will im­prove de­ci­sion-mak­ing.

Is de­liber­a­tion pos­si­ble?

A key com­po­nent of de­liber­a­tion is the weigh­ing of com­pet­ing ar­gu­ments and de­cid­ing on the mer­its of the ar­gu­ment with good in­for­ma­tion. Is there ev­i­dence that in­di­vi­d­u­als can ac­tu­ally de­liber­ate in an effec­tive way i.e. weigh trade-offs, as­sess com­pet­ing ar­gu­ments, and con­nect de­liber­a­tions with de­ci­sions. Is such a thing pos­si­ble in an era of fake news, so­cial me­dia echo cham­bers, and ide­olog­i­cally sorted com­mu­ni­ties? It may not be im­por­tant for many EAs that de­liber­a­tion meet ex­actly the stan­dards of demo­cratic the­o­rists, so long as bet­ter de­ci­sions re­sult from the pro­cess. How­ever, it is an ad­van­tage of de­liber­a­tive meth­ods over some other pro­pos­als that in­creas­ing de­liber­a­tion is ar­guably demo­crat­i­cally le­gi­t­i­mate and ex­plic­itly di­rected at im­prov­ing rea­son­ing.

Em­piri­cal re­search shows that both poli­ti­ci­ans and av­er­age cit­i­zens have the ca­pac­ity to de­liber­ate when in­sti­tu­tions are ap­pro­pri­ate. The Dis­course Qual­ity In­dex is a con­tent an­a­lyt­i­cal mea­sure for cap­tur­ing the qual­ity of de­liber­a­tive pro­cesses. The qual­ity of de­liber­a­tion is eval­u­ated by (a) the ex­tent to which de­liber­a­tion fulfills a num­ber of vi­tal char­ac­ter­is­tics as­cribed by de­liber­a­tive the­ory and (b) whether de­liber­a­tive be­hav­ior is equally dis­tributed among the par­ti­ci­pants. De­spite rea­son­able con­cerns that poli­ti­ci­ans may be too par­ti­san and short-sighted to de­liber­ate openly, the Dis­course Qual­ity In­dex offers ev­i­dence that poli­ti­cal dis­course can meet the ideals of de­liber­a­tion as en­visaged (Steen­ber­gen, Bächtiger, Spörndli, & Steiner, 2003). Steiner et al. (2004), us­ing the Dis­course Qual­ity In­dex to analyse 4,488 speeches from Ger­man, Swiss, and U.S. par­li­a­men­tary de­bates, find Swiss grand coal­i­tions en­hance re­spect­ful be­havi­our of MPs much more than the US Con­gres­sional rules and Ger­man Par­li­a­men­tary pro­ce­dures. De­liber­a­tive qual­ity is high­est in set­tings of coal­i­tions, sec­ond cham­bers of par­li­a­ment (for ex­am­ple, the US Se­nate or UK House of Lords), se­crecy, low party dis­ci­pline, low is­sue po­lariza­tion, and the strong pres­ence of mod­er­ate par­ties (Fishkin & Mans­bridge, 2017, p. 10). How­ever, com­bi­na­tions of these con­di­tions (e.g. a less po­larized is­sue de­bate in a sec­ond cham­ber of a con­sen­sus sys­tem be­hind closed doors) may rep­re­sent rel­a­tively rare con­di­tions and ones that are harder for EAs to gen­er­ate.

Or­di­nary cit­i­zens are also ca­pa­ble of this level of dis­course in de­liber­a­tive en­vi­ron­ments. For ex­am­ple, Europo­lis, a Euro­pean-wide De­liber­a­tive Poll, pro­vides ev­i­dence that cit­i­zens are able to rea­son in ways com­pa­rable to those of the par­li­a­men­tar­i­ans. The poll brings to­gether a sam­ple of the vot­ers of the en­tire Euro­pean Union, rep­re­sent­ing all 27 coun­tries for a three-day de­liber­a­tion be­fore the Euro­pean Par­li­a­ment elec­tions. Par­ti­ci­pants dis­cuss the is­sues, read bal­anced briefing ma­te­ri­als, ques­tion com­pet­ing ex­perts and poli­ti­ci­ans, and reg­ister their opinions and vot­ing in­ten­tions in con­fi­den­tial ques­tion­naires. The num­ber of par­ti­ci­pants who both pro­vided a so­phis­ti­cated jus­tifi­ca­tion and si­mul­ta­neously en­gaged in re­spect­ful listen­ing is 28 per­cent of those sam­pled (Ger­ber, Bächtiger, Shikano, Re­ber, & Rohr, 2018). Whether or not this is high is up for de­bate. This pub­lic de­liber­a­tion has been found be­yond Western coun­tries with high liter­acy and high ed­u­ca­tion lev­els e.g. Uganda (Fishkin et al., 2017) and China (Fishkin, He, Luskin, & Siu, 2010). The abil­ity to en­gage in high qual­ity ar­gu­ments does not vary across gen­der (Siu, 2008), how­ever in prac­tice women tend to be less ac­tive than men in small-group de­liber­a­tions (Setälä, Grön­lund, & Herne, 2010). Par­ti­ci­pants with lower so­cio-eco­nomic sta­tus tend to have lower ca­pac­i­ties to con­tribute to high-qual­ity de­liber­a­tion (Ger­ber et al., 2018) while oth­ers claim that peo­ple with in­tense prefer­ences and abun­dant re­sources can hi­jack the pro­cess (Shapiro, 2017). It is im­por­tant to weigh up these risks and costs when con­sid­er­ing such re­forms and com­par­ing them to al­ter­na­tives.

How do we de­sign de­liber­a­tive sys­tems?

A “de­liber­a­tive de­sign” is any num­ber of pro­cesses that pro­vides a frame­work wherein a par­ti­ci­pant defends a view by pro­vid­ing rea­sons; oth­ers probe the use­ful­ness of this view through crit­i­cism; by re­flect­ing to­gether on the ev­i­dence for and against var­i­ous views, free and in­de­pen­dent par­ti­ci­pants come to ac­cept what Haber­mas calls “the force of the bet­ter ar­gu­ment” (Haber­mas, 1984, 1996). Com­mon el­e­ments of the de­liber­a­tive de­signs dis­cussed here are the ran­dom se­lec­tion of par­ti­ci­pants, hear­ing of ex­perts, ac­cess to writ­ten ma­te­ri­als, and de­liber­a­tion lead­ing to “ver­dicts” on (policy) is­sues.

Re­searchers have found three con­di­tions that tend to mo­ti­vate in­di­vi­d­u­als to adopt a de­liber­a­tive frame of mind: ac­countabil­ity (Tet­lock, 1983, 1985), high stakes ( Taber, Lodge, & Glathar, 2001 ), and di­ver­sity of per­spec­tives (Mutz, 2002a, 2002b; Mutz & Martin, 2001; Nemeth, 1986). There­fore, de­liber­a­tive de­signs should en­sure that par­ti­ci­pants know that they will have to dis­cuss their judg­ments pub­li­cly, they per­ceive that there will be con­se­quences, and that a range of per­spec­tives are rep­re­sented. Beyond this there are a plethora of ways in which de­liber­a­tion can be de­signed de­pend­ing on one’s ob­jec­tives and goals.

Ex­ist­ing poli­ti­cal in­sti­tu­tions and de­signs may al­low for de­liber­a­tion, but there are a num­ber of rea­sons to doubt that they can max­imise it. Ne­go­ti­a­tion rather than de­liber­a­tion ap­pears to be the norm among poli­ti­cal elites, and so changes in opinion are more likely to be the re­sult of changes in cir­cum­stances (which may be harder to in­fluence) rather than the effect of weigh­ing the mer­its of com­pet­ing ar­gu­ments (Fishkin & Luskin, 2005). To the ex­tent that one be­lieves poli­tics is in­her­ently an ad­ver­sar­ial com­pe­ti­tion, it is more difficult to imag­ine de­liber­a­tive co­op­er­a­tion as a tractable pro­posal. Elected offi­cials are also likely to have short-term bi­ases be­cause of the pres­sures of elec­toral cy­cles and so may not be the best agents to ad­vo­cate for bet­ter longterm out­comes. As noted above, the struc­tural con­di­tions that ap­pear to fa­cil­i­tate greater de­liber­a­tion among poli­ti­ci­ans seem hard for EAs to in­fluence.

In the­ory, poli­cies in­fluenced by de­liber­a­tive fo­rums will have greater le­gi­t­i­macy if they are de­rived from cit­i­zens and/​or are sup­ported by cit­i­zen groups. There­fore, an­other av­enue of de­liber­a­tive re­form is to use ran­dom se­lec­tion to cre­ate “mini-pub­lics” ( Fung, 2003) drawn from the pop­u­la­tion. The ra­tio­nale for mini-pub­lics is that if the ran­dom sam­ple of cit­i­zens that is gath­ered to de­liber­ate is rep­re­sen­ta­tive of the pop­u­la­tion, and if it de­liber­ates un­der good con­di­tions, then its con­sid­ered judg­ments af­ter de­liber­a­tion should rep­re­sent what the larger pop­u­la­tion would think if some­how those cit­i­zens could en­gage in similarly good con­di­tions for con­sid­er­ing the is­sue. This is a some­what ideal­ized no­tion since we can eas­ily imag­ine that the re­sult of ac­tu­ally gath­er­ing a suffi­ciently large pop­u­la­tion to de­liber­ate would be chaotic [3]. For our pur­poses it may only mat­ter that mini-pub­lics are le­gi­t­imized be­cause of this be­lief rather than due to the re­al­ity of such a claim.

Sor­ti­tion, the ran­dom se­lec­tion of par­ti­ci­pants, is the preferred method for de­liber­a­tive fo­rums. Firstly, the ex­is­tence of a new elected body could un­der­mine the le­gi­t­i­macy of other ex­ist­ing elected bod­ies. Se­condly, elect­ing in­di­vi­d­u­als to par­ti­ci­pate runs the risk of rep­re­sen­ta­tives not be­ing suffi­ciently fu­ture-fo­cused due to short-term elec­toral in­cen­tives. It is pos­si­ble that un­like poli­ti­ci­ans and in­ter­est groups, ran­domly se­lected cit­i­zens are un­likely to have the in­ter­est or the ca­pac­ity to en­trench them­selves in their pub­lic role of de­liber­a­tors, and in any case short term limits as de­liber­a­tors can limit the de­vel­op­ment of ei­ther. Thirdly, sor­ti­tion also avoids the is­sue that self-se­lected par­ti­ci­pants will come from a ho­mo­ge­neous group (e.g. mostly white, col­lege-ed­u­cated, and mid­dle-class in North Amer­ica and Europe) and this ho­mo­gene­ity may im­pede a de­liber­a­tive frame of mind. Some schol­ars are skep­ti­cal that we can achieve ideal ran­domi­sa­tion (Ryfe, 2005), and in­deed, it is rare for such sor­ti­tion to meet stan­dards of statis­ti­cal rep­re­sen­ta­tive­ness be­cause of how de­liber­a­tive de­signs are im­ple­mented in prac­tice. Low re­sponse rates, in­suffi­cient or ab­sent in­cen­tives for par­ti­ci­pa­tion, the im­po­si­tion of de­mo­graphic quo­tas for poli­ti­cal pur­poses, and weak or im­proper use of strat­ified ran­dom sam­pling can lead to mini-pub­lics that do not truly re­flect a wide range of opinions in so­ciety nor their pro­por­tion. There is always some el­e­ment of self-se­lec­tion de­spite de­sign­ers’ best in­ten­tions.

Ex­am­ples of de­liber­a­tion among ran­domly cho­sen cit­i­zens can be found in clas­si­cal Athens, Medieval Ital­ian re­pub­lics and among mod­ern iter­a­tions be­gun in the 1970s with the cre­ation of cit­i­zens’ ju­ries and plan­ning cells ( Car­son & Martin, 1999; Manin, 1997). Depend­ing on the defi­ni­tion one uses there can be hun­dreds of de­liber­a­tive de­signs.[4] The num­ber of par­ti­ci­pants can range from 25 to 400 and can be con­ducted in just a few days to over the course of mul­ti­ple weeks. We will high­light here just some illus­tra­tive ex­am­ples of the most in­ter­est­ing and most re­searched types of de­liber­a­tive fo­rums.

(A De­liber­a­tive Poll of over 200 Cal­ifor­nia pub­lic high school se­niors. Credit: Cen­ter for De­liber­a­tive Democ­racy )

  • De­liber­a­tive pol­ling brings to­gether a ran­dom sam­ple of cit­i­zens, ex­plic­itly as­sert­ing statis­ti­cal rep­re­sen­ta­tive­ness (Mans­bridge, 2010), for a week­end of small group dis­cus­sions with trained ex­perts and gath­ers data, and sur­vey them as in an opinion poll both upon re­cruit­ment and then again at the end of the de­liber­a­tions. Fi­nally, the re­sults of the polls are re­leased to me­dia out­lets. Reg­u­larly run de­liber­a­tive polls now ex­ist such as Europo­lis in the EU and Stan­ford’s De­liber­a­tive Pol­ling across the world. This has been pro­posed as a way to un­der­stand pub­lic opinion on AGI.

  • De­sign pan­els eval­u­ate and po­ten­tially re­vise pro­posed bal­lot mea­sures be­fore be­ing cir­cu­lated for sig­na­tures. Th­ese may be of par­tic­u­lar in­ter­est to EAs con­sid­er­ing bal­lot ini­ti­a­tives as a tool. It offers a quasi seal of ap­proval that can provide in­ex­pen­sive cred­i­bil­ity to a well-crafted, but un­der­funded, bal­lot mea­sure and re­duce the legally re­quired sig­na­ture thresh­old for one (Gastil & Richards, 2013, p. 268).

  • Ci­ti­zens’ Assem­blies for­ward a recom­men­da­tion to the leg­is­la­ture whereby its pro­posal goes di­rectly to the bal­lot un­less the leg­is­la­ture in­tro­duces a mea­sure to block it. This is ideal for ad­dress­ing is­sues on which the leg­is­la­ture has a con­flict of in­ter­est such as term limits and cam­paign re­form. For ex­am­ple, the 2004 Bri­tish Columbia Ci­ti­zens’ Assem­bly, and pro­pos­als for Ci­ti­zens’ Assem­blies on Brexit to ad­dress the gen­er­a­tional fal­lout of such de­ci­sions. Scot­land has or­ganised a cit­i­zens’ as­sem­bly on how to re­spond to the UK’s exit from the EU. One could also ag­gre­gate the con­clu­sions of many small cit­i­zens’ as­sem­blies or mini-pub­lics to form a col­lec­tive mass opinion similar to what Amer­i­caS­peaks does.

  • Ci­ti­zens’ Ini­ti­a­tive Re­views de­velop a one-page an­a­lytic state­ment (typ­i­cally about a bal­lot mea­sure) that ap­pears in the Vot­ers’ Guide. For ex­am­ple, Ore­gon Ci­ti­zens’ Ini­ti­a­tive Re­view.

  • Policy Juries. In this pro­posal (Leib, 2004, p. 12), a strat­ified ran­dom sam­ple of fifty cit­i­zens de­liber­ates for two weeks on a spe­cific piece of leg­is­la­tion. The judg­ment is fi­nal and sub­ject only to ju­di­cial re­view. This is prob­a­bly best suited to ad­dress­ing is­sues on which the un­con­sid­ered opinion of the mass ma­jor­ity opinion is sus­pect, such as the civil rights of minori­ties.

  • Con­sen­sus Con­fer­ences. Den­mark de­signed these in the 1980s to solve prob­lems on is­sues char­ac­ter­ized by tech­ni­cal and moral com­plex­ity. They are now be­ing con­vened in nu­mer­ous coun­tries. This de­sign may be es­pe­cially use­ful for is­sues sur­round­ing biotech­nol­ogy/​se­cu­rity.

  • Pro­pos­als for a non-par­ti­san leg­is­la­tive house, cho­sen by sor­ti­tion, given the power to scru­ti­nise and if nec­es­sary to veto leg­is­la­tive pro­pos­als e.g. A Coun­cil of Guardians of Fu­ture Gen­er­a­tions (Read, 2012) or a “tribunate as­sem­bly” (McCormick, 2006, p. 160)

  • Plan­ning Cells. A num­ber of cells of 25 ran­domly-se­lected cit­i­zens meet over a few days and are pro­vided with in­for­ma­tion on a policy is­sue. They de­liber­ate and pri­ori­tise po­ten­tial courses of ac­tion. Moder­a­tors pre­pare a fi­nal re­port for the com­mis­sion­ing body. Plan­ning cells have been used over 170 times in more than 40 lo­ca­tions, be­ing com­mon in Ger­many for decades.

Lafont (2017) offers a frame­work of when to use differ­ent mini-pub­lics de­pend­ing on how differ­ent the opinions of the mini-pub­lic and mass-pub­lic are likely to be. If the opinion of the mini-pub­lic will likely differ from the ma­jor­ity opinion of the pop­u­la­tion on the poli­ti­cal is­sue at hand (e.g. the his­tor­i­cal marginal­i­sa­tion of one group), then their con­clu­sions can act as a pow­er­ful tool for minori­ties to challenge the sta­tus quo. For ex­am­ple, mini-pub­lics could be rou­tinely con­vened to provide the Supreme Court (or rele­vant ju­di­cial body) with ad­di­tional, sup­pos­edly un­bi­ased, in­for­ma­tion on what the con­sid­ered ma­jor­ity opinion of the coun­try may be at a given time.

If the mini-pub­lic’s recom­men­da­tions are likely to co­in­cide with the ma­jor­ity opinion but differ from ex­ist­ing policy, it can sig­nal to the wider pub­lic the need to scru­ti­nize the poli­ti­cal sys­tem. Reg­u­lar pol­ling could rank im­por­tant poli­ti­cal is­sues that need to be tack­led, and then mini-pub­lics could be con­vened to make recom­men­da­tions con­cern­ing the top-ranked is­sues. This would be par­tic­u­larly helpful con­cern­ing poli­ti­cal is­sues that elected offi­cials may see as in­tractable or not worth con­fronting. We might think that Ci­ti­zens’ Assem­blies here could tackle elec­toral re­forms, while Con­sen­sus Con­fer­ences could tackle highly tech­ni­cal is­sues where ex­it­ing gov­ern­ment policy is lag­ging be­hind such as biose­cu­rity or AI.

If the pub­lic has no opinion at all about the poli­ti­cal is­sues in ques­tion (e.g new gene edit­ing tech­nolo­gies such as CRISPR or in­ter­na­tional trade agree­ments) then the func­tion of mini-pub­lics would not be to di­rectly shape the poli­cies in ques­tion, but in­stead to en­hance the visi­bil­ity of what is at stake so as to en­able pub­lic de­bate among cit­i­zens. De­sign Panels and Ci­ti­zens’ Ini­ti­a­tive Re­views could perform this role for bal­lot ini­ti­a­tives tack­ling is­sues EAs are in­ter­ested in but are not yet sub­ject to much pub­lic dis­cus­sion or par­ti­san­ship.

How likely are these re­forms to be sup­ported by the pub­lic and the poli­ti­cal elite?

In­ter­views with Bel­gian mini-pub­lic par­ti­ci­pants re­veal that they con­ceive mini-pub­lics as a com­ple­ment to elec­toral forms of par­ti­ci­pa­tion, like vot­ing or party mem­ber­ship (Jac­quet, 2019) and Bri­tish opinion poll data sug­gests “not only are peo­ple pre­pared to join ‘ju­ries’, but the pub­lic at large is will­ing to trust their de­ci­sion-mak­ing – even over that of elected rep­re­sen­ta­tives” (Lown­des, Pratch­ett, & Stoker, 2001, p. 448). Coun­tries with higher lev­els of de­liber­a­tion in gov­ern­ment are also more likely to have sup­port for the regime (Seitz & Votta, 2018). A novel two-study de­sign on aware­ness of mini-pub­lics aimed to en­sure repli­ca­bil­ity and is no­table for us­ing real in­for­ma­tion for its vi­gnettes (Bou­li­anne, 2018). Both stud­ies found that be­ing in­formed about mini-pub­lics pos­i­tively af­fected re­spon­dents’ per­cep­tions that they could in­fluence gov­ern­ment and the gov­ern­ment cares about their opinions. The first study also found that be­ing in­formed that policy op­tions were recom­mended by a ran­domly se­lected group of cit­i­zens who met to dis­cuss differ­ent policy op­tions (a mini-pub­lic) had an im­pact on some policy prefer­ences but this was not repli­cated in the sec­ond study. The ex­pla­na­tion given is that the sec­ond study used in­for­ma­tion about mini-pub­lics tack­ling differ­ent policy is­sues. The au­thor’s con­clu­sion was that cit­i­zens can dis­crim­i­nate be­tween a mini-pub­lic’s recom­men­da­tions de­pend­ing on the policy do­main and choose whether to ac­cept the recom­men­da­tion or not. How suc­cess­ful such policy recom­men­da­tions have been in prac­tice is dealt with in the Policy In­fluence sec­tion be­low. How­ever, there are challenges to the claim that de­liber­a­tion in­creases cit­i­zens’ faith and trust in gov­ern­ment and their be­lief that they can un­der­stand and in­fluence poli­ti­cal af­fairs. In fact, stud­ies sug­gest that peo­ple’s sense of in­ter­nal effi­cacy is de­creased rather than in­creased as an out­come of de­liber­a­tion (Mutz, 2006; Mor­rell, 2005). Both pos­i­tive and nega­tive effects have been ob­served on cit­i­zens’ will­ing­ness to par­ti­ci­pate in poli­tics in the fu­ture af­ter de­liber­at­ing (Grön­lund, Setälä, & Herne, 2010).

There are a num­ber of routes through which these re­forms have come to be im­ple­mented by poli­ti­ci­ans. The Ore­gon Ci­ti­zens’ Ini­ti­a­tive Re­view was the re­sult of a pi­lot test run by an NGO, Healthy Democ­racy, that caught the at­ten­tion of the Sec­re­tary of State who then recom­mended it to the leg­is­la­ture. The cit­i­zens as­sem­bly in Bri­tish Columbia came as the re­sult of an elec­toral promise, and Den­mark’s con­sen­sus con­fer­ences were es­tab­lished by the Dan­ish Board of Tech­nol­ogy, an in­de­pen­dent in­sti­tu­tion set up by the par­li­a­ment.

Im­pact of deliberation

We have es­tab­lished above what de­liber­a­tion is, how one could de­sign de­liber­a­tive bod­ies, and that ran­domly cho­sen cit­i­zens are likely ca­pa­ble of de­liber­a­tion. How­ever, does this de­liber­a­tion ac­tu­ally lead to bet­ter de­ci­sion-mak­ing? We are mainly con­cerned here with whether de­liber­a­tion can change opinions and prefer­ences, in­crease fac­tual knowl­edge, and in­crease con­sid­er­a­tion of oth­ers as these seem more di­rectly re­lated to in­sti­tu­tional de­ci­sion-mak­ing than other effects of de­liber­a­tion such as civic en­gage­ment or poli­ti­cal effi­cacy, though the lat­ter are im­por­tant to keep in mind.

There are some rea­sons to be scep­ti­cal of the liter­a­ture due to the file drawer prob­lem. [5] One of the main con­clu­sions of this re­view that we ad­vo­cate for is more re­search into de­liber­a­tive re­forms, which would strengthen the ev­i­dence base available. We did not find any sys­tem­atic re­views, though Kuyper (2018) claimed to be such. There are some other good overviews ( the Sum­mer 2017 is­sue of Dæ­dalus on the prospects and limits of de­liber­a­tive democ­racy; Setälä & Herne, 2014; Fishkin & Luskin, 2005; Men­delberg, 2002) but they do not always men­tion null or mixed find­ings ap­pear­ing el­se­where in the liter­a­ture, which is con­cern­ing and so spe­cial at­ten­tion should be paid to those who are scep­ti­cal or claim to have found nega­tive or null re­sults.

As one such ex­am­ple, Hib­bing & Theiss-Morse argue

real life de­liber­a­tion can fan emo­tions un­pro­duc­tively, can ex­ac­er­bate rather than diminish power differ­en­tials among those de­liber­at­ing, can make peo­ple feel frus­trated with the sys­tem that made them de­liber­ate, is ill-suited to many is­sues and can lead to worse de­ci­sions than would have oc­curred if no de­liber­a­tion had taken place (2002, p. 191).

Their meth­ods have been crit­i­cised how­ever, be­cause they ig­nore con­trary ev­i­dence in their own sur­vey data, and they use the con­clu­sions of fo­cus groups, who are in effect de­liber­at­ing, to show that cit­i­zens do not want to de­liber­ate (Dryzek, 2007). In a similar vein, Bren­nan (2016) re­lies on the 2002 re­view by Men­delberg to ar­gue that null find­ings of de­liber­a­tion are in fact nega­tive find­ings.[6] This, he ar­gues, is be­cause even poorly run de­liber­a­tion in­tro­duces par­ti­ci­pants to new in­for­ma­tion and their failure to up­date in light of this “means de­liber­a­tion made them worse, from an epistem­i­cally point of view”. Bren­nan is mainly con­cerned that de­liber­a­tion leads to po­lariza­tion, ex­clu­sion of cer­tain groups, and de­creased civic en­gage­ment. The lat­ter two of these effects do ap­pear to be sup­ported as men­tioned above, though the oc­cur­rence of po­lariza­tion has been dis­puted if not dis­proven by other work dis­cussed be­low. He does not cite ev­i­dence that de­liber­a­tion fails to lead to opinion change, sin­gle-peaked­ness, or knowl­edge gain, which have more of an em­pha­sis in this re­view. In fact, effects on opinion change, prefer­ence struc­tur­ing, knowl­edge gain and other-re­gard­ing ten­den­cies ap­pear across a range of de­liber­a­tive de­signs, is­sues, and con­texts lend­ing some ro­bust­ness to their claims.

Other works find ev­i­dence that de­liber­a­tion can lead par­ti­ci­pants to re­gret their de­ci­sions (Holt, 1999, 1993; Wil­son & Schooler 1991; Wil­son et al. 1989), to doubt that a “cor­rect” de­ci­sion is available at all (Ar­mor & Tay­lor, 2003; Iyen­gar & Lep­per, 2000), and to feel more anx­ious and frus­trated about the is­sue un­der dis­cus­sion af­ter a de­liber­a­tive en­counter than be­fore(Hen­driks, 2002; But­ton & Matt­son, 1999; Cook & Ja­cobs, 1998). It is un­clear how con­cerned we should be con­cerned about these effects and how to weigh them as costs against other claimed benefits of de­liber­a­tion ex­ist.

Bet­ter de­ci­sion-making

The liter­a­ture ap­pears to be weighted more to­wards where and whether de­liber­a­tion is oc­cur­ing at the in­di­vi­d­ual level, rather than what effect it has on out­comes. A start­ing point could be to con­sider that con­sen­sual democ­ra­cies, such as the Nordic coun­tries, the Nether­lands, Ger­many, and Switzer­land are ar­guably the world’s most suc­cess­ful states on a va­ri­ety of in­di­ca­tors, sug­gest­ing a strong cor­re­la­tion be­tween de­liber­a­tion and pub­lic policy suc­cess. One of the few stud­ies ad­dress­ing this ques­tion looked at four na­tional leg­is­la­tures (Switzer­land, United States, Ger­many, and the UK) and found more unan­i­mous de­ci­sions are typ­i­cally as­so­ci­ated with higher Dis­course Qual­ity In­dex scores, how­ever they did not find ev­i­dence that high dis­course qual­ity in­creases egal­i­tar­ian de­ci­sions, in the sense that the most dis­ad­van­taged in so­ciety are par­tic­u­larly helped (Bächtiger, Spörndli, Steen­ber­gen, & Steiner, 2005).

How­ever, the Dis­course Qual­ity In­dex is bet­ter suited to as­sess­ing and com­par­ing the qual­ity of de­liber­a­tion in ac­tual speeches or de­bates rather than across whole poli­ti­cal sys­tems. One pos­si­ble mea­sure on the struc­tural level for quan­ti­ta­tive cross-na­tional com­par­i­son is the De­liber­a­tive Com­po­nent In­dex of the Va­ri­eties of Democ­racy Pro­ject (Coppedege et al., 2017) but few stud­ies have em­ployed it in the ways we are in­ter­ested in, as far as we know. (A world map in­di­cat­ing the De­liber­a­tive Com­po­nent In­dex score (nor­mal­ized to range from 0 to 1) across the coun­tries. Data weighted to same sam­ple size (=1000). (Seitz & Votta, 2018, p. 17)

The De­liber­a­tive Com­po­nent In­dex is based on ex­pert-based as­sess­ments of the ex­tent to which poli­ti­cal elites give pub­lic jus­tifi­ca­tions for their po­si­tions on mat­ters of pub­lic policy, jus­tify their po­si­tions in terms of the pub­lic good, ac­knowl­edge and re­spect counter-ar­gu­ments, and how wide the range of con­sul­ta­tion is at elite level. To the ex­tent that one re­gards trade liber­al­i­sa­tion as a pos­i­tive out­come, coun­tries that score higher on the De­liber­a­tive Com­po­nent In­dex are nega­tively as­so­ci­ated with trade tar­iffs and pos­i­tively as­so­ci­ated with trade to GDP ra­tio (Singh, 2018). More stud­ies in this area would be a wel­come ad­di­tion to the liter­a­ture. In the mean­time, we can look to meso- and micro-level ev­i­dence of spe­cific de­liber­a­tive ini­ti­a­tives.

Some stud­ies of de­liber­a­tive polls have found lit­tle or no sup­port for the propo­si­tion that par­ti­ci­pa­tion in de­liber­a­tion has sig­nifi­cant effects on knowl­edge or opinion change. How­ever, there are of­ten is­sues with the data or de­liber­a­tion. Some ini­tial work turned up am­biva­lent find­ings be­cause opinion changes were mea­sured at the group level which masked in­di­vi­d­ual shifts (Barabas, 2004). While data in a study in the UK ( Den­ver, Hands, & Jones, 1995 ) are not suffi­ciently re­li­able to sup­port very de­tailed analy­ses of the hy­pothe­ses pro­posed largely be­cause of par­ti­ci­pants drop­ping out be­tween pre- and post-sur­veys. It is also pos­si­ble that the effects dis­cussed be­low may be mainly due to in­for­ma­tion ma­te­rial and in­ter­ac­tion with ex­perts, and un­for­tu­nately many of the stud­ies fail to dis­en­tan­gle the effects of the given in­for­ma­tion from that of de­liber­a­tion. While a de­liber­a­tive poll of a rep­re­sen­ta­tive sam­ple of EU cit­i­zens found that, com­pared with a con­trol group, de­liber­a­tors changed their views sig­nifi­cantly on im­mi­gra­tion (be­com­ing more liberal), cli­mate change (be­com­ing greener) and the EU it­self (be­com­ing more pro-Euro­pean), none of the ex­pla­na­tions they tested were satis­fac­tory: sam­pling bias, in­creased poli­ti­cal knowl­edge, dis­cus­sion qual­ity, small group so­cial con­for­mity pres­sure and the in­fluence of other De­liber­a­tive Poll ac­tors (San­ders, 2012). How­ever, re­sults from a ran­dom­ized field ex­per­i­ment sug­gest de­liber­a­tions them­selves drove most of the change, rather than other as­pects of the De­liber­a­tive Pol­ling treat­ment such as the in­vi­ta­tion, briefing ma­te­ri­als, or ca­sual con­ver­sa­tions with other at­ten­dees (Far­rar et al., 2010). For con­se­quen­tial­ists it may not mat­ter whether it is their de­liber­a­tion or their con­tact with ex­perts that is pro­duc­ing the re­sults, so long as the re­sults are pre­sent.

One of the most com­monly claimed benefits of de­liber­a­tion is that the pro­cess al­ters peo­ples’ prefer­ences and opinions af­ter be­ing pre­sented with bet­ter ar­gu­ments or rele­vant new in­for­ma­tion. Stud­ies based on de­liber­a­tive polls run in coun­tries around the world since the 1990s reg­u­larly re­port that peo­ple who en­gaged in de­liber­a­tion were more likely to have clear and or­dered policy prefer­ences/​sin­gle-peaked prefer­ences (Far­rar et al., 2010) and opinion changes be­tween pre- and post-treat­ment ques­tion­naires, at both ag­gre­gate and in­di­vi­d­ual lev­els (for a de­tailed overview see Setälä & Herne, 2014). More than half of policy at­ti­tude items Fishkin and oth­ers have posed in De­liber­a­tive Pol­ling have shown “statis­ti­cally sig­nifi­cant net change” (Fishkin & Luskin, 2005, p.290). They claim the effects of their re­search( Luskin, Iyen­gar, & Fishkin, 2004;Luskin, Fishkin, & Jow­ell, 2002; Luskin, Fishkin, Jow­ell, & Park, 1999a; Fishkin & Luskin, 1999;Luskin, Fishkin, & Plane, 1999b) are quite large, though it is hard to make a stan­dard­ised effect size com­par­i­son be­cause these stud­ies in­clude ei­ther changes in policy po­si­tions (av­er­age 19% ac­cord­ing to my own calcu­la­tions) or changes on 1-5 and 1-7 scales (av­er­age 0.3 ac­cord­ing to my own calcu­la­tions). Of course, in­di­vi­d­ual level changes could can­cel out if equal num­bers of peo­ple change opinions in each di­rec­tion. There­fore, one should aim to em­ploy de­liber­a­tive de­signs on is­sues that EAs ex­pect bet­ter ar­gu­ments and new in­for­ma­tion will shift more peo­ple in a pos­i­tive di­rec­tion. Far­rar et al. (2010) find that de­liber­a­tion’s effects are larger for less salient is­sues be­cause par­ti­ci­pants have less es­tab­lished views and there­fore more room to move.Sig­nifi­cant opinion changes on policy is­sues as a re­sult of de­liber­a­tion have been found to en­dure even six to nine months af­ter de­liber­a­tion ( Bou­li­anne, Lopt­son, & Ka­hane, 2018; French & Laver, 2009) and opinion change is most likely in de­liber­a­tors un­der 65 with me­dian knowl­edge lev­els (Suiter, Far­rell, & O’Malley, 2016). A sys­tem­atic re­view of effect sizes from other de­liber­a­tive de­signs would be a wel­come, though la­bo­ri­ous, ad­di­tion to the liter­a­ture.

Knowl­edge gain is also cited as a de­sir­able out­come of de­liber­a­tion as it sup­posed to be the back-bone of fact-based rea­son­ing . Fishkin and Luskin re­port statis­ti­cally sig­nifi­cant av­er­age gains in knowl­edge of around 10% are ex­tremely com­mon in De­liber­a­tive Pol­ling (Fishkin & Luskin, 2005, p.291) and my own calcu­la­tions find the same av­er­age in the stud­ies they cite ( Luskin, Iyen­gar, & Fishkin, 2004;Luskin, Fishkin, & Jow­ell, 2002; Luskin, Fishkin, Jow­ell, & Park, 1999a; Fishkin & Luskin, 1999;Luskin, Fishkin, & Plane, 1999b). It is the par­ti­ci­pants who emerge know­ing the most who dis­pro­por­tionately ac­count for the net change in the sam­ple as a whole ( Luskin, Iyen­gar, & Fishkin, 2004;Luskin, Fishkin, & Jow­ell, 2002)

There is ev­i­dence that de­liber­a­tion pro­duces con­sid­er­a­tion of those in the out­group. When de­liber­a­tion changes opinions it doesn’t ap­pear to re­flect en­trenched hi­er­ar­chies (Siu, 2008), and de­liber­at­ing in mixed groups (for ex­am­ple peo­ple both pro and anti cer­tain im­mi­gra­tion poli­cies) in­creases out­group em­pa­thy (Grön­lund, Herne, & Setälä, 2017). A de­liber­a­tive poll in Omagh, North­ern Ire­land, demon­strated that Catholics and Protes­tants were able to gain knowl­edge of op­pos­ing view­points, and ul­ti­mately sup­port greater in­ter­min­gling of ideas in policy out­puts (Luskin, O’Flynn, Fishkin, & Rus­sell, 2014) echo­ing find­ings el­se­where that agree­ment among in­di­vi­d­u­als from di­vided so­cieties oc­curs dur­ing de­liber­a­tion (Steiner, Jaramillo, Maia, & Mameli, 2017).

Men­delberg and Oleske (2000) found that dis­cus­sion did not pro­duce greater tol­er­ance for op­pos­ing views nor miti­gated con­flict, though this work suffered from a method­olog­i­cal bias, be­ing re­li­ant upon par­ti­ci­pants’ self-as­sess­ment for mea­sures of em­pa­thy.Cass Sun­stein (2011, 2002) pro­vides ex­per­i­men­tal ev­i­dence that groups tend to move to­ward the di­rec­tion of the po­si­tion ini­tially dom­i­nat­ing the group. For ex­am­ple, peo­ple who are op­posed to the min­i­mum wage are likely, af­ter talk­ing to each other, to be still more op­posed; peo­ple who tend to sup­port gun con­trol are likely, af­ter dis­cus­sion, to sup­port gun con­trol with con­sid­er­able en­thu­si­asm; peo­ple who be­lieve that global warm­ing is a se­ri­ous prob­lem are likely, af­ter dis­cus­sion, to in­sist on se­vere mea­sures to pre­vent global warm­ing. How­ever, these stud­ies are largely look­ing at group in­ter­ac­tions or dis­cus­sion, but do not fo­cus on how de­liber­a­tion im­pacts po­lariza­tion. Pro­po­nents of de­liber­a­tive pol­ling ar­gue that fears of po­lariza­tion are un­war­ranted be­cause there ap­pears to be “lit­tle” ten­dency for groups to be­come more po­larized as an out­come of group de­liber­a­tion in De­liber­a­tive Polls: on av­er­age 40% of small groups be­come more po­larized (Luskin, Fishkin, & Jow­ell, 2002 ;Luskin, Iyen­gar, & Fishkin, 2004). An ex­per­i­ment on cit­i­zen de­liber­a­tion on the fu­ture of the Swedish lan­guage in Fin­land found dis­cus­sion with a fa­cil­i­ta­tor and de­liber­a­tive norms re­versed ten­den­cies to group po­lariza­tion, whereas “free” dis­cus­sion with­out a fa­cil­i­ta­tor and ex­plicit de­liber­a­tive norms pro­duced the un­de­sired po­lariza­tion pat­terns de­scribed by Sun­stein(Grön­lund et al., 2010).

The longterm future

From a demo­cratic per­spec­tive, it may be ille­gi­t­i­mate to make de­ci­sions that do not ad­e­quately rep­re­sent the in­ter­ests of those af­fected, in­clud­ing fu­ture gen­er­a­tions (Goodin, 2007; Tännsjö, 2007). Even leav­ing aside demo­cratic ideals, oth­ers may re­gard it as un­just to make de­ci­sions that ig­nore or vi­o­late the in­ter­ests or rights of fu­ture gen­er­a­tions (Barry, 1978). De­liber­a­tion may help to mo­ti­vate longterm think­ing and so may ap­peal to tem­po­ral cos­mopoli­tans in EA. A num­ber of schol­ars have sug­gested that there may be a re­la­tion­ship be­tween de­liber­a­tion and longterm think­ing about fu­ture gen­er­a­tions (mainly con­cern­ing is­sues around the en­vi­ron­ment and cli­mate change, af­fect­ing 50 years in the fu­ture) (Bovenkerk, 2015; Dietz, C. Stern, & Dan, 2009; Ekeli, 2009; Gun­der­sen, 1995; Nie­meyer & Jennstål, 2016).

Michael Macken­zie has writ­ten a large piece on de­liber­a­tion and longter­mism (MacKen­zie, 2018). He ar­gues that the an­a­lyt­i­cal think­ing and pub­lic rea­son­ing re­quired by de­liber­a­tion can over­come our cog­ni­tive bi­ases, which are of­ten against con­sid­er­a­tion of the fu­ture. In this en­vi­ron­ment it is eas­ier, and even ad­van­ta­geous for ar­gu­ments to be made on be­half of fu­ture gen­er­a­tions be­cause ap­peals to chil­dren and fu­ture gen­er­a­tions are likely to get a fair hear­ing (even if they are made by those not try­ing to gen­uinely rep­re­sent the in­ter­ests of the fu­ture). If this were to be done at a large scale it may foster co­or­di­na­tion be­tween non-over­lap­ping gen­er­a­tions and pro­tect against time-in­con­sis­tency prob­lems. This view is based on the as­sump­tion that our dis­re­gard for the fu­ture is mainly the product of our cog­ni­tive bi­ases rather than just self­ish­ness and a re­luc­tance to ex­pand our moral cir­cle. More re­search in un­der­stand­ing the causal mechanism here would be in­ter­est­ing but is not nec­es­sary so long as bet­ter out­comes for fu­ture gen­er­a­tions are pro­duced as a re­sult of de­liber­a­tion.

Macken­zie sug­gests three con­di­tions are nec­es­sary for de­liber­a­tion to gen­er­ate longterm think­ing. First, col­lec­tive de­ci­sions on those is­sues must be based on gen­uine de­liber­a­tion (rather than ad­ver­sar­ial de­bate or ne­go­ti­a­tion one as­sumes). Se­cond, the in­ter­ests of the fu­ture will not be rele­vant in de­liber­a­tion un­less we rec­og­nize that our de­ci­sions will af­fect the fu­ture (achiev­ing what Nie­meyer & Jennstål (2016) el­se­where call a ‘de­liber­a­tive stance’ when a cog­ni­tive di­men­sion is pre­sent that re­flects upon the con­se­quences of de­ci­sions). Thirdly, there must be some dis­agree­ment about how differ­ent groups will be af­fected by the de­ci­sions taken on longterm is­sues. Others would add that the cur­rent gen­er­a­tion’s de­ci­sion to benefit the fu­ture gen­er­a­tions will be de­pen­dant on their level of al­tru­ism (Kamijo, Komiya, Mifune, & Saijo, 2017) Nie­meyer & Jennstål( 2016) in an ear­lier pro­posal offered a similar frame­work but also in­cluded that fu­ture gen­er­a­tions must be ‘emo­tion­ally pre­sent’ in that pro­cess, and not ab­stract or statis­ti­cal en­tities. We can­not as­sume that de­liber­a­tion will au­to­mat­i­cally in­clude fu­ture gen­er­a­tions if it is not em­bed­ded in the de­sign and agenda, but it may still benefit them to the ex­tent that it gen­er­ates less bi­ased judge­ment, and more sta­ble de­ci­sions and in­sti­tu­tions post-de­liber­a­tion. For our pur­poses we may not be con­cerned that those de­liber­at­ing ac­tu­ally en­gage in longterm think­ing so long as their de­ci­sions have pos­i­tive longterm out­comes.

Some fas­ci­nat­ing work that pro­vides data in sup­port of MacKen­zie’s hy­poth­e­sis that peo­ple can be guided into longterm think­ing by a prompt to con­sider fu­ture gen­er­a­tions has been done in Ja­pan and is worth dis­cussing in some de­tail. One study shows the pro-longter­mism effect of in­tro­duc­ing a ne­go­tia­tor on be­half of fu­ture gen­er­a­tions, and the other of hav­ing one group imag­ine them­selves as fu­ture gen­er­a­tions.

In the first (Kamijo et al., 2017), a lab­o­ra­tory-con­trol­led in­ter­gen­er­a­tional sus­tain­abil­ity dilemma game (ISDG) shows how the pres­ence of ne­go­tia­tors for a fu­ture gen­er­a­tion in­creases the benefits af­forded to fu­ture gen­er­a­tions. Par­ti­ci­pants were di­vided into two groups, which were then fur­ther di­vided into teams of 3 who rep­re­sented one of 5 con­cate­nate gen­er­a­tions. Each gen­er­a­tion was pre­sented with op­tion A and op­tion B on how to di­vide money. An es­sen­tial fea­ture of the ISDG is that the choice of the cur­rent gen­er­a­tion af­fects the size of the next gen­er­a­tion’s money. Op­tion A brings a larger benefit to the cur­rent gen­er­a­tion, but it is detri­men­tal to the benefit of the next gen­er­a­tion. Op­tion B brings less benefit to the cur­rent gen­er­a­tion, but pre­serves the size of the pie as it is. All gen­er­a­tions ob­tain the same amount when they con­tinue to choose Op­tion B, but if they con­tinue to choose Op­tion A their re­sources shrink grad­u­ally.

An im­por­tant el­e­ment here is the treat­ment con­di­tion. In one of the two groups, one of the mem­bers in each gen­er­a­tion mak­ing the choice acts as a ne­go­tia­tor on be­half of fu­ture gen­er­a­tions. When faced with mem­bers of an imag­i­nary fu­ture gen­er­a­tion, 60% of par­ti­ci­pants opted to leave re­sources for fu­ture gen­er­a­tions (choos­ing Op­tion B), even if that meant re­duc­ing the re­mu­ner­a­tion which the group it­self would take home. In con­trast, when the imag­i­nary fu­ture gen­er­a­tion was not salient, only 28% of par­ti­ci­pants chose the sus­tain­able op­tion. Ad­di­tion­ally, the imag­i­nary fu­ture gen­er­a­tions, as well as other mem­bers (i.e. not-imag­i­nary-fu­ture-gen­er­a­tion mem­bers) in the treat­ment con­di­tion, pro­duced more pos­i­tive state­ments on a sus­tain­able op­tion than par­ti­ci­pants in the con­trol con­di­tion. Th­ese games do not nec­es­sar­ily en­tail de­liber­a­tion since only 10 min­utes were al­lot­ted for dis­cus­sion but the idea was that peo­ple in the same gen­er­a­tion (should) dis­cuss and take a de­ci­sion as a group. The same ex­per­i­ment was con­ducted in Dhaka (an ur­ban com­mu­nity) and a ru­ral com­mu­nity in Bangladesh as well as in Kath­mandu (an ur­ban com­mu­nity) and a forested com­mu­nity in Nepal; in nonur­ban com­mu­ni­ties the re­sults repli­cated the pos­i­tive benefits to fu­ture gen­er­a­tions of a fu­ture ne­go­tia­tor, but no effect was de­tected in ur­ban com­mu­ni­ties (Shahrier, Kotani, & Saijo, 2017).

In an ex­am­ple much closer to gen­uine de­liber­a­tion (Hara, Yosh­ioka, Kuroda, Ku­ri­moto, & Saijo, 2019), in Ya­haba Town, Ja­pan, a se­ries of work­shops di­vided par­ti­ci­pants into fu­ture and pre­sent gen­er­a­tion groups to first de­liber­ate sep­a­rately and then to­gether in or­der to form a con­sen­sus over pri­ori­tiz­ing policy mea­sures for 2060. Each work­shop lasted 2.5 hours and par­ti­ci­pants had ac­cess to town hall staff mem­bers, uni­ver­sity fac­ulty mem­bers, and re­search ma­te­ri­als. As a con­di­tion for be­com­ing a fu­ture per­son of 2060, the par­ti­ci­pants were asked to as­sume that they had time-trav­eled to the year 2060 with­out ag­ing (i.e., they were of the same age in 2060 as at the pre­sent). The mem­bers of the imag­i­nary fu­ture-gen­er­a­tion groups wore spe­cial Ya­haba Town happi coats to help them iden­tify as part of the imag­i­nary fu­ture gen­er­a­tions.

(Par­ti­ci­pants in pre­sent day cloth­ing alongside those in imag­ined fu­ture gen­er­a­tion cloth­ing. Credit: Rit­suji Yosh­ioka )

The pre­sent-gen­er­a­tion groups drew up a vi­sion as an ex­ten­sion of the sta­tus quo, tak­ing the con­straints and challenges that ex­ist to­day as given, while the fu­ture-gen­er­a­tion groups called for efforts to ad­dress tough challenges in or­der to en­hance the strengths of the town. When both groups de­liber­ated to­gether the con­sen­sus ar­rived at in­cluded more than half of the mea­sures origi­nally pro­posed by the imag­i­nary fu­ture-gen­er­a­tion groups (in many cases be­ing ideas that were ab­sent from those pro­duced by the pre­sent-gen­er­a­tions group). This study is no­table for us­ing a real world set­ting. All of the vi­sions and mea­sures iden­ti­fied through­out the work­shop were crafted for in­clu­sion in a policy doc­u­ment pre­pared by the town to ad­dress Ja­panese gov­ern­ment policy.

The two ex­am­ples above seem to counter doubts that de­liber­a­tion on be­half of fu­ture gen­er­a­tions would be dis­torted and detri­men­tal (Jensen, 2015, p. 541). Longterm think­ing is not always suc­cess­fully achieved how­ever. Nie­meyer & Jennstål (2016) high­light a num­ber of ex­am­ples where fu­ture gen­er­a­tion ori­en­tated de­liber­a­tion was at­tempted but failed to pro­duce no­table effects. [7] The au­thors put this down to poor de­liber­a­tive de­sign in­clud­ing limited time available, the ab­sence of ac­tivi­ties fa­cil­i­tat­ing group de­vel­op­ment, the use of vot­ing on de­ci­sions, and a lack of di­ver­sity in per­spec­tives of the par­ti­ci­pants.

Non-hu­man animals

Similarly to the case of fu­ture gen­er­a­tions, it is pos­si­ble that de­liber­a­tion al­lows for ar­gu­ments on be­half of non-hu­man an­i­mals to get a fairer hear­ing. The de­liber­a­tive ex­er­cises we re­viewed had rel­a­tively lit­tle im­pact on policy mak­ers, al­though there was some ev­i­dence of an at­ti­tude shift amongst the par­ti­ci­pants, and these tended to be in the di­rec­tion of sup­port for greater pro­tec­tion for non-hu­man an­i­mals. The pro­vi­sion of in­for­ma­tion may be play­ing a big­ger role in these effects than the de­liber­a­tion it­self, but for our pur­poses this may not be a cru­cial con­sid­er­a­tion.

One case study ex­am­ines a de­liber­a­tive fo­rum, the Boyd Group, con­vened by civil so­ciety to find agree­ment on re­duc­ing, if not elimi­nat­ing, the use of non-hu­man an­i­mals in ex­per­i­ments (Garner, 2017). This does not meet all the crite­ria out­lined above since par­ti­ci­pants rep­re­sented groups, rather than ap­pear­ing as in­di­vi­d­ual cit­i­zens. As such the group con­sisted of ex­perts and par­ti­sans on the is­sue. While de­liber­a­tion had the effect of soft­en­ing some of the views and at­ti­tudes, there is lit­tle ev­i­dence that de­liber­a­tion sig­nifi­cantly shifted par­ti­ci­pants’ views on sub­stan­tive is­sues. The near con­sen­sus em­a­nat­ing from the Boyd Group de­liber­a­tion was to main­tain the sta­tus quo of the reg­u­la­tory pro­cess. The few changes that the Boyd Group did recom­mend (for ex­am­ple, the ban­ning of the test­ing of cos­metic in­gre­di­ents, and the ban on the use of the Great Apes) did find their way into the Bri­tish Govern­ment’s pro­gramme. How­ever, it is difficult to know whether the Home Office was already mov­ing in the di­rec­tion recom­mended by Boyd Group re­ports.

In the Welfare Qual­ity pro­ject,[8] the aim of the cit­i­zens’ ju­ries set up as part of the ex­er­cise was to as­sess cit­i­zens’ re­sponses to the farm an­i­mal welfare pro­to­cols drawn up by an­i­mal welfare sci­en­tists. It was re­ported that some ju­rors were ‘quite shocked and sur­prised’ by the re­al­ity of in­ten­sive an­i­mal agri­cul­ture, and all of the mem­bers of the Ital­ian jury, and the vast ma­jor­ity of the UK ju­rors, ad­mit­ted they were not aware of the sheer ex­tent of in­ten­sifi­ca­tion. In par­tic­u­lar, they were shocked about the stock­ing den­si­ties in broiler sheds, and the short life-spans of broiler chick­ens (Miele, Veissier, Evans, & Botreau, 2011, p. 113). The ju­rors in gen­eral were fully sup­port­ive of strong an­i­mal welfare mea­sures and they were not con­vinced that the pro­to­cols de­vel­oped by an­i­mal welfare sci­en­tists went far enough. Op­po­si­tion to fac­tory farm­ing in the Welfare Qual­ity pro­ject oc­curred be­cause the par­ti­ci­pants, hav­ing pos­ses­sion of the facts about the an­i­mal suffer­ing in­volved, came to the con­clu­sion that this suffer­ing out­weighed the hu­man benefits to be gained (for ex­am­ple, the cheap and plen­tiful sup­ply of meat) from the prac­tice. There was never any ques­tion of the de­liber­a­tors be­ing con­verted to the view that an­i­mals should not be used as sources of food. We do not know if there was a shift in opinion as a re­sult of de­liber­a­tion be­cause their views at the out­set of de­liber­a­tion were not mea­sured.

A de­liber­a­tive ex­er­cise on an­i­mal re­search was con­ducted in 2013. It came about as a re­sult of Ip­sos MORI (a Euro­pean pol­ling or­gani­sa­tion) be­ing com­mis­sioned by the lob­by­ing group Un­der­stand­ing An­i­mal Re­search (UAR). After hear­ing fac­tual in­for­ma­tion and the case for and against an­i­mal re­search, the par­ti­ci­pants across all of the work­shops be­came more favourably in­clined to­wards an­i­mal re­search (Ip­sos MORI, 2013, p. 24). How­ever, when un­der­cover footage of mis­de­meanors in lab­o­ra­to­ries was pro­vided ‘many re­verted to an op­po­si­tional stance in re­la­tion to an­i­mal re­search’ (2013, p. 25). As a re­sult, par­ti­ci­pants were much more will­ing to con­sider more rigor­ous scrutiny in­clud­ing in­sist­ing that li­cense ap­pli­ca­tions be sub­ject to ex­ter­nal scrutiny, and even that CCTV be placed in labs to be screened in pub­lic, an idea that gained ‘much sup­port’ (2013, p. 6, 42-4). How­ever, there was no con­ver­sion to the po­si­tion that an­i­mals should not be used as ex­per­i­men­tal sub­jects.

There was a ‘con­sis­tent lack of sup­port’ for xeno­trans­plan­ta­tion[9] in the so-called De­liber­a­tive Map­ping Pro­ject, and it was the worst perform­ing op­tion (out of nine) across all of the Ci­ti­zens’ Panels (Eames et al., 2004a). Par­ti­ci­pa­tory Tech­nol­ogy Assess­ment ex­er­cises in the Nether­lands and Canada recom­mended a mora­to­rium on xeno­trans­plan­ta­tion, whereas a minor­ity in the Swiss Par­ti­ci­pa­tory Tech­nol­ogy Assess­ment did so too (the ma­jor­ity opt­ing for reg­u­la­tion) (Griessler, 2011, p.38). In the De­liber­a­tive Map­ping Pro­ject too, many held a nega­tive view of xeno­trans­plan­ta­tion from the start but it also ‘suffered sig­nifi­cant nega­tive shifts in perfor­mance’ par­tic­u­larly af­ter the joint work­shop when mem­bers of the Ci­ti­zens’ Panels were ex­posed to the views of the ex­pert (Eames et al., 2004b, p. 39). How­ever, an­i­mal welfare was never the only, or the most im­por­tant, is­sue in any jury.

Policy influence

It may be all well and good if de­liber­a­tion can im­prove the de­ci­sion-mak­ing of its par­ti­ci­pants, but un­less we aim to con­vene the en­tire pop­u­la­tion in a ran­dom as­sem­bly (or tar­get small groups of key de­ci­sion-mak­ers) then it is im­por­tant to see if these mini-pub­lics have flowthrough effects on policy stem­ming from the con­clu­sions de­liber­a­tions reach.

Direct influence

Demo­cratic the­o­rist Dryzek has noted

di­rect in­fluence on and in policy mak­ing is a hard test for mini-pub­lics to pass. While ex­am­ples ex­ist of in­fluence and im­pact, they are out­num­bered by cases where a mini-pub­lic is es­tab­lished but turns out to have lit­tle or no effect on pub­lic de­ci­sion-mak­ing (2012, p. 170).

While seem­ingly rare, there are cases where de­liber­a­tive fo­rums have been given a role in the de­ci­sion-mak­ing pro­cess.

Ore­gon Ci­ti­zens’ Ini­ti­a­tive Re­view eval­u­ated two bal­lot mea­sures ap­pear­ing in the state’s gen­eral elec­tion and their con­clu­sions were in­cluded in voter guides sent out to the elec­torate. Through a pair of statewide sur­veys (one rol­ling cross-sec­tional and one on­line panel sur­vey), re­searchers found that Ore­gon vot­ers who read the Ci­ti­zens’ Ini­ti­a­tive Re­views’ state­ments be­came more knowl­edge­able about both mea­sures and that they were helpful in de­cid­ing how to vote on the is­sues that the mini-pub­lic pan­els stud­ied. How­ever, a ma­jor­ity of Ore­gon vot­ers re­mained un­aware of the pro­cess and did not read the state­ments. Fol­low­ing the pi­lot study the Ore­gon leg­is­la­ture cre­ated the Ci­ti­zens’ Ini­ti­a­tive Re­view Com­mis­sion in 2011. When re­peated in 2012, 52% of Ore­go­ni­ans who com­pleted their bal­lots were aware of the Ci­ti­zens’ Ini­ti­a­tive Re­view, and pre­limi­nary analy­ses again showed clear signs of in­fluence on voter de­ci­sion-mak­ing (Gastil & Richards, 2013). A re­view of Ci­ti­zens’ Ini­ti­a­tive Re­view pi­lot tests around the USA in 2018 found that vot­ers found the out­put use­ful and “at least two in five said it would make them more likely” to vote on the is­sue it ad­dressed and may have in­creased voter knowl­edge- in some cases “in­crease re­spon­dents’ ac­cu­racy rate by 10%—roughly the equiv­a­lent of ris­ing one grade level if tak­ing a true-or-false knowl­edge exam” (Gastil, 2019, p. 21)

Fol­low­ing de­liber­a­tion, the Bri­tish Columbia Ci­ti­zens’ Assem­bly made a recom­men­da­tion on elec­toral rules that was put to the vot­ers in a refer­en­dum. The bal­lot ul­ti­mate lost be­cause poli­cy­mak­ers re­quired a min­i­mum of 60% sup­port in provin­cial refer­enda be­fore they would com­mit to the recom­men­da­tions. Similarly to the limited effects of Ore­gon’s Ci­ti­zens’ Ini­ti­a­tive Re­view, the failures were at­tributed to a lack of pub­lic aware­ness and knowl­edge (Fournier, Kolk, Carty, Blais, & Rose, 2011). How­ever, hav­ing its recom­men­da­tions con­sid­ered and re­jected is im­por­tantly differ­ent from hav­ing them ig­nored al­to­gether, which seems more likely for purely ad­vi­sory recom­men­da­tions of mini-pub­lics.

(Bri­tish Columbia Ci­ti­zens’ Assem­bly on Elec­toral Re­form. Credit: Stu­art Davis, Van­cou­ver Sun)

The Ir­ish Con­sti­tu­tional Con­ven­tion gave its recom­men­da­tions on a list of pro­pos­als for con­sti­tu­tional re­forms and it could pro­pose any con­sti­tu­tional amend­ment it wished to be con­sid­ered. The gov­ern­ment com­mit­ted it­self to re­spond­ing to the recom­men­da­tions of the Con­sti­tu­tional Con­ven­tion in a timely fash­ion. It recom­mended le­gal­iz­ing gay mar­riage, which was in­tro­duced by pub­lic refer­en­dum in 2015. This was fol­lowed by a Ci­ti­zens’ Assem­bly which de­liber­ated on is­sues of abor­tion, fixed-term par­li­a­ments, refer­en­dums, pop­u­la­tion ag­ing and cli­mate change. The Ci­ti­zens’ Assem­bly recom­mended end­ing a con­sti­tu­tional ban on abor­tion and al­low­ing ter­mi­na­tions with­out re­stric­tion un­til up to 12 weeks of preg­nancy. The lat­ter is now law. How­ever, other recom­men­da­tions of both the Con­ven­tion and Ci­ti­zens’ as­sem­bly were ig­nored or re­main in the leg­is­la­tive grinder.

(Mem­bers of the Ir­ish Ci­ti­zens’ Assem­bly vote on the word­ing of the bal­lots that were to be sub­se­quently voted on. Credit: Maxwell’s )

Par­ti­ci­pa­tory Bud­get­ing in the Brazilian city of Porto Ale­gre, in which at the low­est of 3 or­gani­sa­tional tiers in­cluded a set of pop­u­lar Re­gional Assem­blies open to all, showed “great suc­cess in terms of macro-poli­ti­cal im­pact” (Baioc­chi, 2001) , with bud­get pri­ori­ties and rep­re­sen­ta­tives to the Fo­rums and Coun­cils be­ing de­ter­mined by di­rect vote in these pop­u­lar Re­gional Assem­blies. How­ever, par­ti­ci­pants were ei­ther self-se­lected (in the case of the Re­gional Assem­blies) or elected (for the other two tiers) and so do not fit all the crite­ria.

As dis­cussed above, a move­ment in Ja­pan called Fu­ture De­sign, led by economist Tat­suyoshi Saijo of the Re­search In­sti­tute for Hu­man­ity and Na­ture in Ky­oto, has been con­duct­ing cit­i­zen as­sem­blies in mu­ni­ci­pal­ities across the coun­try. Mul­ti­ple stud­ies in towns of Ya­haba, Suita, Mat­sumo­toto have shown that the fu­ture res­i­dents de­vise far more rad­i­cal and pro­gres­sive city plans com­pared to cur­rent ones.

A De­liber­a­tive Poll con­ducted by What’s Next Cal­ifor­nia in 2011 led to the draft­ing of Propo­si­tion 31, which ap­peared on the 2012 statewide bal­lot in Cal­ifor­nia, al­though it re­ceived only forty per­cent of the vote on Elec­tion Day. De­liber­a­tive polls have been vested with ac­tual de­ci­sion-mak­ing pow­ers on bud­getary mat­ters in some Chi­nese cities (Fishkin et al., 2010).

In­ter­est­ingly, it has been sug­gested that the more de­liber­a­tive mini-pub­lics are, the less likely they are to in­fluence policy (Cu­rato & Böker, 2016; Smith, Richards, & Gastil, 2015). Mini-pub­lics with low rep­re­sen­ta­tive­ness and low de­liber­a­tive qual­ity are most likely to pro­duce im­por­tant policy effects. Fur­ther­more, the more vot­ers knew about the Bri­tish Columbia Ci­ti­zen Assem­bly and Ir­ish Ci­ti­zen Con­ven­tion, such as their re­cruit­ment mechanisms or their free­dom from par­ti­san in­struc­tions, the more likely they were to vote for the mini-pub­lic’s policy recom­men­da­tion in the later cit­i­zen refer­en­dum (MacKen­zie & War­ren, 2012).

While mini-pub­lics have been used in a va­ri­ety of ways to help in­form policy de­ci­sions, they have less of­ten been em­pow­ered to di­rectly af­fect policy (Goodin & Dryzek, 2006). Even though the Bri­tish Columbia and On­tario Ci­ti­zens’ Assem­blies were each em­pow­ered to ini­ti­ate a refer­en­dum on elec­toral re­form in their re­spec­tive provinces, they were not em­pow­ered to ne­go­ti­ate policy op­tions with other in­fluen­tial ac­tors (Fournier et al., 2011)

Indi­rect influence

The di­rect poli­ti­cal and policy im­pact of de­liber­a­tive fo­rums may tend to be low, al­though this im­pres­sion is pos­si­bly due to there be­ing only a few ex­am­ples of mini-pub­lics be­ing for­mally em­pow­ered as part of the de­ci­sion-mak­ing pro­cess (Goodin & Dryzek, 2006, p.7). Real im­pact may have so far come mostly in­di­rectly by work­ing through the broader pub­lic sphere, or­di­nary leg­is­la­tive in­sti­tu­tions, and the mass me­dia. Even with­out be­ing part of the for­mal de­ci­sion-mak­ing pro­cess, they some­times have an in­fluence on policy. This is of­ten be­cause policy mak­ers take the opinion of peo­ple in­formed in the course of these events as au­thor­i­ta­tive, in prefer­ence to “raw” pub­lic opinion and take the op­tion to out­source/​del­e­gate the de­ci­sion for a con­tentious is­sue.

Some­times there will be soft guaran­tees that recom­men­da­tions will be acted upon as is the case for Amer­i­caS­peaks, Ci­ti­zens’ Juries and Con­sen­sus Con­fer­ences. For ex­am­ple, Richard E. Sclove, found­ing di­rec­tor of the Loka In­sti­tute found that

[con­sen­sus] con­fer­ences that were held in the late 1980s in­fluenced the Dan­ish Par­li­a­ment to pass leg­is­la­tion limit­ing the use of ge­netic screen­ing in hiring and in­surance de­ci­sions, to ex­clude ge­net­i­cally mod­ified an­i­mals from the gov­ern­ment’s ini­tial biotech­nol­ogy re­search and de­vel­op­ment pro­gram, and to pro­hibit food ir­ra­di­a­tion for ev­ery­thing ex­cept dry spices.(Sclove, n.d.)

Of course, it can­not be proven that the Con­sen­sus Con­fer­ence was the de­ci­sive in­fluence; skep­tics might say this is what gov­ern­ment policy could have been any­way.

The Cen­ter for Global Devel­op­ment has high­lighted a num­ber of ex­am­ples. In Zeguo, China, a city of about 250,000 in­hab­itants, de­liber­a­tive polls helped to chan­nel pub­lic spend­ing from high-visi­bil­ity pub­lic works to more ba­sic en­vi­ron­men­tal pro­jects, in­clud­ing a sewage treat­ment plant. Zeguo has con­tinued De­liber­a­tive Pol­ling for bud­get and in­fras­truc­ture on a nearly an­nual ba­sis since 2005 and De­liber­a­tive Pol­ling has spread to other ar­eas of China. In Bul­garia, a na­tional De­liber­a­tive Poll con­ducted with the par­ti­ci­pa­tion of the prime minister led to the end of seg­re­gated schools for the his­tor­i­cally shunned Roma minor­ity and the clos­ing of Roma-only schools. In Italy’s Re­gione Lazio, the state that in­cludes Rome, a De­liber­a­tive Poll on bud­getary is­sues for health policy helped solve a long-stand­ing prob­lem of too many hos­pi­tal beds and not enough walk-in clinics. Poli­ti­ci­ans said the poll re­sults pro­vided cover to do the right thing (Fishkin, 2011). In Ger­many, a Green-Left gov­ern­ment in­tro­duced and in­sti­tu­tion­al­ized fo­rums for cit­i­zen par­ti­ci­pa­tion and de­liber­a­tion, sub­se­quently tak­ing up the policy recom­men­da­tions of those fo­rums (Bächtiger & Beste, 2017, p. 113).

Tak­ing a step away from the poli­ti­cal pro­cess, eight elec­tric util­ities in differ­ent parts of Texas com­mis­sioned De­liber­a­tive Polls be­tween 1996 and 1998, ask­ing cus­tomers how they preferred that fu­ture elec­tric­ity re­quire­ments to be met. After­wards util­ity com­pa­nies be­gan to in­te­grate such con­sumer val­ues about en­ergy choices into their de­ci­sions (Lehr, Guild, Thomas, & Swezey, 2003). Ob­servers of the poll in­cluded elected offi­cials, pub­lic util­ity com­mis­sion­ers, and power com­pany ex­ec­u­tives, many of whom said in sub­se­quent in­ter­views that watch­ing the de­liber­a­tion was an im­por­tant fac­tor in chang­ing their views. The Texas util­ity com­mis­sioner said that sub­se­quent changes in state policy and util­ity com­pany in­vest­ment poli­cies were a di­rect re­sult of the de­liber­a­tive polls.

Novo Nordisk, a large Dan­ish biotech­nol­ogy com­pany, reeval­u­ated its re­search and de­vel­op­ment strate­gies af­ter a 1992 panel de­plored the de­sign of an­i­mals suited to the rigors of ex­ist­ing agri­cul­tural sys­tems but en­dorsed the use of ge­netic en­g­ineer­ing to help treat in­cur­able dis­eases. The firm moved to con­cen­trate on work more likely to win pop­u­lar ap­proval, such as an­i­mal-based pro­duc­tion of drugs for se­vere hu­man ill­nesses (Sclove, n.d.).

An Aus­tralian con­sen­sus con­fer­ence on GM foods in 1999 was spon­sored by the Con­umser’s As­so­ci­a­tion, en­dorsed by gov­ern­ment, recog­nised by anti-GM ac­tivists, and allegedly in­fluenced the po­si­tions taken by the biotech in­dus­try (Hen­driks, 2004). How­ever, the policy im­pact of other con­sen­sus con­fer­ences on GM food in Den­mark, France, and the US, has been harder to show, in part be­cause their recom­men­da­tions were in­line with ex­ist­ing gov­ern­ment policy (Dryzek & Tucker, 2008).

In re­sponse to a tes­ti­mony from the Loka In­sti­tute a 2003 en­act­ment of the U.S. Congress speci­fied “mechanisms such as cit­i­zens’ pan­els, con­sen­sus con­fer­ences, and ed­u­ca­tional events, as ap­pro­pri­ate” as ways of “en­sur­ing that eth­i­cal, le­gal, en­vi­ron­men­tal and other ap­pro­pri­ate so­cial con­cerns … are con­sid­ered dur­ing the de­vel­op­ment of nan­otech­nol­ogy”.[10] How­ever, we have not seen much ev­i­dence of this be­ing im­ple­mented yet.

Rea­sons to doubt de­liber­a­tive mini-publics

One of the ma­jor claimed ad­van­tages of de­liber­a­tive bod­ies also re­veals a ma­jor demo­cratic cri­tique. De­liber­a­tion helps par­ti­ci­pants to make con­sid­ered judg­ments that are, be­cause of rep­re­sen­ta­tive sam­pling, judged to be what the wider pub­lic would share if they were to par­ti­ci­pate. How­ever, the larger pop­u­la­tion are left with “un­con­sid­ered” opinions. The ob­vi­ous im­pli­ca­tion here is that the raw voice of the ac­tual peo­ple “is not a voice that by it­self de­serves any spe­cial hear­ing” (Fishkin, 2013, p. 504) or that the de­liber­a­tive opinion is worth more. One need not be com­mit­ted to this view that the raw-opinions do not de­serve any spe­cial hear­ing, they might merely be as­signed some weight alongside the de­liber­a­tive views. Selected de­liber­a­tive cit­i­zens groups may also be able to act as rep­re­sen­ta­tives of the pub­lic (as per elected rep­re­sen­ta­tives) even if they don’t en­act what the pub­lic would them­selves se­lect. There are also many in­stances one can imag­ine where the raw pub­lic opinion would not be an in­put any­way and so mini-pub­lics are not su­perced­ing them. It also seems defend­able to en­dorse the out­comes from de­liber­a­tive mini-pub­lics as le­gi­t­i­mate even if we do not be­lieve a de­liber­a­tion of the en­tire pop­u­la­tion would reach the same con­clu­sion. For ex­am­ple, this seems to be the case for ju­ries.

There are con­cerns that while ran­dom sam­pling pre­vents overt self-se­lec­tion, there is still self-se­lec­tion in who ac­cepts be­ing in­vited. There is some ev­i­dence that peo­ple with more ed­u­ca­tion and in­ter­est in poli­tics have been found to be more likely to vol­un­teer to par­ti­ci­pate in mini-pub­lics (Kar­jalainen & Rapeli, 2015). Fur­ther­more, even within mini-pub­lics cer­tain peo­ple tend to dom­i­nate pro­ceed­ings and am­plify in­equal­ities in the wider so­ciety from which they are drawn. Stud­ies of De­liber­a­tive Polls have found far less dis­tor­tion than crit­ics ex­pect though (Siu, 2017). More re­search with con­trol­led ex­per­i­ments could clar­ify this is­sue fur­ther. Again, one may only be con­cerned with this is­sue to the ex­tent that greater rep­re­sen­ta­tive­ness leads to more pub­lic and/​or elite sup­port for de­liber­a­tive bod­ies’ con­clu­sions.

It is un­clear whether changes in peo­ple’s views pro­duced by De­liber­a­tive Polls and other con­sul­ta­tive mechanisms tried thus far are re­ally im­prove­ments on their pre-de­liber­a­tive views or sim­ply changes. This clearly then de­pends on what out­comes one judges to be good or bad. With re­gard to benefit­ing fu­ture gen­er­a­tions the ex­am­ples given above have pro­vided some ideas on how to pro­mote pos­i­tive longterm changes.

Policy-mak­ers may or­ga­nize mini-pub­lics in or­der to strengthen their own po­si­tion in the eyes of the pub­lic, or to ad­vance and le­gi­t­imize poli­cies they pur­sue, as some­times in the case of di­rect democ­racy ini­ti­a­tives like refer­enda. The use of mini-pub­lics could be re­quired when par­li­a­ments are leg­is­lat­ing on cer­tain types of is­sues. Th­ese could in­clude, for ex­am­ple, con­sti­tu­tional is­sues, elec­toral re­form and party fi­nanc­ing, and per­haps cer­tain types of tech­ni­cally com­plex mat­ters, which was the case in the Dan­ish prac­tice of con­sen­sus con­fer­ences.

Some may be con­cerned that or­ganis­ing such mini-pub­lics im­posed too large a fi­nan­cial bur­den. A 2017 re­port for the POWER in­quiry in Bri­tain sug­gests costs can range from £16,000 to £200,000 (~$19,500 to $245,000 in 2019 USD) (Smith, 2005, p. 51). In par­tic­u­lar, tele­vis­ing part of the con­tent through mass me­dia can be very ex­pen­sive. It can be ex­pen­sive to get suffi­cient num­bers of par­ti­ci­pants to cre­ate a good rep­re­sen­ta­tive sam­ple and im­prov­ing the odds that mem­bers of more marginal­ized groups will at­tend. Since it’s not a duty like jury duty, stud­ies con­ducted so far re­quire in­cen­tives, with costs that have in­cluded pay­ing for the trips, the ho­tel and the food for each par­ti­ci­pant, and book­ing an at­trac­tive venue; in any case, hiring the re­search crew and mod­er­a­tors will in­cur costs. Ad­di­tional costs have in­cluded pay­ing for par­ti­ci­pants’ com­pen­sa­tion so that the peo­ple who are ran­domly se­lected can put aside their du­ties to at­tend the events (e.g., pro­vid­ing child care.). There­fore, if nei­ther the gov­ern­ing body nor other or­ga­ni­za­tions are will­ing to fund such a poll, there is no way to get it started. How­ever, these still seem likely to be cheaper and eas­ier than other pro­posed re­forms like chang­ing the elec­toral sys­tem.

Groups do­ing this work

There ap­pear to be many or­gani­sa­tions ex­per­i­ment­ing with the var­i­ous de­liber­a­tive de­signs de­scribed above. It’s pos­si­ble fu­ture work could iden­tify the most effec­tive or­gani­sa­tions or strate­gies and in­form bet­ter co­or­di­na­tion and de­ploy­ment of the­ses or­ga­ni­za­tions’ re­sources. This is not an ex­haus­tive list, but just some of the most promi­nent or­gani­sa­tions work­ing in the de­liber­a­tive space.

The World Fu­ture Coun­cil is propos­ing to es­tab­lish a de­liber­a­tive coun­cil of Guardians for Fu­ture Gen­er­a­tions, an in­de­pen­dent body that offers ad­vice, recom­men­da­tions, anal­y­sis, and ac­tively ad­vo­cates for long-term in­ter­ests .

The Alli­ance for Fu­ture Gen­er­a­tions, a group of more than forty or­gani­sa­tions and in­di­vi­d­u­als work­ing to bring longter­mism and the needs of fu­ture gen­er­a­tions into the heart of UK democ­racy and policy pro­cesses, want to try to de­velop an (NGO-led) pi­lot scheme for lo­cal coun­cils of Guardians of the Fu­ture Gen­er­a­tions (Read, 2012).

The Cen­ter for De­liber­a­tive Democ­racy, housed in the Depart­ment of Com­mu­ni­ca­tion at Stan­ford Univer­sity, is de­voted to re­search about democ­racy and pub­lic opinion ob­tained through De­liber­a­tive Pol­ling.

The Re­search In­sti­tute for Hu­man­ity and Na­ture & Re­search In­sti­tute for Fu­ture De­sign work on psy­chol­ogy ex­per­i­ments and fu­ture gen­er­a­tion games in Ja­panese mu­ni­ci­pal­ities, as dis­cussed in the ex­am­ples above.

Healthy Democ­racy has con­tinued to con­duct bi­en­nial Ci­ti­zens’ Ini­ti­a­tive Re­views in Ore­gon. They have also run statewide Ci­ti­zens’ Ini­ti­a­tive Re­views in Ari­zona, Colorado, and Mas­sachusetts, in ad­di­tion to Ci­ti­zens’ Ini­ti­a­tive Re­views at the city and county level. In March 2017, they con­ducted their first Ci­ti­zens’ Ini­ti­a­tive Re­view demon­stra­tion in Cal­ifor­nia.

The Jeffer­son Cen­ter for New Demo­cratic Pro­cesses hold Ci­ti­zen’s Juries and in­formed the cre­ation of Ci­ti­zens’ Ini­ti­a­tive Re­views by Healthy Democ­racy.

In­volve(UK) raise aware­ness, build coal­i­tions of civil so­ciety, poli­ti­cal in­sti­tu­tions and me­dia, and use demon­stra­tion pro­jects to show the effi­cacy of pub­lic par­ti­ci­pa­tion.


Many of the re­forms dis­cussed here are com­pli­men­tary to other pro­pos­als for im­prov­ing in­sti­tu­tional de­ci­sion-mak­ing but there are also some po­ten­tial ad­van­tages of us­ing de­liber­a­tive bod­ies in com­par­i­son to them. As has been men­tioned above, de­liber­a­tive mini-pub­lics seem to be en­dorsed by the pub­lic as le­gi­t­i­mate at least in part be­cause of their com­po­si­tion of ran­domly cho­sen cit­i­zens. This may offer a more promis­ing route than es­tab­lish­ing more (po­ten­tially dis­trusted) elected in­sti­tu­tions, tech­no­cratic offices, or tin­ker­ing with the norms around the weight­ing of votes. Task­ing cit­i­zens ex­plic­itly with judg­ing whether poli­cies pro­duce the best out­come may be prefer­able to us­ing prox­ies for bet­ter de­ci­sion-mak­ing, as in the case of age-weighted vot­ing.

This re­view has also high­lighted a num­ber of ar­eas for fur­ther re­search. We have sketched out some ex­am­ples here of the im­pact of de­liber­a­tion but a more thor­ough quan­ti­ta­tive anal­y­sis would help in weigh­ing the cost benefit anal­y­sis of such re­forms. We also need more in­for­ma­tion on the fac­tors that may in­fluence the im­pact of mini-pub­lics on pub­lic opinion such as the size of the mini-pub­lic, length of time spent on de­liber­a­tion, fram­ing of in­for­ma­tion re­ceived and de­ci­sions made, spon­sor­ship of the de­liber­a­tive event and level of gov­ern­ment tar­geted with the de­liber­a­tive event. It is too early to know what the long run effects of many of the de­liber­a­tive bod­ies will be. It may be in­for­ma­tive then to con­duct case stud­ies of de­liber­a­tive fo­rums from the past and con­sider what the im­pact of any longterm de­ci­sions they made have been.


Thought­ful com­ments by Matt_Lerner, Gre­gory_Lewis, and Ste­fan_Schu­bert pro­vided some rea­sons to more clearly iden­tify the coun­ter­ar­gu­ments and any pub­li­ca­tion bias in the ex­ist­ing liter­a­ture. Spe­cific ex­am­ples of de­liber­a­tive stud­ies in Ari­zona and Den­mark were re­moved be­cause with­out know­ing how rep­re­sen­ta­tive they are it may have sug­gested greater con­fi­dence and pre­ci­sion in effect sizes than is ap­pro­pri­ate.


This es­say is a pro­ject of Re­think Pri­ori­ties. It was writ­ten by Neil Dul­laghan. Thanks to David Moss, Ja­son Schukraft, John W. Gastil and Do­minic Roser for com­ments. If you like our work, please con­sider sub­scribing to our newslet­ter. You can see all our work to date here.

Works cited

An­der­sen, V. N., & Hansen, K. M. (2007). How de­liber­a­tion makes bet­ter cit­i­zens: The Dan­ish De­liber­a­tive Poll on the euro. Euro­pean Jour­nal of Poli­ti­cal Re­search, 46(4), 531–556.

Ar­mor, D. A., & Tay­lor, S. E. (2003). The Effects of Mind­set on Be­hav­ior: Self-Reg­u­la­tion in De­liber­a­tive and Im­ple­men­tal Frames of Mind. Per­son­al­ity and So­cial Psy­chol­ogy Bul­letin, 29(1),86–95.

Bächtiger, A., & Beste, S. (2017). De­liber­a­tive Ci­ti­zens, (Non)De­liber­a­tive Poli­ti­ci­ans: A Re­join­der. Daedalus, 146(3), 106–118.

Bächtiger, A., Spörndli, M., Steen­ber­gen, M. R., & Steiner, J. (2005). The De­liber­a­tive Di­men­sions of Leg­is­la­tures. Acta Poli­tica, 40(2), 225–238.

Baioc­chi, G. (2001). Par­ti­ci­pa­tion, Ac­tivism, and Poli­tics: The Porto Ale­gre Ex­per­i­ment and De­liber­a­tive Demo­cratic The­ory. Poli­tics & So­ciety, 29(1), 43–72.

Barabas, J. (2004). How De­liber­a­tion Affects Policy Opinions. The Amer­i­can Poli­ti­cal Science Re­view, 98(4), 687–701.

Barry, B. (1978). Cir­cum­stances of Jus­tice and Fu­ture Gen­er­a­tions. In R. I. Sikora & B. M. Barry (Eds.), Obli­ga­tions to Fu­ture Gen­er­a­tions (pp. 204–48). White Horse Press.

Bou­li­anne, S. (2018). Mini-pub­lics and Public Opinion: Two Sur­vey-Based Ex­per­i­ments. Poli­ti­cal Stud­ies, 66(1), 119–136.

Bou­li­anne, S., Lopt­son, K., & Ka­hane, D. (2018). Ci­ti­zen pan­els and opinion polls: Con­ver­gence and di­ver­gence in policy prefer­ences. Jour­nal of Public De­liber­a­tion, 14.

Bovenkerk, B. (2015). Public De­liber­a­tion and the In­clu­sion of Fu­ture Gen­er­a­tions. Jurispru­dence, 6(3), 496–515.

Bren­nan, J. (2016). Against Democ­racy (First Edi­tion edi­tion). Prince­ton: Prince­ton Univer­sity Press.

But­ton, M., & Matt­son, K. (1999). De­liber­a­tive Democ­racy in Prac­tice: Challenges and Prospects for Civic De­liber­a­tion. Polity, 31(4), 609–637.

Car­son, L., & Martin, B. (1999). Ran­dom Selec­tion in Poli­tics. Fac­ulty of Arts—Papers (Archive). Retrieved from

Cook, F. L., & Ja­cobs, L. (1998). De­liber­a­tive Democ­racy in Ac­tion: Eval­u­a­tion of Amer­i­cans Dis­cuss So­cial Se­cu­rity: Pew Char­i­ta­ble Trusts.

Coppedege, M., Ger­ring, J., Lind­berg, S. I., Skaan­ing, S.-E., Te­o­rell, J., Alt­man, D., & Wil­son, S. (2017). V-dem coun­try-year data, ver­sion 7.1. Va­ri­eties of Democ­racy (V-Dem) Pro­ject.

Cu­rato, N., & Böker, M. (2016). Link­ing mini-pub­lics to the de­liber­a­tive sys­tem: A re­search agenda. Policy Sciences, 49(2), 173–190.

Den­ver, D., Hands, G., & Jones, B. (1995). Fishkin and the de­liber­a­tive opinion poll: Les­sons from a study of the Granada 500 tele­vi­sion pro­gram. Poli­ti­cal Com­mu­ni­ca­tion, 12(2), 147–156.

Dietz, T., C. Stern, P., & Dan, A. (2009). How De­liber­a­tion Affects Stated Willing­ness to Pay for Miti­ga­tion of Car­bon Diox­ide Emis­sions: An Ex­per­i­ment. Land Eco­nomics, 85, 329–347.

Dryzek, J. S. (2007). The­ory, Ev­i­dence, and the Tasks of De­liber­a­tion. In S. W. Rosen­berg (Ed.), De­liber­a­tion, Par­ti­ci­pa­tion and Democ­racy: Can the Peo­ple Govern? (pp. 237–250).

Dryzek, J. S. (2012). Foun­da­tions and Fron­tiers of De­liber­a­tive Gover­nance. Oxford Univer­sity Press.

Dryzek, J. S., & Tucker, A. (2008). De­liber­a­tive In­no­va­tion to Differ­ent Effect: Con­sen­sus Con­fer­ences in Den­mark, France, and the United States. Public Ad­minis­tra­tion Re­view, 68(5), 864–876.

Eames, M., Burgess, J., Davies, G., Mayer, S., Staley, K., Stir­ling, A., & Willi­am­son, S. (2004a). Xeno­trans­plan­ta­tion and its al­ter­na­tives for ad­dress­ing the ‘kid­ney gap’: A case study re­port from the de­liber­a­tive map­ping pro­ject [Re­port].

Eames, M., Burgess, J., Davies, G., Mayer, S., Staley, K., Stir­ling, A., & Willi­am­son, S. (2004b). Xeno­trans­plan­ta­tion and its al­ter­na­tives for ad­dress­ing the ‘kid­ney gap’: A case study re­port from the de­liber­a­tive map­ping pro­ject [Re­port].

Ekeli, K. sk­a­gen. (2009). Con­sti­tu­tional Ex­per­i­ments: Rep­re­sent­ing Fu­ture Gen­er­a­tions Through Sub­ma­jor­ity Rules. Jour­nal of Poli­ti­cal Philos­o­phy, 17(4), 440–461.

Far­rar, C., Fishkin, J. S., Green, D. P., List, C., Luskin, R. C., & Paluck, E. L. (2010). Disag­gre­gat­ing De­liber­a­tion’s Effects: An Ex­per­i­ment within a De­liber­a­tive Poll. Bri­tish Jour­nal of Poli­ti­cal Science, 40(2), 333–347.

Fishkin, J. S. (2011). When the Peo­ple Speak: De­liber­a­tive Democ­racy and Public Con­sul­ta­tion.

Fishkin, J. S. (2013). De­liber­a­tion by the Peo­ple Them­selves: En­try Points for the Public Voice. Elec­tion Law Jour­nal: Rules, Poli­tics, and Policy, 12(4), 490–507.

Fishkin, J. S., & Luskin, R. C. (1999). Bring­ing De­liber­a­tion to the Demo­cratic Dialogue. In M. McCombs & A. Reynolds, The Poll With A Hu­man Face The Na­tional Is­sues Con­ven­tion Ex­per­i­ment in Poli­ti­cal Com­mu­ni­ca­tion.

Fishkin, J. S., He, B., Luskin, R. C., & Siu, A. (2010). De­liber­a­tive democ­racy in an un­likely place: De­liber­a­tive Pol­ling in China. Bri­tish Jour­nal of Poli­ti­cal Science, 40(2), 435–448.

Fishkin, J. S., & Luskin, R. C. (2005). Ex­per­i­ment­ing with a Demo­cratic Ideal: De­liber­a­tive Pol­ling and Public Opinion. Acta Poli­tica, 40(3), 284–298.

Fishkin, J. S., & Mans­bridge, J. (2017). The Prospects & Limits of De­liber­a­tive Democ­racy: In­tro­duc­tion. Jour­nal of the Amer­i­can Academy of Arts & Sciences, 146(3), 6–13.

Fishkin, J. S., Mayega, R. W., Atuyambe, L., Tu­muhamye, N., Ssen­tongo, J., Siu, A., & Bazeyo, W. (2017). Ap­ply­ing De­liber­a­tive Democ­racy in Africa: Uganda’s First De­liber­a­tive Polls. Daedalus, 146(3).

Fournier, P., Kolk, H. van der, Carty, R. K., Blais, A., & Rose, J. (2011). When Ci­ti­zens De­cide: Les­sons from Ci­ti­zen Assem­blies on Elec­toral Re­form.

French, D., & Laver, M. (2009). Par­ti­ci­pa­tion Bias, Durable Opinion Shifts and Sab­o­tage through With­drawal in Ci­ti­zens’ Juries. Poli­ti­cal Stud­ies, 57(2), 422–450.

Fung, A. (2003). Sur­vey Ar­ti­cle: Recipes for Public Spheres: Eight In­sti­tu­tional De­sign Choices and Their Con­se­quences. Jour­nal of Poli­ti­cal Philos­o­phy, 11(3), 338–367.

Garner, R. (2017). The Boyd Group and An­i­mal Ex­per­i­men­ta­tion. Global Jour­nal of An­i­mal Law, 5(1).

Gastil, J. (2019). Assess­ment of Three 2018 Ci­ti­zens’ Ini­ti­a­tive Re­view Pilot Pro­jects. [Re­port pre­pared for Healthy Democ­racy]. Univer­sity Park, PA: Penn­syl­va­nia State Univer­sity.

Gastil, J., Deess, E. P., & Weiser, P. (2002). Civic Awak­en­ing in the Jury Room: A Test of the Con­nec­tion be­tween Jury De­liber­a­tion and Poli­ti­cal Par­ti­ci­pa­tion. The Jour­nal of Poli­tics, 64(2), 585–595.

Gastil, J., & Dillard, J. P. (1999). In­creas­ing Poli­ti­cal So­phis­ti­ca­tion Through Public De­liber­a­tion. Poli­ti­cal Com­mu­ni­ca­tion, 16(1), 3–23.

Gastil, J., & Richards, R. (2013). Mak­ing Direct Democ­racy De­liber­a­tive through Ran­dom Assem­blies. Poli­tics & So­ciety, 41(2), 253–281.

Ger­ber, M., Bächtiger, A., Shikano, S., Re­ber, S., & Rohr, S. (2018). De­liber­a­tive Abil­ities and In­fluence in a Transna­tional De­liber­a­tive Poll (EuroPo­lis). Bri­tish Jour­nal of Poli­ti­cal Science, 48(4), 1093–1118.

Goodin, R. E. (2007). En­fran­chis­ing All Affected In­ter­ests, and Its Alter­na­tives. Philos­o­phy & Public Af­fairs, 35(1), 40–68.

Goodin, R. E., & Dryzek, J. S. (2006). De­liber­a­tive Im­pacts: The Macro-Poli­ti­cal Up­take of Mini-Publics. Poli­tics & So­ciety, 34(2), 219–244.

Gun­der­sen, A. G. (1995). The En­vi­ron­men­tal Promise of Demo­cratic De­liber­a­tion. Univ of Wis­con­sin Press.

Grön­lund, K., Setälä, M., & Herne, K. (2010). De­liber­a­tion and civic virtue: Les­sons from a cit­i­zen de­liber­a­tion ex­per­i­ment. Euro­pean Poli­ti­cal Science Re­view, 2(1), 95–117.

Grön­lund, K., Herne, K., & Setälä, M. (2017). Em­pa­thy in a Ci­ti­zen De­liber­a­tion Ex­per­i­ment. Scan­d­i­na­vian Poli­ti­cal Stud­ies, 40(4), 457–480.

Griessler, R. (2011) ‘Xeno­trans­plan­ta­tion Policy and Par­ti­ci­pa­tory Tech­nol­ogy Assess­ment in Switzer­land’, In­sti­tute for Ad­vanced Stud­ies, So­ciolo­gial Series 95, Vienna

Haber­mas, J. (1984). The The­ory of Com­mu­nica­tive Ac­tion. Bea­con Press.

Haber­mas, J. (1996). Between Facts and Norms: Con­tri­bu­tions to a Dis­course The­ory of Law and Democ­racy. Polity.

Hara, K., Yosh­ioka, R., Kuroda, M., Ku­ri­moto, S., & Saijo, T. (2019). Rec­on­cil­ing in­ter­gen­er­a­tional con­flicts with imag­i­nary fu­ture gen­er­a­tions: Ev­i­dence from a par­ti­ci­pa­tory de­liber­a­tion prac­tice in a mu­ni­ci­pal­ity in Ja­pan. Sus­tain­abil­ity Science.

He, B., & War­ren, M. E. (2011). Author­i­tar­ian De­liber­a­tion: The De­liber­a­tive Turn in Chi­nese Poli­ti­cal Devel­op­ment. Per­spec­tives on Poli­tics, 9(2), 269–289.

Hen­driks, C. (2002). In­sti­tu­tions of De­liber­a­tive Demo­cratic Pro­cesses and In­ter­est Groups: Roles, Ten­sions and In­cen­tives. Aus­tralian Jour­nal of Public Ad­minis­tra­tion, 61(1), 64–75.

Hen­driks, C. (2004). Public de­liber­a­tion and in­ter­est or­gani­sa­tions: A study of re­sponses to lay cit­i­zen en­gage­ment in pub­lic policy.

Hib­bing, J. R., & Theiss-Morse, E. (2002). Stealth Democ­racy: Amer­i­cans’ Beliefs About How Govern­ment Should Work. Cam­bridge ; New York: Cam­bridge Univer­sity Press.

Holt, D. L. (1993). Ra­tion­al­ity is hard work: An al­ter­na­tive in­ter­pre­ta­tion of the dis­rup­tive effects of think­ing about rea­sons. Philo­soph­i­cal Psy­chol­ogy, 6(3), 251–266. Holt, L. (1999). Ra­tion­al­ity is still hard work: Some fur­ther notes on the dis­rup­tive effects of de­liber­a­tion. Philo­soph­i­cal Psy­chol­ogy, 12(2), 215–219.

Ip­sos MORI (2013) ’Ope­ness in An­i­mal Re­search: The Public’s Views on Open­ness and Trans­parency in An­i­mal Re­search.

Iyen­gar, S. S., & Lep­per, M. R. (2000). When choice is de­mo­ti­vat­ing: Can one de­sire too much of a good thing? Jour­nal of Per­son­al­ity and So­cial Psy­chol­ogy, 79(6), 995–1006.

Jac­quet, V. (2019). The Role and the Fu­ture of De­liber­a­tive Mini-pub­lics: A Ci­ti­zen Per­spec­tive. Poli­ti­cal Stud­ies, 67(3), 639–657.

Jensen, K. K. (2015). Fu­ture Gen­er­a­tions in Democ­racy: Rep­re­sen­ta­tion or Con­sid­er­a­tion? Jurispru­dence, 6(3), 535–548.

Kamijo, Y., Komiya, A., Mifune, N., & Saijo, T. (2017). Ne­go­ti­at­ing with the fu­ture: In­cor­po­rat­ing imag­i­nary fu­ture gen­er­a­tions into ne­go­ti­a­tions. Sus­tain­abil­ity Science, 12(3), 409–420.

Kar­jalainen, M., & Rapeli, L. (2015). Who will not de­liber­ate? At­tri­tion in a multi-stage cit­i­zen de­liber­a­tion ex­per­i­ment. Qual­ity & Quan­tity: In­ter­na­tional Jour­nal of Method­ol­ogy, 49(1), 407–422.

Kuyper, J. (2018). The In­stru­men­tal Value of De­liber­a­tive Democ­racy – Or, do we have Good Rea­sons to be De­liber­a­tive Democrats? Jour­nal of Public De­liber­a­tion, 14(1).

Lafont, C. (2017). Can Democ­racy be De­liber­a­tive & Par­ti­ci­pa­tory? The Demo­cratic Case for Poli­ti­cal Uses of Mini-Publics. Jour­nal of the Amer­i­can Academy of Arts & Sciences, 146(3), 85–105.

Lehr, R. L., Guild, W., Thomas, D. L., & Swezey, B. G. (2003). Listen­ing to Cus­tomers: How De­liber­a­tive Pol­ling Helped Build 1,000 MW of New Re­new­able En­ergy Pro­jects in Texas (No. NREL/​TP-620-33177).

Leib, E. J. (2004). De­liber­a­tive Democ­racy in Amer­ica: A Pro­posal for a Pop­u­lar Branch of Govern­ment. Retrieved from

Lown­des, V., Pratch­ett, L., & Stoker, G. (2001). Trends in Public Par­ti­ci­pa­tion: Part 2 – Ci­ti­zens’ Per­spec­tives. Public Ad­minis­tra­tion, 79(2), 445–455.

Luskin, R. C., Fishkin, J. S., Jow­ell, R., & Park, A. (1999a). Learn­ing and Vot­ing in Bri­tain: In­sights from the De­liber­a­tive Poll. Paper Pre­sented at the An­nual Meet­ing of the Amer­i­can Poli­ti­cal Science As­so­ci­a­tion, At­lanta.

Luskin, R. C., Fishkin, J. S., & Plane, D. L. (1999b). De­liber­a­tive Pol­ling and Policy Out­comes: Elec­tric Utility Is­sues in Texas.

Luskin, R. C., Fishkin, J. S., & Jow­ell, R. (2002). Con­sid­ered opinions: De­liber­a­tive pol­ling in Bri­tain. Bri­tish Jour­nal of Poli­ti­cal Science, 32(3), 455–487.

Luskin, R. C., Iyen­gar, S. S., & Fishkin, J. S. (2004). Con­sid­ered Opinions on U.S. For­eign Policy: Face-to-Face ver­sus On­line De­liber­a­tive Pol­ling*. Re­search Paper, Cen­ter for De­liber­a­tive Democ­racy.

Luskin, R. C., O’Flynn, I., Fishkin, J. S., & Rus­sell, D. (2014). De­liber­at­ing across Deep Di­vides. Poli­ti­cal Stud­ies, 62(1), 116–135.

Ma, D., & Hsu, S. (2018). The Poli­ti­cal Con­se­quences of De­liber­a­tive Democ­racy and Elec­toral Democ­racy in China: An Em­piri­cal Com­par­a­tive Anal­y­sis from Four Coun­ties. China Re­view, 18(2), 1–32.

MacKen­zie, M. K. (2018). De­liber­a­tion and Long-Term De­ci­sions. In The Oxford Hand­book of De­liber­a­tive Democ­racy. Retrieved from

MacKen­zie, M., & War­ren, M. E. (2012). Two Trust-Based Uses of Minipub­lics in Demo­cratic Sys­tems. In J. Park­in­son & J. Mans­bridge (Eds.), De­liber­a­tive Sys­tems: De­liber­a­tive Democ­racy at the Large Scale. Cam­bridge: Cam­bridge Univer­sity Press.

Manin, B. (1997). The Prin­ci­ples of Rep­re­sen­ta­tive Govern­ment.

Mans­bridge, J. (2010). De­liber­a­tive Pol­ling as the Gold Stan­dard. The Good So­ciety, 19(1), 55–62.

McCormick, J. P. (2006). Con­tain the Wealthy and Pa­trol the Mag­is­trates: Restor­ing Elite Ac­countabil­ity to Pop­u­lar Govern­ment. The Amer­i­can Poli­ti­cal Science Re­view, 100(2), 147–163.

Men­delberg, T., & Oleske, J. (2000). Race and Public De­liber­a­tion. Poli­ti­cal Com­mu­ni­ca­tion, 17(2), 169–191.

Miele, M., Veissier, I., Evans, A., & Botreau, R. (2011). An­i­mal welfare: Estab­lish­ing a di­alogue be­tween sci­ence and so­ciety. An­i­mal Welfare (South Mimms, England), 20.

Mor­rell, M. E. (2005). De­liber­a­tion, demo­cratic de­ci­sion-mak­ing and in­ter­nal poli­ti­cal effi­cacy. Poli­ti­cal Be­hav­ior, 27(1), 49–69.

Mutz, D. C. (2002a). Cross-Cut­ting So­cial Net­works: Test­ing Demo­cratic The­ory in Prac­tice. The Amer­i­can Poli­ti­cal Science Re­view, 96(1), 111–126.

Mutz, D. C. (2002b). The Con­se­quences of Cross-Cut­ting Net­works for Poli­ti­cal Par­ti­ci­pa­tion. Amer­i­can Jour­nal of Poli­ti­cal Science, 46(4), 838–855.

Mutz, D. C. (2006). Hear­ing the Other Side: De­liber­a­tive ver­sus Par­ti­ci­pa­tory Democ­racy. Cam­bridge: Cam­bridge Univer­sity Press.

Mutz, D. C., & Martin, P. S. (2001). Fa­cil­i­tat­ing Com­mu­ni­ca­tion across Lines of Poli­ti­cal Differ­ence: The Role of Mass Me­dia. The Amer­i­can Poli­ti­cal Science Re­view, 95(1), 97–114.

Nemeth, C. J. (1986). Differ­en­tial con­tri­bu­tions of ma­jor­ity and minor­ity in­fluence. Psy­cholog­i­cal Re­view, 93(1), 23–32.

Nie­meyer, S., & Jennstål, J. (2016). The De­liber­a­tive Demo­cratic In­clu­sion of Fu­ture Gen­er­a­tions. In I. González-Ri­coy & A. Gosseries (Eds.), In­sti­tu­tions For Fu­ture Gen­er­a­tions (pp. 247–265).

Read, R. (2012). Guardians of the Fu­ture: A Con­sti­tu­tional Case for Rep­re­sent­ing and Pro­tect­ing Fu­ture Peo­ple. Wey­mouth: Green House.

Ryfe, D. M. (2005). Does De­liber­a­tive Democ­racy Work? An­nual Re­view of Poli­ti­cal Science, 8(1), 49–71.

San­ders, D. (2012). The Effects of De­liber­a­tive Pol­ling in an EU-wide Ex­per­i­ment: Five Mechanisms in Search of an Ex­pla­na­tion. Bri­tish Jour­nal of Poli­ti­cal Science, 42(3), 617–640.

Sclove, R. (n.d.). Town Meet­ings on Tech­nol­ogy.

Seitz, R., & Votta, F. (2018). De­liber­a­tion Across the World—A Cross-Na­tional Ex­am­i­na­tion of the Link Between De­liber­a­tion and Regime Le­gi­t­i­macy. Con­fer­ence Paper.

Setälä, Maija, Grön­lund, K., & Herne, K. (2010). Ci­ti­zen De­liber­a­tion on Nu­clear Power: A Com­par­i­son of Two De­ci­sion-Mak­ing Meth­ods. Poli­ti­cal Stud­ies, 58(4), 688–714.

Setälä, M., & Herne, K. (2014). Nor­ma­tive the­ory and ex­per­i­men­tal re­search in the study of de­liber­a­tive mini-pub­lics. In K. Grön­lund, A. Bächtiger, & M. Setälä (Eds.), De­liber­a­tive MiniPublics. Colch­ester: ECPR Press.

Shahrier, S., Kotani, K., & Saijo, T. (2017). In­ter­gen­er­a­tional sus­tain­abil­ity dilemma and the de­gree of cap­i­tal­ism in so­cieties: A field ex­per­i­ment. Sus­tain­abil­ity Science, 12, 957–967.

Shapiro, I. (2017). Col­lu­sion in Res­traint of Democ­racy: Against Poli­ti­cal De­liber­a­tion. Daedalus, 146(3).

Singh, A. S. P. (2018). Democ­racy and Trade Liber­al­i­sa­tion. Va­ri­eties of Democ­racy In­sti­tute: Users’ Work­ing Paper No. 17.

Siu, A. (2008). Look who’s talk­ing: Ex­am­in­ing so­cial in­fluence, opinion change, and ar­gu­ment qual­ity in de­liber­a­tion.

Siu, A. (2017). De­liber­a­tion & the Challenge of Inequal­ity. Jour­nal of the Amer­i­can Academy of Arts & Sciences, 146(3).

Smith, G. (2005). Beyond the bal­lot: 57 demo­cratic in­no­va­tions from around the world [Pro­ject re­port].

Smith, Gra­ham, Richards, R. C., & Gastil, J. (2015). The Po­ten­tial of Par­tici­pedia as a Crowd­sourc­ing Tool for Com­par­a­tive Anal­y­sis of Demo­cratic In­no­va­tions. Policy & In­ter­net, 7(2), 243–262.

Steen­ber­gen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Mea­sur­ing Poli­ti­cal De­liber­a­tion: A Dis­course Qual­ity In­dex. Com­par­a­tive Euro­pean Poli­tics, 1, 21–48.

Steiner, J., Bächtiger, an­dre, Sporndli, M., & Steen­ber­gen, M. (2004). De­liber­a­tive Poli­tics in Ac­tion: An­a­lyz­ing Par­li­a­men­tary Dis­course.

Steiner, J., Jaramillo, M. C., Maia, R. C. M., & Mameli, S. (2017). De­liber­a­tion across Deeply Di­vided So­cieties: Trans­for­ma­tive Mo­ments. Cam­bridge, United King­dom ; New York, NY: Cam­bridge Univer­sity Press.

Suiter, J., Far­rell, D. M., & O’Malley, E. (2016). When do de­liber­a­tive cit­i­zens change their opinions? Ev­i­dence from the Ir­ish Ci­ti­zens’ Assem­bly. In­ter­na­tional Poli­ti­cal Science Re­view, 37(2), 198–212.

Sun­stein, C. R. (2002). The Law of Group Po­lariza­tion. Jour­nal of Poli­ti­cal Philos­o­phy, 10(2), 175–195.

Sun­stein, C. R. (2011). Go­ing to Ex­tremes: How Like Minds Unite and Divide (Reprint edi­tion). Oxford; Madrid: Oxford Univer­sity Press.

Taber, C. S., Lodge, M., & Glathar, J. (2001). The mo­ti­vated con­struc­tion of poli­ti­cal judg­ments. In Cam­bridge Stud­ies in Poli­ti­cal Psy­chol­ogy and Public Opinion. Ci­ti­zens and poli­tics: Per­spec­tives from poli­ti­cal psy­chol­ogy (pp. 198–226).

Tännsjö, T. (2007). Fu­ture Peo­ple, the All Affected Prin­ci­ple, and the Limits of the Ag­gre­ga­tion Model of Democ­racy 1 In J. Josefs­son D. Egon­s­son (ed.), Hom­mage à Wlodek. Philo­soph­i­cal Papers Ded­i­cated to Wlodek Rabinow­icz.

Tet­lock, P. E. (1983). Ac­countabil­ity and the Per­se­ver­ance of First Im­pres­sions. So­cial Psy­chol­ogy Quar­terly, 46(4), 285–292.

Tet­lock, P. E. (1985). Ac­countabil­ity: A So­cial Check on the Fun­da­men­tal At­tri­bu­tion Er­ror. So­cial Psy­chol­ogy Quar­terly, 48(3), 227–236.

Thaler, R. H., & Sun­stein, C. R. (2008). Nudge: Im­prov­ing de­ci­sions about health, wealth, and hap­piness. New Haven, CT, US: Yale Univer­sity Press

Whit­tle­stone, J. (2017a). Im­prov­ing in­sti­tu­tional de­ci­sion-mak­ing—80,000 Hours. 80,000 Hours.

Whit­tle­stone, J. (2017b). Two ways of ‘im­prov­ing de­ci­sion-mak­ing’—Jess Whit­tle­stone.


  1. 80,000 Hours ranks Im­prov­ing In­sti­tu­tional De­ci­sion-Mak­ing as 4th in its list of global is­sues and 35% of re­spon­dents in the 2018 EA Sur­vey ranked im­prov­ing ra­tio­nal­ity and de­ci­sion-mak­ing as a top or near top pri­or­ity. An un­clear but sig­nifi­cant num­ber (~147) pri­ori­tised Poli­tics in the 2017 EA Sur­vey and while not listed in the 2018 EA Sur­vey it was one of the largest open com­ment cat­e­gories. ↩︎

  2. 39% of re­spon­dents in the 2018 EA sur­vey ranked An­i­mal Welfare as a top or near-top pri­or­ity, and the group­ing of “Long-Term Fu­ture” causes was the sec­ond most pop­u­lar cause area af­ter Global Poverty ↩︎

  3. At 2019’s EA Global Lon­don, Ma­hen­dra Prasad dis­cussed how mul­ti­a­gent sys­tems could be used to ad­dress the pos­si­ble prob­lem de­liber­a­tion pro­cesses de­volv­ing into a mob men­tal­ity and group­think. ↩︎

  4. The most ex­pan­sive defi­ni­tion comes from Fung (2003), while the most re­stric­tive is em­ployed in the works of Fishkin cited through­out this piece. ↩︎

  5. Thanks to com­menters for em­pha­sis­ing that this point needed to be made. ↩︎

  6. Thanks to Ste­fan_Schu­bert for point­ing out this piece of liter­a­ture ↩︎

  7. In a num­ber of ex­am­ples, par­ti­ci­pants were asked about their agree­ment with the state­ment ‘it is un­fair that we are go­ing to leave the cli­mate in a mess for fu­ture gen­er­a­tions’, but pre- and post- de­liber­a­tion there was not a sig­nifi­cant differ­ence in re­sponses. ↩︎

  8. The Welfare Qual­ity (WQ) pro­ject was an EU-funded ex­er­cise which sought to as­cer­tain so­cietal views in draw­ing up a pro­to­col for as­sess­ing an­i­mal welfare on farms and at slaugh­ter plants This was achieved through the cre­ation of al­most 50 fo­cus group dis­cus­sions in a va­ri­ety of EU mem­ber states, and cit­i­zens’ ju­ries in Italy, the UK and Nor­way. The lat­ter con­sisted of 10-12 peo­ple rep­re­sent­ing differ­ent sides of the de­bate (veg­e­tar­i­ans, con­sumers on a bud­get, health con­scious con­sumers, en­vi­ron­men­tally aware con­sumers, ha­lal or kosher eaters, ‘main­stream’ con­sumers and so on) who met on a weekly ba­sis for a num­ber of ses­sions last­ing for two hours. The ses­sions in­cluded one where ex­perts pre­sented three al­ter­na­tive eth­i­cal po­si­tions con­cern­ing hu­man-an­i­mal re­la­tions; an an­i­mal rights per­spec­tive, an an­i­mal welfare per­spec­tive and a more in­stru­men­tal view of hu­man/​an­i­mal re­la­tions. ↩︎

  9. The pro­cess of graft­ing or trans­plant­ing or­gans or tis­sues be­tween mem­bers of differ­ent species. ↩︎

  10. “21st Cen­tury Nan­otech­nol­ogy Re­search and Devel­op­ment Act of 2003,” 15 USC 7501, sec. 2 (b)(10), http://​​fr­we­b­­​​cgi-bin/​​get­doc.cgi?db­name=108_cong_pub­lic_laws&do­cid=f:publ153.108 ↩︎