Wireheading as a Possible Contributor to Civilizational Decline

Epistemic sta­tus: Had these ideas in my mind for years, did my best to pre­sent them in de­tails, need feed­back to make the anal­y­sis bet­ter.

Ab­stract: Ad­vances in new tech­nolo­gies cre­ate new ways to stim­u­late the plea­sure cen­ter of the hu­man brain via new chem­i­cals, di­rect ap­pli­ca­tion of elec­tric­ity, elec­tro­mag­netic fields, “re­ward hack­ing” in games and so­cial net­works, and, in the fu­ture, pos­si­bly via ge­netic ma­nipu­la­tion, nanorobots and AI sys­tems. This may have two con­se­quences: a) hu­man life may be­come more in­ter­est­ing, b) hu­mans may stop par­ti­ci­pat­ing in any ex­ter­nal ac­tivi­ties, in­clud­ing work, main­te­nance, re­pro­duc­tion, and even car­ing for their own health, which could slowly con­tribute to a de­cline in hu­man pop­u­la­tions. As wire­head­ing is not self-limit­ing, it may push the ad­dict to his/​her limits and ul­ti­mately re­sult in life-threat­en­ing be­hav­ior. Mass un­em­ploy­ment and the pro­vi­sion of ba­sic in­come would likely feed the in­dus­try of plea­sures. DIY syn­thetic biol­ogy and com­put­ers may cre­ate the pos­si­bil­ity of home­made very ad­dic­tive drugs through var­i­ous routes, like in­ser­tion of a mor­phine-pro­duc­ing gene in plants or stim­u­la­tion of the brain via elec­tro­mag­netic fields, which may not be pos­si­ble to con­trol via limit­ing the sup­ply chain. If “wire­head­ing knowl­edge” be­comes freely available, it could cre­ate a re­ward-hack­ing epi­demic larger than the cur­rent U.S. opi­oid crisis and pos­si­bly con­tribute to a global civ­i­liza­tional de­cline.

Key points:

· Tech­nolog­i­cal progress in­creases the num­ber of ways, in­ten­sity and availa­bil­ity of the ways of brain stim­u­la­tion.

· DIY brain stim­u­la­tion meth­ods could be dis­sem­i­nated as pure knowl­edge.

· Un­limited hu­man re­ward cen­ter stim­u­la­tion could cause death or sig­nifi­cant im­pair­ments in many ways.

· Re­ward cen­ter stim­u­la­tion tech­nol­ogy alone is very un­likely to cause hu­man ex­tinc­tion di­rectly.

· Re­ward cen­ter stim­u­la­tion tech­nol­ogy could con­tribute to pos­si­ble civ­i­liza­tional col­lapse via low­er­ing fer­til­ity, over­dose mor­tal­ity, mor­tal­ity from lack of self-care, etc, – and this could even­tu­ally re­sult in ex­tinc­tion.

Writ­ten by Alexey Turchin + 1 coauthor

Link on the pdf: https://​​philpa­pers.org/​​rec/​​TURWAA

1 Introduction

Mu­sic video by Ar­gonov as epi­graph (in Rus­sian)

Plea­sure is the re­ward func­tion cre­ated by pow­er­ful evolu­tion­ary pres­sure to mo­ti­vate hu­man minds do what is dic­tated by in­nate drives, like sex, sur­vival, eat­ing and even growth of so­cial sta­tus (Sim­ler and Han­son 2017). This re­ward func­tion is effec­tively pro­tected from self-hack­ing in an­i­mals liv­ing in na­ture, per­haps be­cause species ca­pa­ble of such self-stim­u­la­tion be­come ex­tinct. Hu­man civ­i­liza­tion cre­ated many new ways to stim­u­late plea­sure cen­ters, in­clud­ing art, pro­tected sex, par­ty­ing, etc. How­ever, some of the ways to hack the re­ward func­tion are so effec­tive that they effec­tively in­ca­pac­i­tate the in­di­vi­d­ual and cre­ate per­sis­tent so­cial prob­lems (e.g. al­co­holism). More­over, as plea­sure seek­ing is one of the main mo­tives of hu­man be­hav­ior, hu­mans are very cre­ative in in­vent­ing new ways to stim­u­late plea­sure cen­ters.

Here we in­ves­ti­gate a new global risk which has been barely men­tioned in the aca­demic liter­a­ture, with some forays in fic­tion. Ad­vances in new tech­nolo­gies cre­ate the pos­si­bil­ity of af­fect­ing re­ward cen­ters of the brains in new ways which could be even stronger and could be dis­sem­i­nated as pure knowl­edge. For ex­am­ple, the idea that a cheap, su­per-ad­dic­tive home­made drug could ap­pear was ex­plored in the Stru­gatskys’ novel “The fi­nal cir­cle of par­adise” (Stru­gatsky and Stru­gatsky 1976). It is de­scribed as an elec­tronic drug which cre­ates such pleas­ant and re­al­is­tic hal­lu­ci­na­tions that users do not want to re­turn to re­al­ity and die of ex­haus­tion; the drug is very sim­ple and could be built from an or­di­nary ra­dio. The same prob­lem is ad­dressed in Ar­gonov’s techno-opera “Beyond the line”, which de­scribed crys­tals of ar­tifi­cial “love” (similar to 3,4-Methyl​ene­dioxy​metham­phetamine, MDMA) which could se­duce hu­man­ity to stop space ex­plo­ra­tion and in­stead choose in­finite plea­sures now (Ar­gonov 2015) and in the StarTrek epi­sode “The Game” (Allen 1991).

The term “wire­head­ing” ap­peared as a short de­scrip­tion of the deep brain stim­u­la­tion in rat ex­per­i­ments (Olds and Milner 1954). The rats were able to self-stim­u­late by press­ing a pedal and they even­tu­ally died from ex­haus­tion (in many cases, the ex­per­i­ments were stopped be­fore they died for anatomic study, and we will look later at how differ­ent causes of death are dis­tributed for drugs self-ad­minis­trated by an­i­mals).

While wire­head­ing is of­ten used as a scary tale, many util­i­tar­ian philoso­phers have writ­ten pos­i­tively about it, and the risks of global catas­trophic it poses is un­der­ex­plored. J. Tay­lor started her ar­ti­cle about wire­head­ing with a quote from Dalai Lama about elec­trodes: “If it was pos­si­ble to be­come free of nega­tive emo­tions by a riskless im­ple­men­ta­tion of an elec­trode – with­out im­pairing in­tel­li­gence and the crit­i­cal mind – I would be the first pa­tient” (Tay­lor 2012). She then ar­gues for cheap and safe wire­head­ing; she claims it will in­crease com­pas­sion and mo­ti­va­tion and cites ex­per­i­ments where de­pressed peo­ple self-stim­u­lated only when they were de­pressed. Similarly, To­ma­sic wrote that hu­man­ity as whole is mov­ing in the di­rec­tion of global wire­head­ing (To­masik 2015). The whole site wire­head­ing.com is de­voted to pos­si­ble “pos­i­tive wire­head­ing” in the form of par­adise en­g­ineer­ing.

Only a few ac­ci­den­tal hu­man wire­head­ing cases are known to date; how­ever, they are not ex­am­ples of “full-blown” re­ward hack­ing, as only some part of re­ward cen­ters was stim­u­lated (Portenoy et al. 1986). Wire­head­ing is a prob­lem not only for hu­mans, but also for AIs which use re­ward func­tions as was de­scribed by Yam­ploskiy (2014). Ex­per­i­ments on rats have shown that a rat will con­stantly press a lever to get the next drip of co­caine un­til the point of star­va­tion and even death (Olds and Milner 1954).

Some sci­en­tists have looked at past re­ward-hack­ing tech­nolo­gies as a cause of so­cietal crises (Khal­tu­rina and Koro­taev 2008). In this ar­ti­cle we ex­plore if the trend to more and more effec­tive hu­man wire­head­ing via quick ad­vances of sev­eral tech­nolo­gies could re­sult in civ­i­liza­tional scale con­se­quences, like so­cial col­lapse, global catas­tro­phe, or even hu­man ex­tinc­tion.

In sec­tion 2 we provide an overview of known ex­am­ples of catas­trophic wire­head­ing and the cur­rent meth­ods to stim­u­late the hu­man brain. In sec­tion 3 we look more deeply into the na­ture of the hu­man re­ward cen­ter and those tech­nolo­gies which cur­rently ex­ist or could ap­pear for its stim­u­la­tion. Sec­tion 4 builds a bridge be­tween wire­head­ing and global catas­trophic risks by ex­plor­ing the ways in which wire­head­ing could kill and spread to be­come a global prob­lem. In sec­tion 5, the global risk of wire­head­ing epi­demics are placed in the wider per­spec­tive of ac­cel­er­at­ing tech­nolog­i­cal progress, tech­nolog­i­cal un­em­ploy­ment, and other catas­trophic risks.

2 His­tor­i­cal ex­am­ples of so­cietal prob­lems aris­ing from mass addictions

2.1 Con­nec­tion be­tween differ­ent ways of re­ward hack­ing and so­cial problems

Ad­dic­tions have cre­ated per­sis­tent so­cial prob­lems through­out his­tory but have never reached a globally catas­trophic level.

Al­co­hol ad­dic­tion is known to be a wide­spread so­cial prob­lem in many coun­tries. In Rus­sia, it was shown to be re­spon­si­ble for su­per-mor­tal­ity in the sec­ond half of the 20th cen­tury (Khal­tu­rina and Koro­taev 2008). Al­co­hol hit peo­ple of east­ern Asian ori­gin and na­tive peo­ple, who do not have an evolu­tion­ary adap­tion to al­co­hol in the form of en­zymes, es­pe­cially hard. High al­co­hol con­sump­tion (above 100 g a week) has re­duced the av­er­age life ex­pec­tancy by five years, ac­cord­ing to re­cent meta-anal­y­sis (Wood et al. 2018).

Al­co­hol ad­dic­tion is slow but can kill in sev­eral ways: di­rect dam­age to health, crimes com­mit­ted un­der the effect of al­co­hol, al­co­hol-re­lated ac­ci­dents, and fe­tal dam­age dur­ing preg­nancy. The most dev­as­tat­ing effects re­sulted from the in­tro­duc­tion of al­co­hol to Sibe­rian and Amer­i­can na­tive pop­u­la­tions, which suffered a sig­nifi­cant de­cline. But even in Rus­sia, al­co­holism is cur­rently in de­cline as peo­ple find bet­ter al­ter­na­tives like so­cial nets and games (Neyaskin 2018).

Smok­ing has sig­nifi­cantly con­tributed to shorter life ex­pec­tancy in the con­tem­po­rary world. Some es­ti­mates sug­gest that it re­duces life ex­pec­tancy by an av­er­age of 10 years (CDC 2018). Ni­co­tine is ex­tremely ad­dic­tive, as it po­ten­ti­ates learn­ing, and the brain learns to as­so­ci­ate plea­sure with cigarettes. Ni­co­tine it­self is not as toxic as some other chem­i­cals in to­bacco smoke.

Overeat­ing and the cor­re­spond­ing obe­sity epi­demic is also con­nected with re­ward self-stim­u­la­tion via eat­ing foods high in sugar and fat.

Opi­ate ad­dic­tion con­tributed to a so­cial crisis in 19th cen­tury China. Cur­rently in the U.S., 49 000 peo­ple in 2017 died be­cause of opi­ate over­doses and 72 000 from all other drugs (Na­tional In­sti­tute on Drug Abuse 2018), the other drugs mak­ing up around 2.6 per cent of to­tal mor­tal­ity of 2 712 630 in 2015 (Na­tional Cen­ter for Health Statis­tics 2018). The cur­rent opi­oid crisis in the U.S. rep­re­sents an ex­po­nen­tial growth in the num­ber of over­dose deaths due to heroin and syn­thetic opi­oids, from thou­sands in the 2000s to tens of thou­sands in the 2010s. Its causes are com­plex, but one cause is con­nected with tech­nolog­i­cal ad­vances in the pro­duc­tion of more and more effec­tive opi­oid re­cep­tor stim­u­la­tors. The illicit dilu­tion of heroin by the more-po­tent fen­tanyl has in­creased the prob­a­bil­ity of over­doses.

HIV takes ad­van­tage of hu­man ad­dic­tion for trans­mis­sion: ad­dic­tion to heroin (via in­jec­tions) and some would ar­gue to sex. In South Africa the HIV epi­demic re­duced life ex­pec­tancy at peak for 10 years, but the situ­a­tion re­cently im­proved with anti-retro­virus ther­apy (Suthar and Bärnighausen 2017).

Gam­ing ad­dic­tion: some peo­ple have played com­puter games for days while ig­nor­ing their bod­ily needs and died (Reuters 2007). Others be­come ad­dicted to gam­bling. Cur­rent gam­ing can­not be catas­trophic as mil­lions of peo­ple play but only a few have nega­tive con­se­quences like wors­en­ing health and obe­sity be­cause of sed­i­men­tary life-style and de-so­cial­iza­tion. The im­por­tant point here is that a care­fully de­signed re­ward may not need a phys­i­cal brain in­ter­ven­tion to cre­ate in­tense plea­sure, per­haps if it cre­ates some men­tal pro­gram, typ­i­cally around a nu­mer­i­cal score.

So­cial net­work ad­dic­tion is based around scor­ing feed­back in the form of “likes” (An­dreassen 2015). Some peo­ple have un­der­taken dan­ger­ous ac­tivity to get great shots for In­sta­gram and are kil­led, e.g. by fal­ling from heights or be­ing mauled or tram­pled by an­i­mals (Po­li­an­skaia 2018). Other peo­ple who get a small num­ber of likes be­come de­pressed and need to com­pen­sate by other plea­sure-seek­ing ac­tivity or risk suicide (Oberst et al. 2017).

Ex­treme sport and dan­ger­ous ac­tivity. Some peo­ple stim­u­late their plea­sure cen­ters by de­liber­ately tak­ing risks. Ex­am­ples in­clude risky driv­ing, ex­treme sports, par­ti­ci­pa­tion in crim­i­nal ac­tivity, and even rol­ler coast­ers. Un­safe driv­ing is prob­a­bly the dead­liest of the “adrenal­ine” ad­dic­tions.

Hack­ing so­cial re­ward as crime. Most crim­i­nal ac­tivi­ties are at­tempts to hi­jack the so­cially ac­cepted re­ward func­tion: steal­ing is about money, which is a re­ward in a cor­rectly func­tion­ing econ­omy, rape is about hi­jack­ing con­sen­sual sex, and drug-re­lated crimes are ob­vi­ously con­nected to ei­ther money or hi­jack­ing of the re­ward func­tion.

Some ad­dic­tions may be benefi­cial (Miller 2018), like those to sport, ca­reer, or mak­ing money. How­ever, any ad­dic­tion tends to push the ad­dict to a limit at which it be­comes detri­men­tal be­cause of the grow­ing tol­er­ance or slip­pery slope to reach higher plea­sures.

In gen­eral, so­ciety is able to cope with dan­ger­ous ad­dic­tions, as they ei­ther progress slowly, and so al­low time for coun­ter­mea­sures to be de­vel­oped, or they af­fect only a small sub­pop­u­la­tion, the col­lapse of which is not dam­ag­ing to civ­i­liza­tion as a whole.

2.2 His­tor­i­cal growth of the num­ber and effec­tive­ness of meth­ods for the re­ward cen­ter stimulation

Dur­ing hu­man his­tory, there has been a slow growth of meth­ods to hack the hu­man re­ward func­tion. In an­cient times, only a few ways to get plea­sures with­out fulfilling biolog­i­cal func­tions were known, in­clud­ing al­co­hol, opium, overeat­ing, non-re­pro­duc­tive sex, and gam­bling. How­ever, it is in­ter­est­ing to note that biolog­i­cally use­less plea­sures be­come im­por­tant pillars of the so­cial struc­tures, like col­lec­tive drink­ing, and can sup­port the econ­omy (e.g. wine pro­duc­tion). The dis­cov­ery of the New World al­lowed ex­change not only of in­fec­tious dis­eases but also of new ways to get plea­sure, the ad­vent of nico­tine and co­caine in the West.

Ad­vances in chem­istry in the 20th cen­tury brought us heroin, LSD, and many new syn­thetic drugs, in­clud­ing an­tide­pres­sants. Brain re­search pro­duced elec­tric wire­head­ing. At the be­gin­ning of the 21st cen­tury, thou­sands of psy­choac­tive drugs were known and it be­came pos­si­ble to pre­dict—to some ex­tent—the prop­er­ties of new chem­i­cals, which cre­ated the phe­nomenon of “de­signer drugs”. In this phe­nomenon, known drugs are changed to es­cape le­gal bar­ri­ers or for cheaper man­u­fac­tur­ing. More po­tent drugs provide more in­come to drug car­tels as they earn more money from each kilo­gram of man­u­fac­tured drug; smaller quan­tities are also sim­pler to ille­gally trans­port (Frank and Pol­lack 2017).

In gen­eral, the law of techno-hu­man­i­tar­ian bal­ance (Nazare­tian 2004) is ap­pli­ca­ble not only to mil­i­tary pro­duc­tion, but to new “re­ward hack­ing” ways to get plea­sures: so­ciety is able to adapt ei­ther by ban­ning these new in­stru­ments or by in­clud­ing lighter forms of them within the so­cial struc­ture. But will it be pos­si­ble to con­tinue to sup­port such a bal­ance in a situ­a­tion of very rapid tech­nolog­i­cal progress? Any­thing we cre­ate to in­ter­act with the brain or to im­prove mood and mo­ti­va­tion could be con­verted into a re­ward stim­u­la­tion sys­tem.

3 New ways of stim­u­lat­ing the plea­sure cen­ter in the brain

We had two bags of grass, sev­enty-five pel­lets of mescal­ine, five sheets of high pow­ered blot­ter acid, a salt shaker half full of co­caine, and a whole galaxy of multi-col­ored up­pers, down­ers, scream­ers, laugh­ers… and also a quart of tequila, a quart of rum, a case of Bud­weiser, a pint of raw ether and two dozen amyls.
Not that we needed all that for the trip, but once you get locked into a se­ri­ous drug col­lec­tion, the ten­dency is to push it as far as you can.

“Fear and Loathing in Las Ve­gas”, (Gilliam 1998)

3.1 Overview of the hu­man re­ward sys­tem and its elec­tri­cal stim­u­la­tion

The hu­man re­ward sys­tem con­sists of sev­eral in­ter­con­nected nodes. If one node is stim­u­lated, oth­ers fire too (Yager et al. 2015). In­tracra­nial self-stim­u­la­tion in rats is pos­si­ble via elec­trodes im­planted in the me­dial fore­brain bun­dle (Car­lezon Jr and Chartoff 2007), as it re­quires less cur­rent than other parts of re­ward sys­tem.

It was found that hu­mans have two in­ter­con­nected sys­tems, roughly cor­re­spond­ing to “want­ing” and “lik­ing”. The fact of heroin ad­dic­tion in rats and hu­mans is ev­i­dence that elec­tric self-stim­u­la­tion of the plea­sure cen­ter is also pos­si­ble in hu­mans.

One hu­man case study draws a bleak pic­ture of a woman who be­came ad­dicted to elec­tri­cal self-stim­u­la­tion, which was not a full re­ward stim­u­la­tion, but only par­tial or­gas­mic stim­u­la­tion. She had in­tractable pain in her ver­te­brae and so had sev­eral deep brain-stim­u­lat­ing elec­trodes in­serted in her brain.

Soon af­ter in­ser­tion of the nVPL elec­trode, the pa­tient noted that stim­u­la­tion also pro­duced erotic sen­sa­tions. This plea­surable re­sponse was height­ened by con­tin­u­ous stim­u­la­tion at 75% max­i­mal am­pli­tude, fre­quently aug­mented by short bursts at max­i­mal am­pli­tude. Though sex­ual arousal was promi­nent, no or­gasm oc­curred with these brief in­creases in stim­u­la­tion in­ten­sity. De­spite sev­eral epi­sodes of parox­ys­mal atria tachy­car­dia and the de­vel­op­ment of ad­verse be­hav­ioral and neu­rolog­i­cal symp­toms dur­ing max­i­mal stim­u­la­tion, com­pul­sive use of the stim­u­la­tor de­vel­oped. At its most fre­quent, the pa­tient self-stim­u­lated through­out the day, ne­glect­ing per­sonal hy­giene and fam­ily com­mit­ments. A chronic ul­cer­a­tion de­vel­oped at the tip of the finger used to ad­just the am­pli­tude dial and she fre­quently tam­pered with the de­vice in an effort to in­crease the stim­u­la­tion am­pli­tude. At times, she im­plored her fam­ily to limit her ac­cess to the stim­u­la­tor, each time de­mand­ing its re­turn af­ter a short hi­a­tus. Dur­ing the past 2 years, com­pul­sive use has be­come as­so­ci­ated with fre­quent at­tacks of anx­iety, de­per­son­al­iza­tion, pe­ri­ods of psy­chogenic poly­dip­sia, and vir­tu­ally com­plete in­ac­tivity (Portenoy et al. 1986).

Deep brain stim­u­la­tion (DBS) via im­planted elec­trodes deep into hu­man brain was ap­proved by the FDA in 2003 for Park­in­son dis­ease, but not for de­pres­sion, which is still un­der clini­cal tri­als. For de­pres­sion, DBS tar­gets ex­actly the me­dial fore­brain bun­dle which is in­volved in the re­ward and mo­ti­va­tion, and it is not sur­pris­ing that af­fect­ing re­gions close to re­ward cen­ter im­proves mood (Sch­laepfer et al. 2013). DBS is ma­jor surgery; its price and com­plex­ity is pro­hibitive for recre­ational use, with costs in the range of 30 000–50 000 USD (Good­man 2011).

3.2 Differ­ent ways to reach hu­man re­ward cen­ter via new technologies

The num­ber of meth­ods for hu­man wire­head­ing is grow­ing ex­po­nen­tially, and at the same time, the price is de­clin­ing, thanks to ad­vances in neu­ro­biol­ogy, syn­thetic biol­ogy and psy­chol­ogy. In this sec­tion, we an­a­lyze meth­ods of plea­sure cen­ter stim­u­la­tion which could ap­pear in the 21st cen­tury.

Ge­net­i­cally mod­ified plants and bac­te­ria could pro­duce ad­dic­tive and psy­chotropic drugs. Many drugs are rel­a­tively sim­ple molecules which are syn­the­sized by a cell, like co­caine, psilo­cy­bin, and opi­ates, and are en­coded by just a sin­gle gene or group of genes (Docimo et al. 2012). Such a gene could be in­serted in other plants or bac­te­ria, and such tech­nol­ogy is already used for syn­the­sis of many med­i­cal drugs, like in­sulin (Baeshen et al. 2014). In an­other ex­am­ple, a hy­drocodone-pro­duc­ing yeast was cre­ated in 2015 (Galanie et al. 2015).

This ap­proach offers the op­por­tu­nity for crim­i­nals to hide their ac­tivity, as ge­net­i­cally mod­ified plants or yeast will look like or­di­nary strains of the species. How­ever, if such mod­ified plants leak into en­vi­ron­ment, they could be a sig­nifi­cant health haz­ard, as an an­i­mal or a per­son could eat them and over­dose. Peo­ple will likely grow such plants at home, in the gar­den or in the for­est nearby, es­cap­ing the need to pay deal­ers for drugs. It is pos­si­ble that they could con­tinu­ally re­dose and kill them­selves in the pro­cess.

Ad­vances in the un­der­stand­ing of brain chem­istry and ar­chi­tec­ture, as well as in biosyn­the­sis and drug dis­cov­ery, will open the pos­si­bil­ity of new de­signer drugs with differ­ent prop­er­ties. But hu­mans re­act to drugs in­di­vi­d­u­ally, and some could die from the first dose of co­caine (Browne 2011), so start­ing self-ex­per­i­men­ta­tion with differ­ent drugs may end in an even­tual in­di­vi­d­ual ad­verse re­ac­tion.

Or­gas­motron”. One may not even need reach deep into a brain to get plea­sure cen­ter stim­u­la­tion. A de­vice built to cure back pain ac­ci­den­tally cre­ated or­gasms (ABC 2017). Sex­ual self-stim­u­la­tion with as­phyxia (and the “chok­ing game” among teenagers (Lin­klet­ter et al. 2010)) is an ex­am­ple of a “knowl­edge-only” dan­ger­ous ad­dic­tive drug. This kil­led at least 82 peo­ple in US in 1995-2007 (Toblin et al. 2008).

Phys­iolog­i­cal stim­u­la­tion. Warm wa­ter, and some forms of phys­i­cal ac­tivity, like the so-called “run­ner’s high” and kun­dal­ini yoga could in­crease re­ward cen­ter stim­u­la­tion. Con­cen­tra­tion of at­ten­tion on some points of the body’s sur­face is known to change the prac­ti­tioner’s psy­cholog­i­cal state and is used in some forms of Eastern med­i­ta­tion (Amihai and Kozhevnikov 2015).

Drug com­bi­na­tions. Some drug com­bi­na­tions are known to be es­pe­cially dan­ger­ous, like the com­bi­na­tion of co­caine and heroin col­lo­quially known as “speed­balls”. But more sinister is the situ­a­tion when a com­bi­na­tion of some be­nign things be­comes ad­dic­tive, like small money prizes and a ran­dom num­ber gen­er­a­tor com­bin­ing to cre­ate a gam­ing ad­dic­tion.

Au­dio stim­u­la­tion by rel­a­tively con­ven­tional meth­ods, for ex­am­ple, in the form of mu­sic or drums, is part of hu­man cul­tural her­i­tage. New ideas have ap­peared in the field, like bi­nau­ral rhythms (Oster 1973), isochronic tones, brain ma­chines us­ing puls­ing light (e.g. “Kasina”), but most of them pro­duce only limited and tem­po­ral mind-chang­ing effect. Biofeed­back also may be used to train the brain to stim­u­late its re­ward cen­ters.

3.3 Use of elec­tric fields and cur­rents for brain stimulation

Elec­tro­mag­netic fields also offer op­por­tu­ni­ties for brain stim­u­la­tion. There are sev­eral new ways to non-in­va­sively af­fect brain states, in­clud­ing tran­scra­nial di­rect and al­ter­nat­ing cur­rent stim­u­la­tion, tran­scra­nial mag­netic stim­u­la­tion, and in­frared and ul­tra­sound stim­u­la­tion. The re­gions for re­ward cen­ter stim­u­la­tion are deep in­side the brain and thus not eas­ily reach­able. Re­gard­less, the mar­ket for neu­rostim­u­la­tion de­vices is grow­ing and is ex­pected to reach 13 billion USD in 2023 (Global Mar­ket In­sights 2017).

While ex­ter­nal brain stim­u­la­tion is not pre­cise enough to tar­get only the re­ward cen­ter, the use of the math­e­mat­i­cal meth­ods, brain imag­ing and AI may provide much bet­ter tar­get­ing in the fu­ture. The com­plex ac­tivity of many elec­tro­mag­netic coils could be used to cre­ate a field of com­plex-space ge­om­e­try to stim­u­late pre­cisely the de­sired re­gion of the brain us­ing Fourier anal­y­sis to get the needed form of the field. Pierre and Pers­inger have cre­ated a helmet which re­port­edly in­duces re­li­gious vi­sions; it uses some form of tran­scra­nial mag­netic stim­u­la­tion via com­plex set of coils which mimic the effects of tem­po­ral lobe epilepsy (Pierre and Pers­inger 2006).

Lasers are used for brain stim­u­la­tion in the form of low-level light ther­apy (LLLT) (Dis­ner et al. 2016). The in­ter­sec­tion of many in­frared lights from lasers could be able to reach some re­gions in the brain, as the tis­sues of the head in­clud­ing the skull are rel­a­tively trans­par­ent to in­frared light.

Tran­scra­nial di­rect cur­rent stim­u­la­tion (tDCS) de­vices could be as sim­ple as 9V bat­tery with two wires. We now know that they do not cause eu­pho­ria, but there were med­i­cal con­cerns raised at their ini­tial un­veiling. Anec­do­tal re­ports say that mood im­me­di­ately im­proves when tDCS is used for ther­apy in pa­tients with de­pres­sion (Fran­son 2013).

Cra­nial elec­trother­apy stim­u­la­tion (CES) is the use of al­ter­nat­ing cur­rent to mimic some brain fre­quency to change work­ing of the brain. There is less re­search on this ap­proach than on tDCS. It has been sug­gested that it could be effec­tively used to in­duce lu­cid dreams. The Delight Pro is a con­sumer de­vice which com­bines CES with au­dio-video stim­u­la­tion for “en­train­ment” of brain waves (Delight Pro 2018).

Pulsed mag­netic field ther­apy (PEMF) uses a chang­ing mag­netic field to stim­u­late ar­eas deep in­side the brain via in­duced cur­rents. This ap­proach is differ­ent from tran­scra­nial mag­netic stim­u­la­tion (TMS), which uses shorter but more in­tense pulses cre­ated by very pow­er­ful coils to sup­press ac­tivity of a brain re­gion (Groppa et al. 2012).

3.4 Neu­roim­plants and other in­va­sive meth­ods

It seems that the most pow­er­ful meth­ods of stim­u­la­tion are via di­rect elec­tric stim­u­la­tion of brain tis­sue, a fact which has given rise to the name “wire­head­ing”. Chem­i­cal stim­u­la­tion could lose its effi­ciency as tol­er­ance is built. Elec­tric stim­u­la­tion does not have these limi­ta­tions, as it di­rectly stim­u­lates neu­rons.

For­tu­nately, the hu­man re­ward cen­ter is in a pro­tected lo­ca­tion deep within the brain, which is in turn pro­tected by the skull. Must hu­mans can­not reach it with­out com­plex in­va­sive surgery. If a per­son is not yet ad­dicted to brain stim­u­la­tion, s/​he gen­er­ally will not risk brain surgery to ob­tain it. There are only a few ex­am­ples of peo­ple who have tried vol­un­tary brain surgery in non-med­i­cal con­texts, mostly for ex­per­i­men­tal pro­poses, not for stim­u­la­tion. For ex­am­ple, there are non-tra­di­tional med­i­cal pro­ce­dures, like trepa­na­tion for brain en­hance­ment (Cox 2013).

The ad­vanc­ing field of neu­roim­plants will make fu­ture brain surg­eries much sim­pler, safer, and more stan­dard­ized. It is still un­likely that peo­ple new to elec­tric re­ward-cen­ter stim­u­la­tion will seek this type of neu­roim­plants, as they will know that re­ward cen­ter con­stant self-stim­u­la­tion could kill them in a few days. But if they were sure that they would have ac­cess to large sup­ply of food and care, they could pre­fer con­stant wire­head­ing bliss to, say, sim­ple suicide.

How­ever, the situ­a­tion may be differ­ent with neu­roim­plants based on some form of nanobots or smart pow­der, where a per­son could con­trol where they move in his/​her brain. There would be a ten­dency to move the neu­roim­plant closer and closer to neu­ral path­ways con­nected with the re­ward cen­ter, with even­tual wire­head­ing. This can be con­sid­ered an anal­ogy to the way as a per­son may ad­vance from light recre­ational drugs to heavy ones, or in­crease doses of anal­gesics.

Elec­trode ar­rays. DBS via a sin­gle elec­trode is a rather un­so­phis­ti­cated way to get plea­sure, as such stim­u­la­tion does not take into ac­count the com­plex struc­ture of the hu­man re­ward cen­ter and stim­u­lates the whole area. Thus—pre­sum­ably—it can­not reach the max­i­mum level of plea­sure. How­ever, a more pow­er­ful sys­tem might in­clude many smaller elec­trodes in many parts of the brain con­nected to spe­cific groups of neu­rons. This ap­proach may be more difficult: sci­ence do not yet know the struc­tures of these plea­sure cen­ters; smaller elec­trodes de­grade more quickly, los­ing elec­tric con­nec­tions with the brain tis­sue or kil­ling neu­rons around them; and such surgery is also cur­rently difficult.

Neu­ral pros­the­sis. The most com­mon cur­rent im­plants are cochlear im­plants. They are con­nected to nerve tis­sue, not brain tis­sue, have up to 22 elec­trodes (Van Be­souw 2013) and cost up to 100 000 USD. A more ad­vanced neu­ral pros­the­sis has been re­searched for the hip­pocam­pus (Gon­za­lez 2018) as mem­ory im­prove­ment im­plants. In DBS, no di­rect cur­rent is used, but in­stead a com­bi­na­tion of pulses of some fre­quency (like 100 Hz) which is reg­u­lated by a sig­nal pro­ces­sor (Fa­gun­des et al. 2016). For most effec­tive re­ward stim­u­la­tion, a spe­cial digi­tal pro­ces­sor is needed which would cre­ate differ­ent sig­nals for differ­ent parts of the brain. It could be put in­side the brain to­gether with elec­trodes, cre­at­ing some­thing like a re­ward-stim­u­lat­ing im­plant.

Neu­ral dust. This idea is to cre­ate small self-suffi­cient elec­tric ma­chines which could be in­jected in the brain where they would ex­change in­for­ma­tion and recharge wire­lessly. The idea is ex­plored in Musk’s pro­ject Neu­ral­ink (Tem­ple­ton 2017).

Tar­geted chem­i­cals and nanopar­ti­cles. The “ad­van­tage” of chem­i­cals like opi­oids in re­ward cen­ter stim­u­la­tion is that they non-in­va­sively find their way into the brain via the blood stream, cross the blood–brain bar­rier and reach the tar­get cells, which have unique re­cep­tors to which the molecules bind (Mer­rer et al. 2009). Spe­cially de­signed lig­ands or nanopar­ti­cles func­tion­al­ized with such lig­ands could be used in the same man­ner, to per­ma­nently at­tach to re­ward cen­ter cells and then be used as the ba­sis of a sys­tem for its con­stant stim­u­la­tion.

Op­to­ge­net­ics and liv­ing neu­rons as brain im­plants. Some viral vec­tors could be used to de­liver genes (Ramos et al. 2017) that code for light-sen­si­tive pro­teins into spe­cific neu­rons, and such neu­rons could later be stim­u­lated by send­ing in­frared laser light in their di­rec­tion.

Ar­tifi­cial biolog­i­cal neu­rons as im­plants. If hu­man­ity’s un­der­stand­ing of biolog­i­cal tis­sue be­comes suffi­ciently ad­vanced, we could har­ness the im­plant­ing mechanism used dur­ing the early em­bry­onal de­vel­op­ment of the hu­man brain. This is the abil­ity of new neu­rons to travel through the brain, find a spe­cific lo­ca­tion, and from there, send ax­onal con­nec­tions to other brain re­gions. Ax­ons from live neu­rons could grow into re­mote re­gions of the brain and be used as rel­a­tively safe brain im­plants (Ade­wole et al. 2018). This tech­nol­ogy would have im­por­tant ap­pli­ca­tions, like restor­ing brain func­tion, cre­at­ing non-degrad­able brain–com­puter in­ter­faces, and ul­ti­mately, up­load­ing brains, but it could be also used for wire­head­ing.

Nanorobots. Clas­si­cal self-repli­cat­ing nanorobots (Drexler 1986) may be the ul­ti­mate brain hack­ing ma­chines. Molec­u­lar man­u­fac­tur­ing will also en­able pro­duc­tion of all pos­si­ble drugs.

Qualia en­g­ineer­ing. By us­ing differ­ent neu­ral im­plants, tech­nol­ogy could not only stim­u­late the brain cen­ters, but cre­ate new qualia of plea­sures, more in­tense or more vari­able.

3.5 Brain stim­u­la­tion and AI

It seems likely that in the fu­ture, AI will be the main player in the en­ter­tain­ment in­dus­try. AI will ad­vance in com­puter games, vir­tual re­al­ity (VR), movies and pro­duc­tion of other art forms. Ex­ter­nal com­put­ers with some form of AI will be used to con­trol neu­roim­plants and nanobots in brains (Price 2018).

AI op­ti­mizes re­ward in games and so­cial net­works. An in­crease in the hu­man re­ward re­sponse could be seen as an op­ti­miza­tion prob­lem, and AI could find in­di­vi­d­ual ways to in­crease per­ceived re­ward. This might be in much the same way as Face­book’s al­gorithm in­creases peo­ple’s so­cial net­work ad­dic­tion by ma­nipu­lat­ing what they see and en­courag­ing them to cre­ate more lik­able posts. This could be seen as an adap­tive com­puter game, which perfectly stim­u­lates one’s plea­sure cen­ters via in­ter­est­ing plot, neu­ral in­ter­faces and many other sub­tler clues, like sounds, which max­i­mize ad­dic­tion or re­ward.

AI-em­pow­ered VR games which also in­clude brain stim­u­la­tion could put a per­son in a com­plete simu­lated re­al­ity in­clud­ing meet­ing vir­tual sex part­ners or us­ing imag­i­nary recre­ational drugs.

Drug-en­hanced lu­cid dreams and dream record­ing could har­ness the nat­u­ral hu­man abil­ity to by­pass the wall to the plea­sure cen­ter while dream­ing, us­ing some com­bi­na­tion of drugs, e.g. galan­tamine (Spar­row et al. 2018), lu­cidity-in­duc­ing glasses, and brain stim­u­la­tion.

Sex­ual AI-em­pow­ered “sexbots” may de­velop a way to the hu­man re­ward cen­ter via simu­lat­ing some­thing like mul­ti­ple or­gasm, or just de­stroy the hu­man fam­ily and lower fer­til­ity. In some similar sense, own­ing a pet could satisfy a de­sire for chil­dren and thus con­tribute to lower fer­til­ity: the pet is re­ward-hack­ing for de­sire of tak­ing care of some­thing small.

A global “Friendly” AI with faulty pro­gram­ming could wire­head hu­mans, think­ing that or­gas­mium is what we ac­tu­ally want (Turchin and Denken­berger 2018a). The prob­lem is not eas­ily es­capable, as if peo­ple had an AI sys­tem, many would want to ask it to make ev­ery given situ­a­tion bet­ter, and even­tu­ally it may fall into a death spiral of ask­ing for more and more pleas­ant ex­pe­riences.

Hu­man up­loads, or ems (Han­son 2016), will have higher-level ac­cess to their re­ward func­tion and thus be likely to slip into wire­head­ing.

Fu­ture global AI col­lapse. A fu­ture pos­si­ble global AI work­ing as a world gov­ern­ment (Sin­gle­ton AI) could sud­denly wire­head it­self and halt, that is, find the way to by­pass do­ing use­ful work and in­crease its own re­ward to max­i­mum (Yam­polskiy 2014, Turchin and Denken­berger 2018a). In that case, all robotic sys­tems it con­trols will prob­a­bly halt or start de­viant be­hav­ior, in­clud­ing med­i­cal im­plants, home robots and trans­port.

Fu­ture Suffer­ing Risks (S-risks). Knowl­edge of ma­nipu­lat­ing the hu­man re­ward cen­ter cre­ates the tech­nol­ogy to pro­duce in­tense suffer­ing by stim­u­lat­ing the nearby cen­ters of pain. It could be turned into a suffer­ing mode ei­ther ac­ci­den­tally, if a Global AI has some kind of fatal er­ror, like in the plot of (Elli­son 1967), or with the goal of tor­ture for black­mail. Turn­ing off the stim­u­la­tion in an ad­dict will make the per­son crave stim­u­la­tion and suffer from “wire­head­ing with­drawal”. Last but not least, even con­stant wire­head­ing could be re­garded as a form of suffer­ing from the point of view of the other goals (in prefer­en­tial util­i­tar­i­anism) or other parts of the brain, which could have qualia of suffer­ing at the same time, like hunger.

4 Pos­si­ble catas­trophic consequences

4.1 Mechanisms of re­ward stim­u­la­tion and ad­dic­tion lead­ing to death

There are many ways re­ward cen­ter stim­u­la­tion could kill a per­son or dam­age so­ciety. Most of them are per­son­ally but not globally catas­trophic. Ad­di­tional con­di­tions are needed to make an ad­dic­tion a global dan­ger.

Here we first look at the ways in which re­ward cen­ter stim­u­la­tion could kill.

Wire­head­ing to death. Ex­per­i­ments on rats showed that a rat (Olds and Milner 1954) will con­stantly press a lever to get the next stim­u­lat­ing im­pulse. It is of­ten re­ported that rats will die be­cause of star­va­tion but this was not ac­tu­ally mea­sured in Olds and Milner’s ex­per­i­ments, as the rats were ar­tifi­cially kil­led for ex­am­i­na­tion. The only ex­per­i­ment where rats ac­tu­ally starved them­selves is de­scribed by Rout­ten­berg (Rout­ten­berg 1964), and the ex­per­i­menters only mea­sured weight loss, as they needed the rats al­ive at the end of the ex­per­i­ment for anatomic analy­ses (Rout­ten­berg and Lindy 1965).

In an ex­per­i­ment by Jo­han­son et al., half of mon­keys which in­tra­venously self-ad­minis­trated differ­ent drugs died from differ­ent forms of over­dos­ing in 3-5 days, but oth­ers sur­vived 30 days to the end of the ex­per­i­ment and con­tinued to eat to some ex­tent: “For An­i­mal 4029, drug in­take varied over the 29-day ac­cess pe­riod, but did not seem to in­crease with con­tinued ac­cess. Although food in­take was ini­tially sup­pressed, it re­turned to pre­drug lev­els by Day 13. Food con­sump­tion con­tinued to fluc­tu­ate from 50 to 200 g over the next 16 days; how­ever, food and drug in­take were not sys­tem­at­i­cally re­lated” (Jo­han­son et al. 1976). Another an­i­mal likely died of star­va­tion on a stim­u­lant drug: “How­ever, not only was food in­take ini­tially sup­pressed when ac­cess to drug be­gan, but con­sump­tion never re­turned to more than 20 per­cent of pre­drug lev­els” (Jo­han­son et al. 1976). “Rh­e­sus mon­keys given un­limited ac­cess to a va­ri­ety of psy­chomo­tor stim­u­lant drugs self-ad­minister them in amounts suffi­cient to re­sult in death af­ter less than 3 weeks of availa­bil­ity. Most of the an­i­mals who had ac­cess to co­caine, d-am­phetamine, l-am­phetamine and d-metham­phetamine sur­vived less than 5 days in the 23 hr/​day reg­i­men and never more than 15 days ex­cept for one an­i­mal on 1-am­phetamine. Three of the 5 an­i­mals will ac­cess to diethylpro­pion, how­ever, sur­vived for the du­ra­tion of the 28-30 day ex­per­i­ment, de­spite, in 2 of the an­i­mals, high lev­els of daily in­take up to 196 mg/​kg” (Jo­han­son et al. 1976). In a re­view of these and similar stud­ies, Wise con­cluded: “Rats and mon­keys will work similarly com­pul­sively for in­tra­venous stim­u­lants; if given un­limited ac­cess, they will self-ad­minister in­tra­venous in­jec­tions of these drugs to the point of se­vere weight loss and death” (Wise 2002).

In other words, the com­bi­na­tion of star­va­tion and over­dose was deadly for the ma­jor­ity of an­i­mals, but some of them clearly died of over­dose, and oth­ers from star­va­tion.

Over­doses are the main cause of death from opi­ates, largely be­cause this class of drug af­fects the breath­ing con­trol cen­ters within the brain (Maron 2018). Suicides are also very high in heroin users (Darke and Ross 2002). The ille­gal mar­ket is con­stantly chang­ing the con­cen­tra­tion of the ac­tive in­gre­di­ent of the drug, in­creas­ing chance of over­dose. Some drugs are sub­sti­tuted for or mixed with oth­ers with differ­ent effects.

One can imag­ine that pure wire­head­ing will pose less of a risk of over­dose, as it would not af­fect the breath­ing con­trol cen­ter. But al­most any ad­dic­tion has some deadly mode (overeat­ing – heart dis­ease, sex – sex­u­ally trans­mit­ted in­fec­tions (STIs), and so on), the rea­son for which is that ad­dic­tion is a self-ac­cel­er­at­ing pro­cess. There is a slip­pery slope to seek­ing greater and greater plea­sures, and at some point, such a self-ac­cel­er­at­ing pro­cess reaches a point where it af­fects some life-sup­port­ing sys­tem.

Lower life ex­pec­tancy. Many ad­dic­tive drugs and ac­tivi­ties lower life ex­pec­tancy via differ­ent mechanisms. The route of ad­minis­tra­tion may be dam­ag­ing in the case of in­jec­tions. The long-term side-effects of tox­i­c­ity may af­fect health. Un­healthy life habits and self-ne­glect also shorten lifes­pans.

Da­m­age to re­ward func­tion. In­ten­sive stim­u­la­tion of the re­ward func­tion low­ers a per­son’s abil­ity to be stim­u­lated by any­thing else (Garfield et al. 2014), which cre­ates an­he­do­nia ac­com­panied by de­pres­sion, so­cial with­drawal, non-adap­tive be­hav­ior, less self-care and shorter life (Blan­chard et al. 1998) Even try­ing some­thing very re­ward­ing once could cre­ate a new psy­cholog­i­cal “bea­con” in the re­ward func­tion that a per­son would con­sciously or sub­con­sciously pur­sue. Me­mory of an in­tense plea­sure above some thresh­old is, in some sense, similar to post-trau­matic stress di­s­or­der, as it cre­ates com­pul­sive mem­o­ries of the event. The hu­man re­ward func­tion is able to self-adapt by pro­vid­ing differ­ent re­wards for differ­ent ac­tivi­ties that are needed for the or­ganism at a spe­cific mo­ment. For ex­am­ple, if glu­cose is low, it will provide a higher re­ward for eat­ing, and lower the re­ward af­ter get­ting sig­nals from a full stom­ach. But ar­tifi­cial wire­head­ing could dam­age this self-adapt­ing ca­pa­bil­ity of the hu­man re­ward cen­ter, which will dam­age ba­sic phys­iolog­i­cal func­tions as well as com­plex so­cial be­hav­ior.

So­cially dan­ger­ous be­hav­ior. Peo­ple un­der the effect of drugs may act dan­ger­ously and ir­ra­tionally. Most typ­i­cally this be­hav­ior ap­pears un­der the in­fluence of al­co­hol, but it can also oc­cur with am­phetamines and other drugs (e.g. bath salts, PCP). Crav­ings dur­ing with­drawal may cause ad­dicts to com­mit se­ri­ous crimes for small amounts of money. A drug ad­dict be­comes some­thing similar to what Bostrom called “pa­per­clip max­i­mizer” (an AI which max­i­mizes a com­pletely use­less util­ity func­tion (Bostrom 2014)). Crimes are also com­mit­ted by drug-deal­ing gangs.

Pain­ful with­drawal and risk­ing toxic cures like the lethal com­bi­na­tion of some illicit drugs and an­tide­pres­sants (Do­bry et al. 2013) is an­other failure mode.

Lower fer­til­ity and dam­age to chil­dren. Some drugs dam­age in­ter­est in re­pro­duc­tion, re­pro­duc­tion ca­pa­bil­ities, and health of the chil­dren of users (e.g. fe­tal al­co­hol syn­drome) (Wil­son Jones and Thomas Bass 2003).

In­tel­li­gence de­cline. Not only the hu­man value sys­tem is dam­aged by drugs, but also ra­tio­nal­ity and other cog­ni­tive abil­ities (Gold­stein et al. 2009).

Fuel­ing of mal­i­cious bio­hack­ers. Home-made drug pro­duc­tion would fuel DIY bio­hack­ers and cre­ate in­stru­ments for lo­cal­ized biotech pro­duc­tion. More­over, it would fuel wide­spread ac­cess to ge­netic ma­nipu­la­tion tech­nol­ogy, that is, the ca­pa­bil­ity to gen­er­ate cus­tom DNA at home af­ter down­load­ing its code from the in­ter­net. The DNA is in­stalled in some stan­dard car­rier, like E. coli or yeast; this is es­pe­cially likely if the uni­ver­sal bio-syn­the­siz­ers in­vented by Craig Ven­ter be­come pop­u­lar (Boles et al. 2017).

In this sce­nario, a widely available biotech­nol­ogy could give ac­cess to all pos­si­ble ille­gal drugs to each per­son, in the same way in which one could watch ev­ery movie that has ever ex­isted via pirate tor­rents (Mor­ris 2008). Surely, many “bio­hack­ers” would be in­ter­ested in hav­ing such labs, but the same labs could be used to syn­the­size dan­ger­ous tox­ins or even viruses, if, for ex­am­ple, they are re­motely hacked (Turchin et al. 2017). In other words, pro­lifer­a­tion of the ille­gal biosyn­the­sis labs would likely be fueled by de­mand for differ­ent recre­ational drugs, but a large net­work of such labs in­creases the chance that they will be used for toxin or bioweapons pro­duc­tion.

A new risk pro­posed here is that the loss of con­trol of eu­phoric drug-pro­duc­ing bac­te­ria could start a “hap­piness pan­demic”, for ex­am­ple, by in­fect­ing gut micro­biomes. The prob­lem with its con­tain­ment would be that its car­ri­ers will likely be happy to share their dis­ease. The re­sult would be some­thing like a zom­bie-pan­demic. Some viral vec­tors already pro­mote their own dis­sem­i­na­tion in mam­mals (e.g. ra­bies and tox­o­plas­mo­sis) by in­duc­ing be­hav­ioral changes in their hosts (Flegr 2007).

En­vi­ron­men­tal pol­lu­tion by ge­net­i­cally mod­ified or­ganism (GMO) bac­te­ria and plants. Drug-pro­duc­ing home­made GMO bac­te­ria/​fungi/​plants could leak drugs into the en­vi­ron­ment and re­sult in ac­ci­den­tal over­doses in an­i­mals or peo­ple. This is un­likely.

Health risks of “dirty” wire­head­ing. In search of a quick de­pres­sion fix, per­ma­nent ca­pa­bil­ity am­plifi­ca­tion or as a one-time pay­ment for per­ma­nent ac­cess to a high, some peo­ple might pay for ille­gal wire­head­ing. The pro­ce­dure could be rel­a­tively sim­ple and in­ex­pen­sive if done with­out many med­i­cal pre­cau­tions. This “dirty” wire­head­ing could re­sult in even­tual long-term dam­age to neu­ronal path­ways. Elec­trodes tend to de­grade over time; an­other risk is that in­creased elec­tric stim­u­la­tion could even­tu­ally kill neu­rons or cause other types of neu­rode­gen­er­a­tion.

Even the best med­i­cal pro­ce­dure of deep brain stim­u­la­tion in­volves risks: a 3 per cent chance of brain hem­or­rhage (an in­sult which it­self could cause per­ma­nent brain dam­age) and a 3–5 per cent risk of in­fec­tion which would re­quire sec­ondary surgery to re­move the de­vice (Tol­le­son et al. 2014). Dirty wire­head­ing done at non-med­i­cal fa­cil­ities would prob­a­bly have even higher risks of com­pli­ca­tions. Tay­lor men­tioned that too many reg­u­la­tions for safe med­i­cal wire­head­ing would only fuel a black mar­ket (Tay­lor 2012). She sug­gests that the high­est available level of stim­u­la­tion should be legally limited and that some mea­sures like tem­po­rary turn-off should be em­ployed to pre­vent a slip­pery slope to higher lev­els of simu­la­tion. Peo­ple who are already ad­dicted to an opi­oid high may pre­fer a one-time pay­ment and the risk of wire­head­ing to the con­stant risks of drug over­dose.

From this list, it could be con­cluded that while the idea that “wire­head­ing will cause in­evitable death by star­va­tion” is not true, the mul­ti­plic­ity of kil­ling mechanisms of wire­head­ing is large enough to pos­si­bly be­come a global threat.

4.2 Catas­trophic wire­head­ing criteria

As we show above, wire­head­ing could be dan­ger­ous, but alone it is not enough to cause a global catas­trophic risk: for in­stance, most peo­ple do not try hard drugs de­spite know­ing of their ex­is­tence and pos­sess­ing the abil­ity to ob­tain them. Fu­ture ad­vances in neu­rostim­u­la­tion could also pre­sum­ably cre­ate “good wire­head­ing” which may be able to stop suffer­ing with­out cre­at­ing health risks, and which will be dis­cussed in the next sec­tion.

A “su­per-drug” needs an abil­ity to act globally, for the du­ra­tion of a hu­man life-time and af­fect large part of the hu­man pop­u­la­tion to be­come a global catas­trophic risk:

· Based on knowl­edge, not be a ma­te­rial ob­ject, or its ma­te­rial part should be eas­ily ac­cessible. For ex­am­ple, games in­volv­ing sex­ual as­phyxia are based on pure knowl­edge, and this in­for­ma­tion can be dis­sem­i­nated through any in­for­ma­tional chan­nel. More im­por­tantly, they can be dis­sem­i­nated through col­lec­tive prac­tice, like the chok­ing game (Lin­klet­ter et al. 2010). If fu­ture wire­head­ing is just an ad­just­ment of ex­ist­ing com­mer­cial neu­roim­plants, it would also be purely knowl­edge-based. Databases like erowid.com serve as mar­ket­places for the ex­change of new in­for­ma­tion (Roth­stein 2015).

· Sim­ple to use. It is known that availa­bil­ity strongly af­fects the rate of drug self-ad­minis­tra­tion. It is difficult not to eat if you have fridge full of ad­dic­tive food.

· Slip­pery slope. An atyp­i­cal an­tide­pres­sant, tianep­tine, was even­tu­ally banned in Rus­sia as an in­crease in dose cre­ates grad­u­ally in­creas­ing eu­pho­ria (Springer and Cubała 2018). Such a slip­pery slope cre­ates a death spiral seek­ing ever-higher lev­els of stim­u­la­tion.

· Pro­motes its shar­ing. Such a drug has an in­ter­nal mechanism not only of re­ward stim­u­la­tion, but pro­mot­ing its dis­sem­i­na­tion to other peo­ple. For ex­am­ple, an ad­dict is mo­ti­vated to give a first dose of his/​her drug to his/​her friends, so they will also be­come ad­dicts and s/​he will sell them the drug, much like a mul­ti­level mar­ket­ing scheme.

· Non-ob­vi­ous long-term effects. Quick death fol­low­ing wire­head­ing will be ob­vi­ously bad and other peo­ple would choose to avoid wire­head­ing due to its ob­vi­ous risk. Very slow death would also al­low time for so­cial adap­ta­tion and cop­ing mea­sures to de­velop. Some­thing which be­comes wide­spread be­fore its po­ten­tial for dan­ger is rec­og­nized, maybe be­cause of the slip­pery slope, could be a real dan­ger.

· Not bad at first glance. This is a risk which does not trig­ger an im­me­di­ate warn­ing or is not ob­vi­ously bad. Nearly ev­ery­body knows that start­ing heroin is the road to hell, with a high prob­a­bil­ity of death in a few years. But it is not so ob­vi­ous if one just takes the first pill of a pre­scribed opi­ate pain kil­ler, or, say, a new brain stim­u­lant. Some stim­u­lants, like am­phetamines, ini­tially in­crease so­cial adap­ta­tion (Ped­er­sen et al. 2015).

· Su­per-drug over­whelms so­ciety’s abil­ity to cope with and con­trol it. If the dan­ger of some form of wire­head­ing is rec­og­nized, the same drug po­lice (e.g. Drug En­force­ment Ad­minis­tra­tion) as well as ad­ver­tis­ing cam­paigns will be used to stop it. How­ever, if the ease to ob­tain and in­cen­tive to abuse the drug were high, it could over­whelm cop­ing abil­ity, like al­co­hol abuse which con­tinued dur­ing Pro­hi­bi­tion in the U.S.

· Con­nected with an ide­ol­ogy like short-term he­do­nism, nihilism, or some form of he­do­nic util­i­tar­i­anism. Hu­mans could cre­ate sup­port­ing ide­olo­gies to find ra­tio­nale for their ad­dic­tions, like some peo­ple praise mar­ijuana as a uni­ver­sal cure and oth­ers drink be­cause they think that this will clean their blood ves­sels (Grimes 2018). There are philo­soph­i­cal pub­li­ca­tions about the pos­si­ble benefits of wire­head­ing done cor­rectly, e.g. (Tay­lor (2012), (Mu­flax 2011), the site wire­head­ing.com. In con­trast, there are al­most no pub­li­ca­tions about the catas­trophic risk of wire­head­ing in English (though some ex­ist in Rus­sian (Stru­gatsky and Stru­gatsky 1976, hit­the­limit 2008, Ar­gonov 2015)).

The main differ­ence be­tween catas­trophic wire­head­ing tech­nol­ogy and other global catas­trophic risks is that in the case of wire­head­ing, hu­mans would gen­er­ally not be averse to suc­cumb­ing to the dan­ger and could even ac­tively par­ti­ci­pate in its pro­mo­tion.

The sec­ond differ­ence is that in the case of other risks, differ­ent small con­stituents could have cu­mu­la­tive nega­tive effect (like many viral pan­demic si­mul­ta­neously could in­crease prob­a­bil­ity of hu­man ex­tinc­tion (Turchin et al. 2018)). For wire­head­ing, a va­ri­ety of pos­si­ble ar­tifi­cial plea­sures cre­ates some form of pro­tec­tion, be­cause search­ing for differ­ent plea­sures is similar to a typ­i­cal hu­man life; most peo­ple would pre­fer to live a di­verse life and would pre­fer not to suc­cumb to sim­ple but­ton press­ing.

4.3 Pos­i­tive world of eter­nal pleasures

It is pos­si­ble that civ­i­liza­tion could cre­ate a world of al­most im­mor­tal peo­ple with in­definitely many in­ter­est­ing ac­tivi­ties and plea­sures, as de­scribed in Yud­kowsky’s “gen­eral the­ory of fun” (Yud­kowsky 2009). How­ever, if these plea­sures are slightly too strong, the re­sult could be a global wire­head­ing dystopia.

The risks of bad wire­head­ing should not stop hu­man­ity from at­tempts to aug­ment the hu­man mind to es­cape un­bear­able suffer­ing, de­pres­sion, and ag­gres­sion. As Tay­lor men­tioned, some form of mild hy­po­ma­nia may be benefi­cial (Tay­lor 2012). How­ever, But such wire­head­ing should be self-ad­minis­trated un­der con­trol of some pa­ter­nal­is­tic en­tity to lower risks.

Good wire­head­ing is some­thing like brain stim­u­la­tion + nootrop­ics; one ex­am­ple is coffee, which not only im­proves qual­ity of life, but based on re­cent re­search, ex­tends life ex­pec­tancy (Gunter et al. 2017). Good wire­head­ing will prob­a­bly in­clude the in­tel­lec­tual plea­sure of com­plex prob­lem solv­ing, many differ­ent pleas­ant ac­tivi­ties, an in­crease in so­cial adap­ta­tion and com­pas­sion, dom­i­na­tion of long-term plan­ning, and im­prove­ment of brain health. Pos­i­tive wire­head­ing should be good enough but not too good. It is great to be in a good work­ing mood, where you are in the flow and ev­ery task is easy, but if one feels “too good”, one will be able only to perform “trainspot­ting”, that is mind­less star­ing at ob­jects.

Good wire­head­ing should have the right slope of the plea­sure gra­di­ent, that is, sub­jec­tive in­crease of plea­sure with in­crease of a dose. If the plea­sure gra­di­ent is too steeply sloped, one would want even more re­ward de­spite already ex­pe­rienc­ing high re­ward. If the plea­sure gra­di­ent has some­thing like a lo­cal max­i­mum, then some­thing which feels “too good” would trig­ger a fear of los­ing re­ac­tion or other back re­ac­tion, and an in­di­vi­d­ual would self-cal­ibrate to the lo­cal op­ti­mum.

In the Strug­taskys’ novel (Stru­gatsky and Stru­gatsky 1976), “sleg” is not just wire­head­ing in the form of a perfect plea­sure: it is some­thing like a con­trol­led lu­cid dream, so pleas­ant and en­ter­tain­ing that peo­ple may kill oth­ers who in­ter­rupt them. This out­come was also en­vi­sioned in Wen­ders’ movie “Un­til the end of the world,” in which the main char­ac­ter be­comes ad­dicted to record­ings of her own dreams (Wen­ders 1991). A “Sleg”-like ad­dic­tion to vir­tual wor­lds may be similar to a com­puter game, but these wor­lds would be un­like mul­ti­player games that cre­ate a new form of so­cial ac­tivity. Mul­ti­player games can even help peo­ple es­cape dan­ger­ous chem­i­cal ad­dic­tions (Clune 2015).

5 Wire­head­ing in the con­text of other catas­trophic risks and civ­i­liza­tional problems

“… Zhilin,” said Ri­mayer, “his­tory is a story of peo­ple. Every per­son wants to live life with good rea­son, and gives you such a slip … Yes, I know, you think that you live with­out a trace for good rea­son, but con­fess, you have never lived so brightly and hotly as you do in the bath­tub to­day. Are you a lit­tle ashamed to re­mem­ber, would you dare to tell oth­ers about this life? Do not need. They have their own lives, you have your own …”

“… Ri­mayer,” Zhin­lin said, “all this is true. But the past! Space, schools, the fight against the fas­cists, with gang­sters—what, all this in vain? Forty years I have lived in vain? And the oth­ers? Too in vain? ..

“… Zhilin,” said Ri­mayer, “in his­tory, noth­ing is wasted. Some fought and did not live to see the sleg. And you fought and lived …”

Stru­garskys’ “The fi­nal cir­cle of the par­adise”

5.1 Re­la­tion be­tween drug le­gal­iza­tion, tech­nolog­i­cal un­em­ploy­ment and ba­sic income

The cur­rent trend in many Western coun­tries is the le­gal­iza­tion of mar­ijuana, and the next prob­a­ble can­di­dates are psychedelics like psilo­cy­bin and ke­tamine, which are pro­moted as effec­tive treat­ments for de­pres­sion (New­port et al. 2015). Fu­ture civ­i­liza­tion will likely have many un­em­ployed peo­ple sup­ported by some form of ba­sic in­come or welfare. Such peo­ple will not have as much busi­ness to oc­cupy their time and will likely turn to more in­ex­pen­sive re­ward stim­u­la­tion: now they have hob­bies/​sports, watch TV, use the In­ter­net, play com­puter games, and drink, but later they may con­sume newly le­gal­ized drugs.

5.2 Could wide­spread wire­head­ing be an ex­pla­na­tion of the Fermi para­dox?

Wire­head­ing, to­gether with recog­ni­tion of on­tolog­i­cal nihilism (Hit­the­limit 2008), that is, the idea that no goal is in­trin­si­cally bet­ter than any other goal, could pre­sum­ably stop the fur­ther de­vel­op­ment of civ­i­liza­tion. Why fly to the stars, if you could live in simu­la­tions or even just live in in­finite bliss via re­ward func­tion hack­ing?

How­ever, the power of nat­u­ral se­lec­tion would prob­a­bly coun­ter­act such an out­come. While some peo­ple will likely suc­cumb to wire­head­ing, oth­ers may be es­pe­cially re­luc­tant to be re­ward-stim­u­lated. We can see that even now, most peo­ple are just not in­ter­ested in start­ing to take opi­ates, even if they know about them and could ob­tain them. On the other hand, availa­bil­ity strongly in­fluences con­sump­tion, and if press­ing a but­ton could be equal to re­ceiv­ing a re­ward burst, nearly ev­ery­one will some­times have an urge to re­ward them­selves.

Some minds might de­velop a con­trol sys­tem which is less re­ward-based and more rule-based—”do what you ought”—and long-term nat­u­ral se­lec­tion will over­come any new ways of re­ward stim­u­la­tion. The main ques­tion is, will hu­man­ity have enough time for such adap­ta­tion to take place?

We define a “re­ward-stim­u­lat­ing tech­nol­ogy over­hang” as …. In this case, even so­cial col­lapse may not stop the us­age of the high-tech brain stim­u­la­tion meth­ods, for ex­am­ple, if some GMO-drug pro­duc­ing plants sur­vive.

5.3 Pos­si­ble pro­tec­tive measures

Tra­di­tional le­gal re­stric­tions could be used for pun­ish­ment and to cre­ate a stigma as­so­ci­ated with new ways of re­ward-hack­ing. A rad­i­cal new way to elimi­nate risk pro­posed here is a hu­man brain up­grade which pre­vents un­limited wire­head­ing, so hu­mans could ex­plore the world of pos­si­ble plea­sures with­out the risk of ad­dic­tion.

There is a sug­ges­tion that hu­mans are more ro­bust to di­rect wire­head­ing than an­i­mals (https://​​www.wire­head­ing.com/​​), as they are able to stop self-ad­minis­trat­ing co­caine for eat­ing and sleep, while other pri­mates have only limited con­trol (and rats have no con­trol). In other words, hu­mans do have some “rule-based” willpower in the neo­cor­tex that is able to—to some ex­tent—over­ride re­ward sys­tem sig­nals.

Hu­man brains are di­verse, and some peo­ple may have a weaker ten­dency to­ward ad­dic­tion, which, among other fac­tors, de­pends on vari­a­tion in the dopamine re­cep­tors. Many differ­ent types of pos­si­ble ad­dic­tions could cre­ate some form of com­plex ecol­ogy, not much differ­ent from cur­rent so­ciety, where differ­ent at­trac­tions are fight­ing for hu­man at­ten­tion.

Wide­spread wire­head­ing is un­likely to cause hu­man ex­tinc­tion as it is difficult to imag­ine that ev­ery­body will do the same thing at the same time, and that one ac­tivity would af­fect ev­ery­body at the same time. How­ever, wire­head­ing could con­tribute to the col­lapse of a so­ciety and be a part of the more com­plex pro­cess which will re­sult in civ­i­liza­tional col­lapse, which it­self is a global catas­trophic risk (Turchin and Denken­berger 2018b) and could later turn into an ex­tinc­tion event (Denken­berger and Pearce 2017).

Fu­ture tech­nolog­i­cal growth will cre­ate many new ways to stim­u­late hu­man re­ward cen­ters, and if these meth­ods arise too quickly, so­ciety may not have time to cope with the changes and cre­ate ad­e­quate so­cial rit­u­als and norms to canal­ize such ac­tivi­ties into pros­per­ity as has been done in the past. For ex­am­ple, some so­cieties have long tra­di­tions of wine con­sump­tion, like France, where it may even con­tribute to an in­crease in life ex­pec­tancy, while other so­cieties, like North­ern na­tive peo­ples, could be al­most wiped out by vodka.

Conclusion

The speed of de­vel­op­ment of hu­man re­ward-hack­ing tech­nolo­gies de­pends on the speed of progress in AI neu­ro­science, neu­roim­plants, and syn­thetic biol­ogy. Given the cur­rent high speed of progress in these fields, an ex­plo­sion of brain-stim­u­lat­ing tech­nolo­gies could oc­cur in the next 10–30 years. Hu­man­ity will be more and more effec­tive in its re­ward stim­u­la­tion, which will be cheaper and of­ten de­pend only on knowl­edge. This com­bi­na­tion of fac­tors could over­come so­ciety’s cop­ing mechanisms and re­sult in its de­cline. More re­search should be done on pre­vent­ing this de­cline.

Disclaimer

This ar­ti­cle rep­re­sents views of the au­thors. No ex­ter­nal sources of fund­ing were used for this work.

Literature

1. ABC. 2017. Back Pain De­vice Pro­duces Or­gasms—ABC News. ABCnews.

2. Ade­wole, D. O., L. A. Struzyna, J. P. Har­ris, A. D. Nemes, J. C. Bur­rell, D. Petrov, R. H. Kraft, H. I. Chen, M. D. Ser­ruya, J. A. Wolf, and D. K. Cul­len. 2018. Op­ti­cally-Con­trol­led “Liv­ing Elec­trodes” with Long-Pro­ject­ing Axon Tracts for a Sy­nap­tic Brain-Ma­chine In­ter­face. bioRxiv:333526.

3. Amihai, I., and M. Kozhevnikov. 2015. The in­fluence of Bud­dhist med­i­ta­tion tra­di­tions on the au­to­nomic sys­tem and at­ten­tion. BioMed re­search in­ter­na­tional 2015.

4. An­dreassen, C. S. 2015. On­line so­cial net­work site ad­dic­tion: A com­pre­hen­sive re­view. Cur­rent Ad­dic­tion Re­ports 2(2):175–184.

5. Ar­gonov, V. 2015. Re­think­ing progress: be­yond the hori­zon.

6. Baeshen, N. A., M. N. Baeshen, A. Sheikh, R. S. Bora, M. M. M. Ahmed, H. A. I. Ra­madan, K. S. Saini, and E. M. Red­wan. 2014. Cell fac­to­ries for in­sulin pro­duc­tion. Micro­bial Cell Fac­to­ries 13.

7. Boles, K. S., K. Kan­nan, J. Gill, M. Fel­der­man, H. Gou­vis, B. Hubby, K. I. Kam­rud, J. C. Ven­ter, and D. G. Gib­son. 2017. Digi­tal-to-biolog­i­cal con­verter for on-de­mand pro­duc­tion of biolog­ics. Na­ture Biotech­nol­ogy, 35, 672–675 (2017).

8. Bostrom, N. 2014. Su­per­in­tel­li­gence. Oxford Univer­sity Press, Oxford.

9. Browne, R. 2011, April 9. A fine line to an early death. The Syd­ney Morn­ing Her­ald.

10. Car­lezon Jr, W. A., and E. H. Chartoff. 2007. In­tracra­nial self-stim­u­la­tion (ICSS) in ro­dents to study the neu­ro­biol­ogy of mo­ti­va­tion. Na­ture pro­to­cols 2(11):2987.

11. CDC. 2018, May 9. Smok­ing and Tobacco Use; Fact Sheet; Tobacco-Re­lated Mor­tal­ity. http://​​www.cdc.gov/​​to­bacco/​​data_statis­tics/​​fact_sheets/​​health_effects/​​to­bacco_re­lated_mor­tal­ity/​​.

12. Cox, J. 2013, Au­gust 14. The Wo­man Who Drilled a Hole in Her Head to Open Up Her Mind. Vice.

13. Darke, S., and J. Ross. 2002. Suicide among heroin users: rates, risk fac­tors and meth­ods. Ad­dic­tion 97(11):1383–1394.

14. Delight Pro. 2018. Store—Mind Alive. https://​​min­dal­ive.com/​​in­dex.cfm/​​store/​​product.cf?product=6.

15. Dis­ner, S. G., C. G. Beev­ers, and F. Gon­za­lez-Lima. 2016. Tran­scra­nial Laser Stim­u­la­tion as Neu­roen­hance­ment for At­ten­tion Bias Mod­ifi­ca­tion in Adults with Ele­vated De­pres­sion Symp­toms. Brain Stim­u­la­tion 9(5):780–787.

16. Do­bry, Y., T. Rice, and L. Sher. 2013. Ec­stasy use and sero­tonin syn­drome: a ne­glected dan­ger to ado­les­cents and young adults pre­scribed se­lec­tive sero­tonin re­up­take in­hibitors. In­ter­na­tional Jour­nal of Ado­les­cent Medicine and Health 25(3):193–199.

17. Docimo, T., M. Re­ichelt, B. Sch­nei­der, M. Kai, G. Kunert, J. Ger­shen­zon, and J. C. D’Auria. 2012. The first step in the biosyn­the­sis of co­caine in Ery­throx­y­lum coca: the char­ac­ter­i­za­tion of argi­nine and or­nithine de­car­boxy­lases. Plant Molec­u­lar Biol­ogy 78(6):599–615.

18. Drexler. 1986. E.: Eng­ines of Creation. An­chor Press.

19. Fa­gun­des, V. de C., C. R. M. Rieder, A. N. da Cruz, B. C. Be­ber, and M. W. Por­tuguez. 2016. Deep Brain Stim­u­la­tion Fre­quency of the Subtha­la­mic Nu­cleus Affects Phone­mic and Ac­tion Fluency in Park­in­son’s Disease. Park­in­son’s Disease 2016.

20. Flegr, J. 2007. Effects of Tox­o­plasma on Hu­man Be­hav­ior. Schizophre­nia Bul­letin 33(3):757–760.

21. Frank, R. G., and H. A. Pol­lack. 2017. Ad­dress­ing the Fen­tanyl Threat to Public Health. New England Jour­nal of Medicine 376(7):605–607.

22. Fran­son, J. 2013. Tran­scra­nial di­rect cur­rent stim­u­la­tion tDCS tech­nol­ogy may im­prove cog­ni­tive abil­ity | Na­tional Post. Na­tional Post.

23. Galanie, S., K. Thodey, I. J. Tren­chard, M. Fils­inger In­ter­rante, and C. D. Smolke. 2015. Com­plete biosyn­the­sis of opi­oids in yeast. Science (New York, N.Y.) 349(6252):1095–1100.

24. Gilliam, T. 1998. Fear and Loathing in Las Ve­gas. Ad­ven­ture, Com­edy, Drama.

25. Global Mar­ket In­sights. 2017. Neu­rostim­u­la­tion De­vices Mar­ket worth over $13 billion by 2023: Global Mar­ket In­sights, Inc. https://​​globe­newswire.com/​​news-re­lease/​​2017/​​07/​​12/​​1042929/​​0/​​en/​​Neu­rostim­u­la­tion-De­vices-Mar­ket-worth-over-13-billion-by-2023-Global-Mar­ket-In­sights-Inc.html.

26. Gon­za­lez, R. 2018, April 6. A Brain-Boost­ing Pros­the­sis Moves From Rats to Hu­mans. Wired.

27. Good­man, B. 2011. Deep Brain Stim­u­la­tion May Offer Last­ing Benefits for Park­in­son’s Disease. We­bMed.

28. Groppa, S., A. Oliviero, A. Eisen, A. Quar­tarone, L. G. Co­hen, V. Mall, A. Kaelin-Lang, T. Mima, S. Rossi, and G. W. Thick­b­room. 2012. A prac­ti­cal guide to di­ag­nos­tic tran­scra­nial mag­netic stim­u­la­tion: re­port of an IFCN com­mit­tee. Clini­cal Neu­ro­phys­iol­ogy 123(5):858–882.

29. Gunter, M. J., N. Mur­phy, A. J. Cross, L. Dos­sus, L. Dar­tois, G. Fagher­azzi, R. Kaaks, T. Kühn, H. Boe­ing, K. Alek­san­drova, A. Tjøn­neland, A. Olsen, K. Over­vad, S. C. Larsen, M. L. Re­dondo Cornejo, A. Agudo, M. J. Sánchez Pérez, J. M. Altz­ibar, C. Navarro, E. Ar­danaz, K.-T. Khaw, A. But­ter­worth, K. E. Brad­bury, A. Tri­chopoulou, P. La­giou, D. Tri­chopoulos, D. Palli, S. Gri­oni, P. Vineis, S. Pan­ico, R. Tu­mino, B. Bueno-de-Mesquita, P. Siersema, M. Leen­ders, J. W. J. Beu­lens, C. U. Uiter­waal, P. Wal­lström, L. M. Nils­son, R. Land­berg, E. Wei­der­pass, G. Skeie, T. Braaten, P. Bren­nan, I. Li­caj, D. C. Mul­ler, R. Sinha, N. Ware­ham, and E. Riboli. 2017. Coffee Drink­ing and Mor­tal­ity in 10 Euro­pean Coun­tries: A Multi­na­tional Co­hort Study. An­nals of In­ter­nal Medicine 167(4):236.

30. Han­son, R. 2008. Catas­tro­phe, so­cial col­lapse, and hu­man ex­tinc­tion. Page 554 in N. Bostrom and M. M. Cirkovic, ed­i­tors. Global catas­trophic risks (p. Oxford Univer­sity Press, Oxford.

31. Han­son, R. 2016. The Age of Em: Work, Love, and Life when Robots Rule the Earth. Oxford Univer­sity Press.

32. hit­the­limit. 2008. Психогенная сингулярность. LJ.

33. Jo­han­son, C. E., R. L. Balster, and K. Bonese. 1976. Self-ad­minis­tra­tion of psy­chomo­tor stim­u­lant drugs: The effects of un­limited ac­cess. Phar­ma­col­ogy Bio­chem­istry and Be­hav­ior 4(1):45–51.

34. Khal­tu­rina, D. A., and A. V. Koro­taev. 2008. Al­co­hol and nar­cotics as fac­tors of the de­mo­graphic crisis. So­ciolog­i­cal Re­search 47(3):18–31.

35. Less­wrong­wiki. 2018. Or­gas­mium—Less­wrong­wiki.

36. Lin­klet­ter, M., K. Gor­don, and J. Dooley. 2010. The chok­ing game and YouTube: a dan­ger­ous com­bi­na­tion. Clini­cal pe­di­atrics 49(3):274–279.

37. Maron, D. F. 2018. How Opi­oids Kill. Scien­tific Amer­i­can.

38. Mer­rer, J. L., J. A. J. BECKER, K. BEFORT, and B. L. KIEFFER. 2009. Re­ward Pro­cess­ing by the Opi­oid Sys­tem in the Brain. Phys­iolog­i­cal re­views 89(4):1379–1412.

39. Miller, J. D. 2018. A Ra­tion­ally Ad­dicted Ar­tifi­cial Su­per­in­tel­li­gence.

40. Mu­flax. 2011. Why No Wire­head­ing?

41. Na­tional Cen­ter for Health Statis­tics. 2018, March 5. FastS­tats. https://​​www.cdc.gov/​​nchs/​​fas­tats/​​deaths.htm.

42. Na­tional in­sti­tute on drug abuse. 2018. Over­dose Death Rates | Na­tional In­sti­tute on Drug Abuse (NIDA). https://​​www.dru­gabuse.gov/​​re­lated-top­ics/​​trends-statis­tics/​​over­dose-death-rates.

43. Nazare­tian, A. 2004. Civ­i­liza­tiona crisises in the con­text iof uni­ver­sal his­tory. (In Rus­sian). Mir.

44. New­port, D. J., L. L. Car­pen­ter, W. M. McDon­ald, J. B. Po­tash, M. To­hen, C. B. Ne­meroff, A. C. of R. T. F. on N. Bio­mark­ers, and Treat­ments. 2015. Ke­tamine and other NMDA an­tag­o­nists: early clini­cal tri­als and pos­si­ble mechanisms in de­pres­sion. Amer­i­can Jour­nal of Psy­chi­a­try 172(10):950–966.

45. Neyaskin, G. 2018, Jan­uary 27. Real­ity Check: Fall in Rus­sian al­co­hol con­sump­tion. BBC News.

46. Oberst, U., E. Weg­mann, B. Stodt, M. Brand, and A. Chamarro. 2017. Nega­tive con­se­quences from heavy so­cial net­work­ing in ado­les­cents: The me­di­at­ing role of fear of miss­ing out. Jour­nal of ado­les­cence 55:51–60.

47. Olds, J., and P. Milner. 1954. Pos­i­tive re­in­force­ment pro­duced by elec­tri­cal stim­u­la­tion of sep­tal area and other re­gions of rat brain. Jour­nal of com­par­a­tive and phys­iolog­i­cal psy­chol­ogy 47(6):419.

48. Oster, G. 1973. Au­di­tory beats in the brain. Scien­tific Amer­i­can 229(4):94–103.

49. Ped­er­sen, W., S. Sand­berg, and H. Copes. 2015. High speed: Am­phetamine use in the con­text of con­ven­tional cul­ture. De­viant Be­hav­ior 36(2):146–165.

50. Pierre, L. S.-, and M. A. Pers­inger. 2006. Ex­per­i­men­tal fa­cil­i­ta­tion of the sensed pres­ence is pre­dicted by the spe­cific pat­terns of the ap­plied mag­netic fields, not by sug­gestibil­ity: re-analy­ses of 19 ex­per­i­ments. In­ter­na­tional Jour­nal of Neu­ro­science 116(19):1079–1096.

51. Po­li­an­skaia, A. 2018, July 7. Jack­son Coe found dead at bot­tom of build­ing in New York. The In­de­pen­dent.

52. Portenoy, R. K., J. O. Jar­den, J. J. Sidtis, R. B. Lip­ton, K. M. Foley, and D. A. Rot­ten­berg. 1986. Com­pul­sive tha­la­mic self-stim­u­la­tion: a case with metabolic, elec­tro­phys­iologic and be­hav­ioral cor­re­lates. Pain 27(3):277–290.

53. Price, L. 2018. Neu­ral im­plants and the race to merge the hu­man brain with Ar­tifi­cial In­tel­li­gence. Thought Lead­er­ship In Digi­tal Health | Lon­don | UK.

54. Ramos, L., J. E. Hunter, and J. H. Wolfe. 2017. Viral Vec­tor Gene De­liv­ery to the Brain for Treat­ing Neu­ro­ge­netic Diseases. Pages 89–125 Drug and Gene De­liv­ery to the Cen­tral Ner­vous Sys­tem for Neu­ro­pro­tec­tion. Springer.

55. Reuters. 2007, Septem­ber 17. Man in China dies af­ter three-day In­ter­net ses­sion. Reuters.

56. Roth­stein, A. 2015, Novem­ber 25. How the Most Ex­treme Trips on Erowid Trans­formed Modern Drug Cul­ture. Mother­board.

57. Rout­ten­berg, A. 1964. SELF-STARVATION CAUSED BY FEEDING CENTER SELF-STIMULATION. Pages 502–502 Amer­i­can Psy­chol­o­gist. AMER PSYCHOLOGICAL ASSOC 750 FIRST ST NE, WASHINGTON, DC 20002-4242.

58. Rout­ten­berg, A., and J. Lindy. 1965. Effects of the availa­bil­ity of re­ward­ing sep­tal and hy­potha­la­mic stim­u­la­tion on bar press­ing for food un­der con­di­tions of de­pri­va­tion. Jour­nal of com­par­a­tive and phys­iolog­i­cal psy­chol­ogy 60(2):158.

59. Sch­laepfer, T. E., B. H. Bew­er­nick, S. Kayser, B. Mädler, and V. A. Coe­nen. 2013. Rapid Effects of Deep Brain Stim­u­la­tion for Treat­ment-Re­sis­tant Ma­jor De­pres­sion. Biolog­i­cal Psy­chi­a­try 73(12):1204–1212.

60. Sim­ler, K., and R. Han­son. 2017. The Elephant in the Brain: Hid­den Mo­tives in Every­day Life. Oxford Univer­sity Press.

61. Spar­row, G., R. Hurd, R. Car­l­son, and A. Molina. 2018. Ex­plor­ing the effects of galan­tamine paired with med­i­ta­tion and dream re­liv­ing on re­called dreams: Toward an in­te­grated pro­to­col for lu­cid dream in­duc­tion and night­mare re­s­olu­tion. Con­scious­ness and Cog­ni­tion 63:74–88.

62. Springer, J., and W. J. Cubała. 2018. Ti­anep­tine Abuse and Depen­dence in Psy­chi­a­tric Pa­tients: A Re­view of 18 Case Re­ports in the Liter­a­ture. Jour­nal of Psy­choac­tive Drugs 50(3):275–280.

63. Stru­gatsky, A., and B. Stru­gatsky. 1976. The Fi­nal Cir­cle of Par­adise. Trans. Leonid Re­nen. New York, DAW.

64. Suthar, A. B., and T. Bärnighausen. 2017. An­tiretro­viral ther­apy and pop­u­la­tion mor­tal­ity: Lev­er­ag­ing rou­tine na­tional data to ad­vance policy. PLoS Medicine 14(12).

65. Tay­lor, J. 2012. Wire­head­ing: Towards a Con­sumer Mar­ket?

66. Tem­ple­ton, G. 2017. Elon Musk’s Neu­raLink Is Not a Neu­ral Lace Com­pany. https://​​www.in­verse.com/​​ar­ti­cle/​​30600-elon-musk-neu­ral­ink-neu­ral-lace-neu­ral-dust-elec­trode.

67. Toblin, R. L., L. J. Paulozzi, J. Gilchrist, and P. J. Rus­sell. 2008. Un­in­ten­tional stran­gu­la­tion deaths from the” Chok­ing Game” among youths aged 6-19 years–United States, 1995—2007. Jour­nal of safety re­search 39(4):445–448.

68. Tol­le­son, C., J. Stroh, J. Ehren­feld, J. Neimat, P. Kon­rad, and F. Phibbs. 2014. The fac­tors in­volved in deep brain stim­u­la­tion in­fec­tion: a large case se­ries. Stereo­tac­tic and Func­tional Neu­ro­surgery 92(4):227–233.

69. To­masik, B. 2015. How Likely is Wire­head­ing?

70. Turchin, A., and D. Denken­berger. 2018. Clas­sifi­ca­tion of Global Catas­trophic Risks Con­nected with Ar­tifi­cial in­tel­li­gence. Un­der re­view in AI&So­ciety.

71. Turchin, A., B. Green, and D. Denken­berger. 2017. Mul­ti­ple Si­mul­ta­neous Pan­demics as Most Danger­ous Global Catas­trophic Risk Con­nected with Bioweapons and Syn­thetic Biol­ogy. Un­der re­view in Health Se­cu­rity.

72. Van Be­souw, R. 2013. Im­plant­ing Aware­ness |. https://​​www.soundon­sound.com/​​sound-ad­vice/​​im­plant­ing-aware­ness.

73. Wen­ders, W. 1991. Un­til the End of the World. Ac­tion, Drama, Sci-Fi.

74. Wise, R. A. 2002. Brain Re­ward Cir­cuitry: In­sights from Unsensed In­cen­tives. Neu­ron 36(2):229–240.

75. Wood, A. M., S. Kap­toge, A. S. But­ter­worth, P. Willeit, S. War­nakula, T. Bolton, E. Paige, D. S. Paul, M. Sweet­ing, and S. Burgess. 2018. Risk thresh­olds for al­co­hol con­sump­tion: com­bined anal­y­sis of in­di­vi­d­ual-par­ti­ci­pant data for 599 912 cur­rent drinkers in 83 prospec­tive stud­ies. The Lancet 391(10129):1513–1523.

76. Yager, L. M., A. F. Gar­cia, A. M. Wun­sch, and S. M. Fer­gu­son. 2015. The ins and outs of the stri­a­tum: Role in drug ad­dic­tion. Neu­ro­science 301:529–541.

77. Yam­polskiy, R. 2014. Utility Func­tion Se­cu­rity in Ar­tifi­cially In­tel­li­gent Agents. Jour­nal of Ex­per­i­men­tal and The­o­ret­i­cal Ar­tifi­cial In­tel­li­gence (JETAI):1–17.

78. Yud­kowsky, E. 2009. The Fun The­ory Se­quence—Less Wrong.