CSER Advice to EU High-Level Expert Group on AI

Link post

Dr Sha­har Avin and Haydn Belfield sub­mit­ted ad­vice to the Euro­pean Union’s High-Level Ex­pert Group on Ar­tifi­cial In­tel­li­gence (AI HLEG).

The AI HLEG was es­tab­lished by the Euro­pean Com­mis­sion in June 2018 to sup­port the im­ple­men­ta­tion of its Strat­egy on Ar­tifi­cial In­tel­li­gence and to pre­pare two de­liv­er­ables: (1) AI Ethics Guidelines and (2) Policy and In­vest­ment Recom­men­da­tions. They con­sulted on their draft AI Ethics Guidelines from 18 De­cem­ber to 1 Fe­bru­ary.

Our full sub­mis­sion is be­low.


Re­sponse to the Euro­pean Com­mis­sion’s High-Level Ex­pert Group on Ar­tifi­cial In­tel­li­gence’s Draft Ethics Guidelines for Trust­wor­thy AI

We are writ­ing from the Cen­tre for the Study of Ex­is­ten­tial Risk, a re­search group at the Univer­sity of Cam­bridge which stud­ies the se­cu­rity im­pli­ca­tions of emerg­ing tech­nolo­gies. For the last five years we have been closely in­volved with the Euro­pean and in­ter­na­tional de­bate about the eth­i­cal and so­cietal im­pli­ca­tions of ar­tifi­cial in­tel­li­gence (AI).

Th­ese Draft Ethics Guidelines are an im­por­tant, con­crete step for­ward in the in­ter­na­tional de­bate on AI ethics. In par­tic­u­lar the list of tech­ni­cal and non-tech­ni­cal meth­ods and the as­sess­ment list will be use­ful to re­searchers and tech­nol­ogy com­pany em­ploy­ees who want to en­sure that the AI sys­tems they are busy de­vel­op­ing and de­ploy­ing are trust­wor­thy.

“The list of “Re­quire­ments of Trust­wor­thy AI” is a use­ful one. ‘Ro­bust­ness’ and ‘Safety’ are par­tic­u­larly im­por­tant re­quire­ments. They are both of­ten in­di­vi­d­u­ally men­tioned in sets of AI prin­ci­ples, and there are ex­ten­sive and dis­tinct fields of study for each of them. Ro­bust­ness is an im­por­tant re­quire­ment be­cause our AI sys­tems must be se­cure and able to cope with er­rors. Safety is an im­por­tant re­quire­ment as our AI sys­tems must not harm users, re­sources or the en­vi­ron­ment.

Ro­bust­ness and safety are cru­cial re­quire­ments for trust­wor­thi­ness. As an anal­ogy, con­sider that we could not call a bridge ‘trust­wor­thy’ if it was not re­li­able and re­silient to at­tack, and also safe for its users and the en­vi­ron­ment. Th­ese two re­quire­ments are im­por­tantly dis­tinct from the other re­quire­ments, and work best as stand-alone re­quire­ments.”

Ad­di­tional tech­ni­cal and non-tech­ni­cal methods

The re­port “in­vite[s] stake­hold­ers par­tak­ing in the con­sul­ta­tion of the Draft Guidelines to share their thoughts on ad­di­tional tech­ni­cal or non-tech­ni­cal meth­ods that can be con­sid­ered in or­der to ad­dress the re­quire­ments of Trust­wor­thy AI.”

We would like to share some ad­di­tional tech­ni­cal and non-tech­ni­cal meth­ods that are not yet on the list. Th­ese are mostly drawn from the ma­jor Fe­buary 2018 re­port The Mal­i­cious Use of Ar­tifi­cial In­tel­li­gence: Fore­cast­ing, Preven­tion, and Miti­ga­tion. We co-au­thored this re­port with 26 in­ter­na­tional ex­perts from academia and in­dus­try to as­sess how crim­i­nals, ter­ror­ists and rogue states could mal­i­ciously use AI over the next five years, and how these mi­suses might be pre­vented and miti­gated.

When re­leased this re­port was cov­ered across Europe and wel­comed by ex­perts in differ­ent do­mains, such as AI policy, cy­ber­se­cu­rity, and ma­chine learn­ing. We have sub­se­quently con­sulted sev­eral Euro­pean gov­ern­ments, com­pa­nies and civil so­ciety groups on the recom­men­da­tions of this re­port.

The Euro­pean Union’s Co­or­di­nated Plan on Ar­tifi­cial In­tel­li­gence, pub­lished on the 7th of De­cem­ber 2018, men­tions the im­por­tance of the se­cu­rity-re­lated AI ap­pli­ca­tions and pre­vent­ing mal­i­cious use:

“2.7. Se­cu­rity-re­lated as­pects of AI ap­pli­ca­tions and in­fras­truc­ture, and in­ter­na­tional se­cu­rity agenda: There is a need to bet­ter un­der­stand how AI can im­pact se­cu­rity in three di­men­sions: how AI could en­hance the ob­jec­tives of the se­cu­rity sec­tor; how AI tech­nolo­gies can be pro­tected from at­tacks; and how to ad­dress any po­ten­tial abuse of AI for mal­i­cious pur­poses.”

Sev­eral of the meth­ods we ex­plored are already men­tioned in the Guidelines, such as codes of con­duct, ed­u­ca­tion and so­cietal di­alogue. How­ever we also ex­plored some meth­ods that you do not yet men­tion. Our re­port made recom­men­da­tions in four ‘pri­or­ity re­search ar­eas’. In this re­sponse we split these into ‘tech­ni­cal’ and ‘non-tech­ni­cal’ meth­ods.

  • Learn­ing from and with the Cy­ber­se­cu­rity Community

  • Ex­plor­ing Differ­ent Open­ness Models

  • Pro­mot­ing a Cul­ture of Responsibility

  • Devel­op­ing Tech­nolog­i­cal and Policy Solutions

Tech­ni­cal meth­ods in­clude:

Learn­ing from and with the Cy­ber­se­cu­rity Community

For­mal ver­ifi­ca­tion. The use of math­e­mat­i­cal meth­ods to offer for­mal proofs that a sys­tem will op­er­ate as in­tended. In re­cent years this has worked on com­plex sys­tems, in­clud­ing the Com­pCert com­piler and the seL4 micro­ker­nel. It could be ap­plied to AI sys­tems.

Se­cu­rity tools. Soft­ware de­vel­op­ment and de­ploy­ment tools now in­clude an ar­ray of se­cu­rity-re­lated ca­pa­bil­ities (test­ing, fuzzing, anomaly de­tec­tion, etc.). Tools could be de­vel­oped to make it stan­dard to test and im­prove the se­cu­rity of AI com­po­nents dur­ing de­vel­op­ment and de­ploy­ment. Tools could in­clude: au­to­matic gen­er­a­tion of ad­ver­sar­ial data; tools for analysing clas­sifi­ca­tion er­rors; au­to­matic de­tec­tion of at­tempts at re­mote model ex­trac­tion or re­mote vuln­er­a­bil­ity scan­ning; and au­to­matic sug­ges­tions for im­prov­ing model ro­bust­ness.

Se­cure hard­ware. In­creas­ingly, AI sys­tems are trained and run on hard­ware that is semi-spe­cial­ized (e.g. GPUs) or fully spe­cial­ized (e.g. TPUs). Se­cu­rity fea­tures could be in­cor­po­rated into AI-spe­cific hard­ware to, for ex­am­ple, pre­vent copy­ing, re­strict ac­cess, and fa­cil­i­tate ac­tivity au­dits.

Ex­plor­ing Differ­ent Open­ness Models

Cen­tral ac­cess li­cens­ing mod­els. In this emerg­ing com­mer­cial struc­ture, cus­tomers use ser­vices (like sen­ti­ment anal­y­sis or image recog­ni­tion) from a cen­tral provider with­out hav­ing ac­cess to the tech­ni­cal de­tails of the sys­tem. This model could provide wide­spread use of a given ca­pa­bil­ity while re­duc­ing mal­i­cious use by, for ex­am­ple: limit­ing the speed of use, pre­vent­ing some large-scale harm­ful ap­pli­ca­tions; and ex­plic­itly pro­hibit­ing mal­i­cious use in the terms and con­di­tions, al­low­ing clear le­gal re­course.

Pro­mot­ing a Cul­ture of Responsibility

Differ­en­tially pri­vate ma­chine learn­ing al­gorithms. Th­ese com­bine their train­ing data with noise to main­tain pri­vacy while min­i­miz­ing effects on perfor­mance. There is in­creas­ing re­search on this tech­nolog­i­cal tool for pre­serv­ing user data pri­vacy.

Se­cure multi-party com­pu­ta­tion. MPC refers to pro­to­cols that al­low mul­ti­ple par­ties to jointly com­pute func­tions, while keep­ing each party’s in­put to the func­tion pri­vate. This makes it pos­si­ble to train ma­chine learn­ing sys­tems on sen­si­tive data with­out sig­nifi­cantly com­pro­mis­ing pri­vacy. For ex­am­ple, med­i­cal re­searchers could train a sys­tem on con­fi­den­tial pa­tient records by en­gag­ing in an MPC pro­to­col with the hos­pi­tal that pos­sesses them.

Co­or­di­nated use of AI for pub­lic-good se­cu­rity. AI-based defen­sive se­cu­rity mea­sures could be de­vel­oped and dis­tributed widely to nudge the offense-defense bal­ance in the di­rec­tion of defense. For ex­am­ple, AI sys­tems could be used to re­fac­tor ex­ist­ing code bases or new soft­ware to se­cu­rity best prac­tices.

Mon­i­tor­ing of AI-rele­vant re­sources. Mon­i­tor­ing regimes are well-es­tab­lished in the con­text of other dual-use tech­nolo­gies, most no­tably the mon­i­tor­ing of fis­sile ma­te­ri­als and chem­i­cal pro­duc­tion fa­cil­ities. Un­der cer­tain cir­cum­stances it might be fea­si­ble and ap­pro­pri­ate to mon­i­tor in­puts to AI tech­nolo­gies such as hard­ware, tal­ent, code, and data.

Non-tech­ni­cal meth­ods in­clude:

Learn­ing from and with the Cy­ber­se­cu­rity Community

Red team­ing. A com­mon tool in cy­ber­se­cu­rity and mil­i­tary prac­tice, where a “red team” com­posed of se­cu­rity ex­perts de­liber­ately plans and car­ries out at­tacks against the sys­tems and prac­tices of the or­ga­ni­za­tion (with some limi­ta­tions to pre­vent last­ing dam­age), with an op­tional “blue team” re­spond­ing to these at­tacks. Ex­ten­sive use of red team­ing to dis­cover and fix po­ten­tial se­cu­rity vuln­er­a­bil­ities and safety is­sues could be a pri­or­ity of AI de­vel­op­ers, es­pe­cially in crit­i­cal sys­tems.

Re­spon­si­ble dis­clo­sure of AI vuln­er­a­bil­ities. In the cy­ber­se­cu­rity com­mu­nity, “0-days” are soft­ware vuln­er­a­bil­ities that have not been made pub­li­cly known, so defen­ders have “zero days” to pre­pare for an at­tack mak­ing use of them. It is com­mon prac­tice to dis­close these vuln­er­a­bil­ities to af­fected par­ties be­fore pub­lish­ing widely about them, in or­der to provide an op­por­tu­nity for a patch to be de­vel­oped. AI-spe­cific pro­ce­dures could be es­tab­lished for con­fi­den­tial re­port­ing of se­cu­rity vuln­er­a­bil­ities, po­ten­tial ad­ver­sar­ial in­puts, and other types of ex­ploits dis­cov­ered in AI sys­tems.

Fore­cast­ing se­cu­rity-rele­vant ca­pa­bil­ities. “White-hat” (or so­cially-minded) efforts to pre­dict how AI ad­vances will en­able more effec­tive cy­ber­at­tacks could al­low for more effec­tive prepa­ra­tions by defen­ders. More rigor­ous track­ing of AI progress and pro­lifer­a­tion would also help defen­sive prepa­ra­tions.

Ex­plor­ing Differ­ent Open­ness Models

Pre-pub­li­ca­tion risk as­sess­ment in tech­ni­cal ar­eas of spe­cial con­cern. In other dual-use ar­eas, such as biotech­nol­ogy and com­puter se­cu­rity, the norm is to analyse the par­tic­u­lar risks (or lack thereof) of a par­tic­u­lar ca­pa­bil­ity if it be­came widely available, and de­cide on that ba­sis whether, and to what ex­tent, to pub­lish it. AI de­vel­op­ers could carry out some kind of risk as­sess­ment to de­ter­mine what level of open­ness is ap­pro­pri­ate for some types of AI re­search re­sults, such as work speci­fi­cally re­lated to digi­tal se­cu­rity, ad­ver­sar­ial ma­chine learn­ing, or crit­i­cal sys­tems.

Shar­ing regimes that favour safety and se­cu­rity. Com­pa­nies cur­rently share in­for­ma­tion about cy­ber-at­tacks amongst them­selves through In­for­ma­tion Shar­ing and Anal­y­sis Cen­ters (ISACs) and In­for­ma­tion Shar­ing and Anal­y­sis Or­ga­ni­za­tions (ISAOs). Analo­gous ar­range­ments could be made for some types of AI re­search re­sults to be se­lec­tively shared among a pre­de­ter­mined set of ‘trusted par­ties’ that meet cer­tain crite­ria, such as effec­tive in­for­ma­tion se­cu­rity and ad­her­ence to eth­i­cal norms. For ex­am­ple, cer­tain forms of offen­sive cy­ber­se­cu­rity re­search that lev­er­age AI could be shared be­tween trusted or­ga­ni­za­tions for vuln­er­a­bil­ity dis­cov­ery pur­poses, but would be harm­ful if more widely dis­tributed.

Pro­mot­ing a Cul­ture of Responsibility

Whistle­blow­ing mea­sures. Whistle­blow­ing is when an em­ployee passes on po­ten­tially con­cern­ing in­for­ma­tion to an out­side source. Whistle­blow­ing pro­tec­tions might use­ful in pre­vent­ing AI-re­lated mi­suse risks.

Nuanced nar­ra­tives. There should be nu­anced, suc­cinct and com­pel­ling nar­ra­tives of AI re­search and its im­pacts that bal­ance op­ti­mism about its vast po­ten­tial with a level-headed recog­ni­tion of its challenges. Ex­ist­ing nar­ra­tives like the dystopian “robot apoc­a­lypse” trope and the utopian “au­toma­tion boon” trope both have ob­vi­ous short­com­ings. A nar­ra­tive like “dual-use” might be more productive

No comments.