Assessing the state of AI R&D in the US, China, and Europe – Part 1: Output indicators

How does the cur­rent state of ar­tifi­cial in­tel­li­gence (AI) re­search and de­vel­op­ment (R&D) com­pare among the US, China, and Europe? I will pub­lish my find­ings on this ques­tion in two posts. This one fo­cuses on R&D out­put (sci­en­tific pub­li­ca­tions and patents), and the sec­ond one will fo­cus on R&D in­put. I might also pub­lish a third ar­ti­cle that cov­ers con­clu­sions from the first two.

The AI gov­er­nance com­mu­nity con­cerned with ex­is­ten­tial risks has so far mainly fo­cused on US–China re­la­tions while ne­glect­ing Europe. Part of the rea­son for this might be that Europe sig­nifi­cantly lags be­hind the two coun­tries in terms of rele­vant AI R&D. How­ever, my im­pres­sion is that this has not been sub­stan­ti­ated since Europe is sim­ply miss­ing from most analy­ses.

I also aim to con­tribute to a more clear-eyed com­par­i­son be­tween the state of AI R&D in the US and China. This may im­prove the gen­eral con­ver­sa­tions about an “AI arms race,” which seem to be es­pe­cially promi­nent in the US, where many be­lieve that the coun­try is “los­ing” to China.

Since many re­ports and ar­ti­cles on this com­par­i­son tend to fo­cus on a sin­gle anal­y­sis or study, my goal was to re­view the available liter­a­ture for the most im­por­tant in­di­ca­tors of AI R&D, al­low­ing a more holis­tic pic­ture. I think ad­di­tional work on con­struct­ing a solid in­dex for AI R&D would be valuable. The Cen­ter for Data In­no­va­tion, Jeffrey Ding, and a Ger­man think tank, Stiftung Neue Ver­ant­wor­tung (only available in Ger­man), have done some pre­limi­nary work on this.

Summary

  • I ex­am­ined the num­ber and qual­ity of pub­li­ca­tions as well as the num­ber and qual­ity of filed patents, since they are the most com­mon R&D out­put in­di­ca­tors.

  • Stud­ies or re­ports in this field do not seem to be par­tic­u­larly rigor­ous or re­li­able. The in­di­ca­tors I ex­am­ined are very coarse and do not cap­ture many rele­vant nu­ances. I did not at­tempt to ar­rive at nu­mer­i­cal es­ti­mates. Most stud­ies and analy­ses do not con­sider ma­chine learn­ing re­search alone, but all re­search re­lated to ar­tifi­cial in­tel­li­gence. This in­cludes work on sym­bolic AI (e.g., ex­pert sys­tems).

  • For al­most all in­di­ca­tors, the US in­sti­tu­tions seem to be the global lead­ers. This is clear­est for con­fer­ence con­tri­bu­tions and top bench­mark pa­pers.

  • The out­put gen­er­ated by Euro­pean and Chi­nese in­sti­tu­tions of­ten seems to be in the same bal­l­park. At times, China seems to have a slight lead (e.g. highly cited pub­li­ca­tions), while at other times, Europe does (e.g. top con­fer­ence con­trib­u­tors). Growth trends seem to fa­vor China.

  • The data from AI con­fer­ences and AI bench­marks seem to con­firm that this is also the pat­tern for ma­chine learn­ing out­put more nar­rowly. How­ever, one source sug­gests that China gen­er­ates much more deep learn­ing out­put than both the US and Europe.

Method­ol­ogy and limitations

Scien­tific pub­li­ca­tions and patents are the most com­mon quan­ti­ta­tive in­di­ca­tors of R&D out­put.[1] One could also use qual­i­ta­tive as­sess­ment tools like ex­pert eval­u­a­tion, but this is out­side the scope of this post. My im­pres­sion is also that these two in­di­ca­tors are widely used in the field of AI (e.g., China AI Devel­op­ment Re­port 2018 and AI In­dex 2018).

My fo­cus was on cre­at­ing a snap­shot of the cur­rent land­scape, as op­posed to a pro­jec­tion into the fu­ture. How­ever, where available and clear, I do point out trends that one could cau­tiously use to ex­trap­o­late into the fu­ture.

I looked for rele­vant sources by con­duct­ing key­word searches for the var­i­ous in­di­ca­tors on Google and Google News. I was also already aware of sev­eral re­ports and stud­ies on this topic. I looked up any cited refer­ences. I wel­come poin­t­ers to ad­di­tional sources.

My gen­eral im­pres­sion was that the qual­ity of stud­ies is not par­tic­u­larly high in this field. There is no peer-re­viewed work available as far as I can tell. Re­ports do not com­pare their find­ings, and method­olo­gies are of­ten vague. Thus, I would treat in­di­vi­d­ual find­ings with the ap­pro­pri­ate care. I have tried to provide method­olog­i­cal de­tails where rele­vant and available. I did not ap­ply a strict defi­ni­tion for “Europe,” since the sources differed in their defi­ni­tions. The reader may as­sume that “Europe,” un­less oth­er­wise stated, refers roughly to the Euro­pean Free Trade As­so­ci­a­tion (EFTA), which in­cludes all EU mem­ber states, Ice­land, Liecht­en­stein, Nor­way, and Switzer­land.

I did not at­tempt to ar­rive at nu­mer­i­cal es­ti­mates for the in­di­ca­tors. In­stead, I de­cided to share my con­clu­sions in qual­i­ta­tive terms to avoid the per­cep­tion of rigor and pre­ci­sion, which the un­der­ly­ing data do not per­mit.

The in­di­ca­tors I ex­am­ined are on a very high level and fail to cap­ture a lot of nu­ances. For in­stance, AI tech­nol­ogy has many differ­ent ap­pli­ca­tions (e.g., nat­u­ral lan­guage pro­cess­ing, com­puter vi­sion), is used in many differ­ent sec­tors (e.g., on­line re­tail, defense), and has differ­ent lay­ers (e.g., de­vel­op­ment plat­forms like Ten­sorFlow com­pared to a con­crete AI al­gorithms). The state of R&D with re­spect to these might differ sig­nifi­cantly across coun­tries or re­gions. How­ever, I be­lieve a more coarse-grained in­ves­ti­ga­tion is still in­for­ma­tive rather than mis­lead­ing.

Al­most all stud­ies and analy­ses I found do not fo­cus only on ma­chine learn­ing. They in­clude all pub­li­ca­tions re­lated to AI, which also in­cludes work on sym­bolic AI (e.g., ex­pert sys­tems). This might severely limit the rele­vance of these in­di­ca­tors, since the most no­table ad­vances in AI ca­pa­bil­ities since 2012 have been in ma­chine learn­ing (deep learn­ing and re­in­force­ment learn­ing in par­tic­u­lar). The sever­ity of this limi­ta­tion de­pends on the rele­vance of sym­bolic AI for fu­ture progress. There is con­sid­er­able dis­agree­ment on this point amongst AI ex­perts.

Scien­tific publications

Numer­ous sources have re­ported data on the to­tal num­ber of pub­li­ca­tions by coun­try or re­gion. Some also in­cluded an anal­y­sis of “highly cited” pub­li­ca­tions, but defi­ni­tions of this were not always clear or con­sis­tent. I also ag­gre­gated nu­mer­ous analy­ses on the dis­tri­bu­tion of con­tri­bu­tions to top AI con­fer­ences as a sep­a­rate in­di­ca­tor. Lastly, I in­cluded my own anal­y­sis of AI bench­mark pa­pers as re­ported by the Elec­tronic Fron­tier Foun­da­tion, since I am not aware of any other anal­y­sis of this dataset but con­sider it a good and unique rep­re­sen­ta­tion of cut­ting-edge AI re­search.

Num­ber of pub­li­ca­tions. In light of all the ev­i­dence I could find, I tend to be­lieve that in terms of the over­all num­ber of sci­en­tific pub­li­ca­tions on AI, Europe is slightly ahead of the US and China, with China gain­ing ground. Ac­cord­ing to the only source that re­stricted the anal­y­sis to deep learn­ing, China seems to have a sig­nifi­cant lead on the US, which, in turn, is sig­nifi­cantly ahead of Europe. The trend lines do not sug­gest that this rank­ing will change any time soon.

  • AI In­dex 2018 (p. 10): “Europe has con­sis­tently been the largest pub­lisher of AI pa­pers — 28% of AI pa­pers on [the Sco­pus database] in 2017 origi­nated in Europe.” How­ever, it seems like China (in sec­ond place) has been gain­ing ground on Europe (see graph be­low). How­ever, pa­pers seem to have been as­signed to coun­tries based on the lo­ca­tion of the au­thor, which would count a pa­per from “Google Zurich” as Euro­pean, not Amer­i­can. This likely un­der­es­ti­mates the share of pa­pers by US-owned in­sti­tu­tions, since I would ex­pect more of them to have rele­vant offices around the world.

  • WIPO Tech­nol­ogy Trends 2019 – Ar­tifi­cial In­tel­li­gence (World In­tel­lec­tual Prop­erty Or­ga­ni­za­tion) (p. 90): Also us­ing the Sco­pus database, the au­thors find that Chi­nese in­sti­tu­tions are lead­ing in terms of sci­en­tific pub­li­ca­tions on AI, fol­lowed by the US, the UK, In­dia, Ja­pan, Ger­many, France, Canada, Italy, and Spain. An eye­balled ag­gre­ga­tion would put all Euro­pean coun­tries (in­clud­ing the UK) on roughly equal foot­ing with China and the US, each ac­count­ing for around 20% of the to­tal. It is hard to com­pare this to the AI In­dex, since I do not know which time pe­riod they an­a­lyzed. I also do not know how they as­signed pub­li­ca­tions to spe­cific coun­tries.

  • China AI Devel­op­ment Re­port 2018[2] (p. 15): Us­ing the Web of Science database, the au­thors find China and the US as the lead­ing coun­tries in terms of AI pa­per out­put from 1997 to 2017. How­ever, if ag­gre­gated, the Euro­pean coun­tries’ out­put is sig­nifi­cantly higher than that for ei­ther of them: ~450,000, com­pared to ~370,000 and ~330,000, re­spec­tively. They do not give pre­cise num­bers for in­di­vi­d­ual years. The trend lines seem to sug­gest that China has had a higher growth rate over the past 20 years. I do not know how they as­signed pub­li­ca­tions to spe­cific coun­tries.

  • Global Ar­tifi­cial In­tel­li­gence In­dus­try Data Re­port 2019[3]: Also us­ing the Web of Science database (and similar key­words, from what I can tell), they echo that China has pub­lished the most AI pa­pers in the last decade and is also now lead­ing in terms of an­nual out­put. “The US ranks sec­ond in terms of pub­li­ca­tion vol­ume, fol­lowed by In­dia, Ger­many, and Ja­pan.” How­ever, given the limited trans­la­tion available, it is im­pos­si­ble for me to ag­gre­gate the Euro­pean coun­tries. I do not know how they as­signed pa­pers to coun­tries.

  • 2018 World AI In­dus­try Devel­op­ment Blue Book (p. 14): Rely­ing on a 2018 re­port by the China Academy of In­for­ma­tion and Com­mu­ni­ca­tions Tech­nol­ogy (CAICT; see foot­note 2), the au­thors of this joint re­port by the CAICT and Gart­ner find that the most AI pub­li­ca­tions from 1998 to 2018 came from the US (149,096), fol­lowed by China (141,840). How­ever, ag­gre­gat­ing all Euro­pean coun­tries (145,551) puts them ahead of China. In terms of an­nual out­put, China is likely in the lead, and the trend line also seems to fa­vor them (see graph be­low). It is not clear what method­ol­ogy they used. It is likely similar to the 2019 re­port from CAICT (see above).

  • US Na­tional Ar­tifi­cial In­tel­li­gence Re­search and Devel­op­ment Strate­gic Plan (p. 13): They also used the Web of Science database but, con­trary to the other re­ports, the au­thors of this doc­u­ment only ex­am­ined jour­nal ar­ti­cles men­tion­ing “deep learn­ing” or “deep neu­ral net­work.” China seems to be in the lead, closely fol­lowed by the US, with all other coun­tries trailing off. Even ag­gre­gated, I would ex­pect Euro­pean coun­tries to be sig­nifi­cantly be­hind. The trend line seems to fa­vor China and the US (see graph be­low). I do not know how they as­signed pa­pers to coun­tries.

Num­ber of highly cited pub­li­ca­tions. There is con­tra­dic­tory ev­i­dence for this in­di­ca­tor. How­ever, all sources agree that Europe is not in the lead. I would ex­pect the US and China to be in the same bal­l­park, with trend lines fa­vor­ing China. The sources where I can best de­ter­mine that they ac­tu­ally an­a­lyzed highly cited work seem to fa­vor the US (AI In­dex 2018, Allen In­sti­tute anal­y­sis). I could not find ev­i­dence for ma­chine learn­ing speci­fi­cally, since the US Na­tional Ar­tifi­cial In­tel­li­gence Re­search and Devel­op­ment Strate­gic Plan does not in­clude anal­y­sis for highly cited work.

  • AI In­dex 2018 (p. 17): Look­ing at the Sco­pus database, the au­thors find US-af­fili­ated au­thors lead­ing in terms of Field-Weighted Ci­ta­tion Im­pact, fol­lowed by Euro­pean- and Chi­nese-af­fili­ated ones. Similar to the raw num­ber of AI pa­pers in the database, China seems to be gain­ing ground on Europe, judg­ing from the trend line over the past few years (see graph be­low) The lat­est num­bers are from 2016.

  • China AI Devel­op­ment Re­port 2018 (p. 20f.): Us­ing the Web of Science database, this re­port puts China in the lead ahead of the US and any Euro­pean coun­try, both for the decade be­tween 2007 and 2017 and in terms of an­nual out­put in 2017. Ag­gre­gat­ing all Euro­pean coun­tries out of the top 10 from 2007 to 2017 (the UK, Ger­many, France, Italy, Spain), they al­most catch up to the US and China, reach­ing 2,096 highly cited pa­pers, com­pared to 2,241 and 2,349, re­spec­tively. The same ap­plies to “hot pa­pers.” Un­for­tu­nately, I do not know how they op­er­a­tional­ized ei­ther “highly cited” or “hot.” Eye­bal­ling com­bined Euro­pean out­put in 2017 alone; China is still in the lead with an in­creas­ingly large mar­gin, ahead of the US and Europe, which seem to be roughly tied (see graph be­low).

  • Global Ar­tifi­cial In­tel­li­gence In­dus­try Data Re­port 2019: They use the same database as the China AI Devel­op­ment Re­port 2018 and seem to find very similar num­bers.

  • 2018 World AI In­dus­try Devel­op­ment Blue Book (p. 15): The au­thors state that the Chi­nese share of the out­put of highly cited AI pa­pers has risen from “less than 15% in 2008 to 47% in 2017.” Since they rely on an in­ac­cessible pri­mary source and do not provide a method­ol­ogy, I find it hard to de­ter­mine the re­li­a­bil­ity of this claim.

  • Anal­y­sis by the Allen In­sti­tute for Ar­tifi­cial In­tel­li­gence (2019)[4]: Us­ing data from the Se­man­tic Scholar pro­ject, the au­thors ex­am­ined the dis­tri­bu­tion be­tween the US and China for highly cited AI pa­pers (top 50%, top 10%, and top 1% in terms of cita­tion count). For the top 50% of pa­pers in 2018, the US and China ac­count for ~25% each; for the top 10%, the US ac­counts for 29% and China for 26.5%; and for the top 1%, the US ac­counts for ~36% and China for ~30%. Ex­trap­o­lat­ing from past trends, the au­thors ex­pect China to take the lead in 2020 (top 10%) and 2025 (top 1%), re­spec­tively. How­ever, they did not in­clude an as­sess­ment of Euro­pean in­sti­tu­tions and could not provide num­bers upon re­quest. Given the com­bined share of US and Chi­nese pub­li­ca­tions, it is very un­likely that Europe en­joys a de­ci­sive lead. At the same time, it also seems im­plau­si­ble that they lag very far be­hind. It seems sen­si­ble to as­sume that Euro­pean coun­tries make up be­tween 50% and 80% of the re­spec­tive re­main­ing shares, which would make them com­pet­i­tive in the top 50% and top 10% of pa­pers, but likely not com­pet­i­tive in the top 1% of pa­pers. How­ever, I do not give this es­ti­mate much weight.

Num­ber of pub­li­ca­tions at top AI con­fer­ences. For this sec­tion, I ex­am­ined con­tri­bu­tions to the most rele­vant AI con­fer­ences as mea­sured by their h5-in­dex on Google Scholar. I in­cluded con­fer­ences in the top 10 pub­li­ca­tions of the cat­e­gories Ar­tifi­cial In­tel­li­gence and Com­puter Vi­sion & Pat­tern Recog­ni­tion. In terms of ac­cepted pa­pers at such con­fer­ences, US in­sti­tu­tions en­joy a clear lead ahead of Euro­pean and Chi­nese ones. The ev­i­dence also shows an edge for Euro­pean in­sti­tu­tions over Chi­nese ones, but not as de­ci­sive as the US lead. This trend is broadly echoed in ag­grega­tive data from the 2019 Global Ar­tifi­cial In­tel­li­gence In­dus­try Data Re­port. They stud­ied ac­cepted pa­pers from the Con­fer­ence on Com­puter Vi­sion and Pat­tern Recog­ni­tion (CVPR), the In­ter­na­tional Con­fer­ence on Com­puter Vi­sion (ICCV), NeurIPS, and the In­ter­na­tional Con­fer­ence on Robotics and Au­toma­tion (ICRA) and found Amer­i­can au­thors to ac­count for 52% of con­tri­bu­tions and Chi­nese au­thors[5] to ac­count for 18% of con­tri­bu­tions. They do not provide num­bers for Euro­pean au­thors. Out of the top 15 in­sti­tu­tions, they found eight to be based in the US, four in Europe (ETH, CNRS, INRIA, Max-Planck-Ge­sel­lschaft), and three in China. The only ex­cep­tion to this gen­eral pat­tern is the AAAI Con­fer­ence on Ar­tifi­cial In­tel­li­gence, at which Chi­nese and US in­sti­tu­tions are on equal foot­ing and Euro­pean ones are lag­ging sig­nifi­cantly be­hind (see be­low). I do not know what ac­counts for this. I would ex­pect most pa­pers at these con­fer­ences to re­late to ma­chine learn­ing, as op­posed to sym­bolic AI.

  • Con­fer­ence on Com­puter Vi­sion and Pat­tern Recog­ni­tion (CVPR): I could not find any pri­mary sources on this in terms of ac­cepted pa­pers.

  • Con­fer­ence on Neu­ral In­for­ma­tion Pro­cess­ing Sys­tems (NeurIPS): Among the top 20 in­sti­tu­tions in terms of cita­tions re­ceived from 1996 to 2019, 15 are based in the US, two in Europe (Max-Planck-Ge­sel­lschaft, UCL), two in Canada, and one in Is­rael. No Chi­nese in­sti­tu­tion made the list. Among the top 20 in­sti­tu­tions in terms of ac­cepted sub­mis­sions in 2019, 15 were from the US, two from Europe (Oxford, INRIA), two from Canada (MILA, Toronto), and one from China (Ts­inghua). In 2018, 17 were from the US, two from Europe (Oxford, ETH), and one from China (Ts­inghua). In 2017, 15 were from the US and four from Europe (Oxford, ETH, INRIA, Cam­bridge). No Chi­nese in­sti­tu­tions were among the top 20. This gen­eral trend is echoed in a more com­pre­hen­sive pub­li­ca­tion in­dex for the same year (see graph be­low).

  • In­ter­na­tional Con­fer­ence on Learn­ing Rep­re­sen­ta­tions (ICLR): The only pri­mary source I could find on ICRL does not provide a year for their anal­y­sis. How­ever, since OpenAI ap­pears on the list, it can­not be older than 2016. Among the top 20 in­sti­tu­tions in terms of ac­cepted pa­pers, 15 are from the US, three from Europe (ETH, Oxford, Cam­bridge), and two from Canada (Toronto, MILA). None are from China.

  • Euro­pean Con­fer­ence on Com­puter Vi­sion (ECCV): I could not find any pri­mary sources on this in terms of ac­cepted pa­pers.

  • In­ter­na­tional Con­fer­ence on Ma­chine Learn­ing (ICML): Among the top 20 in­sti­tu­tions in terms of ac­cepted con­tri­bu­tions in 2019, 14 were from the US, four from Europe (Oxford, ETH, EPFL, INRIA), one from China (Ts­inghua), and one from South Korea (KAIST). In 2018, 13 were from the US, six from Europe (Oxford, INRIA, EPFL, ETH, Max-Planck-Ge­sel­lschaft, Cam­bridge) and one from Canada (Toronto). No Chi­nese in­sti­tu­tion made it into the top 20.

  • In­ter­na­tional Con­fer­ence on Com­puter Vi­sion (ICCV): I could not find any pri­mary sources on this in terms of ac­cepted pa­pers.

  • AAAI Con­fer­ence on Ar­tifi­cial In­tel­li­gence: In 2018, the US and China were clearly lead­ing in terms of ac­cepted pa­pers: 268 and 265, re­spec­tively, ac­cord­ing to the AI In­dex 2018 (p. 20). Euro­pean coun­tries were far be­low that, at around 74.

Num­ber of top bench­mark pub­li­ca­tions. I an­a­lyzed the AI Progress Mea­sure­ment database of the Elec­tronic Fron­tier Foun­da­tion. I ex­cluded du­pli­cate en­tries (i.e., pa­pers that were in­cluded on mul­ti­ple bench­marks) and only stud­ied the five best-perform­ing al­gorithms per bench­mark.[6] Across all bench­marks, US in­sti­tu­tions clearly dom­i­nate. If one counts Deep­Mind as a US in­sti­tu­tion, then US in­sti­tu­tions were in­volved in 91 unique pa­pers of the database, clearly ahead of Canada (14) and China (14). If all Euro­pean coun­tries (in­clud­ing the UK, but ex­clud­ing Deep­Mind) are ag­gre­gated, their in­sti­tu­tions were in­volved in 17 unique pa­pers, which would put them ahead of China. Th­ese bench­marks only cap­ture ad­vances in ma­chine learn­ing.

Patents

I ag­gre­gated the nu­mer­ous sources on the to­tal num­ber of patent fam­i­lies by coun­try or re­gion. I also tried to find re­li­able data on highly cited or highly rele­vant patents, but I could only find a sin­gle graph in a re­port by the World In­tel­lec­tual Prop­erty Or­ga­ni­za­tion.

Num­ber of patent fam­i­lies. Based on the available ev­i­dence, it seems to me that the US is the global leader in terms of AI patent out­put. I have con­sid­er­able un­cer­tainty re­gard­ing the com­par­i­son of Europe and China. Ten­ta­tively, I would put Europe slightly in front for now based on the AI In­dex 2018 and the re­port by the UK In­tel­lec­tual Prop­erty Office (see be­low). There also ap­pears to be some ev­i­dence that the patent count in China is in­flated (via Jeffrey Ding’s US con­gres­sional tes­ti­mony).

  • AI In­dex 2018 (p. 35): The au­thors used the patent database from am­plified.ai (method­ol­ogy). They find that the ag­gre­gate AI patent count of in­ven­tors in the UK, France, and Ger­many – which pre­sum­ably ac­counts for the ma­jor­ity of rele­vant Euro­pean patents – clearly lags be­hind that of the US. They also rank lower than Korea, Ja­pan, and China, but the differ­ence is much smaller and might van­ish if one in­cluded all Euro­pean coun­tries in the anal­y­sis. China, how­ever, seems to have a higher growth rate than the US and Europe (see graph be­low).

  • WIPO Tech­nol­ogy Trends 2019 – Ar­tifi­cial In­tel­li­gence (p. 60) (method­ol­ogy available here): Among the top 30 ap­pli­cant or­ga­ni­za­tions for AI patent fam­i­lies (not ex­clu­sively ma­chine learn­ing) in 2018, 12 were from Ja­pan, six from the US (IBM and Microsoft in the two top spots), five from China, four from Europe, and three from South Korea. It is note­wor­thy that Chi­nese pub­lic in­sti­tu­tions seem to ap­ply for many more patents than those in any other coun­try or re­gion, and by a very sig­nifi­cant mar­gin. The re­port also con­tains a whole chap­ter on the ge­o­graph­i­cal dis­tri­bu­tion of patent filings, which I skimmed but did not in­clude, be­cause the filing lo­ca­tion can be some­what mis­lead­ing. The num­ber of filed patents in a par­tic­u­lar coun­try or re­gion is usu­ally a bet­ter in­di­ca­tor of the re­spec­tive mar­ket for goods or ser­vices re­lat­ing to the patent, as op­posed to the re­search ca­pac­ity be­hind the patent.

  • Ar­tifi­cial In­tel­li­gence – A wor­ld­wide overview of AI patents and patent­ing by the UK AI sec­tor (p. 12f.): The UK In­tel­lec­tual Prop­erty Office finds that among the top 20 ap­pli­cants by to­tal num­ber of AI patent fam­i­lies from 1998–2017, nine are from China, four from the US (IBM and Microsoft in the two top spots), five from Ja­pan, one from South Korea, and one from Ger­many. How­ever, in terms of the to­tal num­ber of ap­pli­cants/​in­ven­tors by coun­try dur­ing the same time pe­riod, the US is clearly in the lead, with at least four times as many patent ap­pli­ca­tions as Ja­pan, ranked sec­ond. China is ranked fourth. Ag­gre­gat­ing the Euro­pean coun­tries in the top 20 yields a to­tal that is ahead of China but still far be­hind the US (see graph be­low). The fact that the anal­y­sis goes back to 1998 makes it less rele­vant for as­sess­ing cur­rent out­put.

  • China AI Devel­op­ment Re­port 2018 (p. 30): The au­thors provide a graph that shows China lead­ing in terms of to­tal patent count, slightly ahead of the US and pre­sum­ably clearly ahead of Euro­pean coun­tries. How­ever, they do not de­tail their method­ol­ogy and do not cite the ac­tual num­bers. They also seem to count patents based on the re­spec­tive filing office as op­posed to in­ven­tor coun­try/​re­gion, which con­founds any in­fer­ences. All of this makes the re­sults less clear-cut and leads me to give their find­ings sig­nifi­cantly lower weight.

Num­ber of highly cited patent fam­i­lies. The only source on this I could find is the WIPO Tech­nol­ogy Trends 2019 – Ar­tifi­cial In­tel­li­gence re­port (p. 88). They find that US in­sti­tu­tions have filed ~28,000 highly cited patent fam­i­lies, fol­lowed by Ja­pan (~6,000), Ger­many (~1,000), South Korea (~1,000), and China (~1,000). I could not find their defi­ni­tion of “highly cited” or the time pe­riod from which the data are sourced. I would not give this a lot of weight, but the mar­gin of the US lead is still note­wor­thy.


  1. Peo­ple in the effec­tive al­tru­ism com­mu­nity might be par­tic­u­larly in­ter­ested in in­di­ca­tors that track AI R&D as it re­lates to the de­vel­op­ment of trans­for­ma­tive AI (TAI). How­ever, this is out­side the scope of this post. I should note that out of the in­di­ca­tors in­cluded in this post, sci­en­tific pub­li­ca­tions are prob­a­bly more rele­vant than patents be­cause the former are more likely to in­clude foun­da­tional break­throughs, which seem to be a bet­ter in­di­ca­tor for R&D rele­vant for TAI. Similarly, highly cited pub­li­ca­tions are likely more in­for­ma­tive than the to­tal num­ber of pub­li­ca­tions. ↩︎

  2. This re­port is au­thored by the China In­sti­tute for Science and Tech­nol­ogy Policy at Ts­inghua Univer­sity. I am not in a po­si­tion to as­sess the ex­tent to which this re­port is shaped by the in­ter­ests of the Chi­nese gov­ern­ment. Ts­inghua Univer­sity is one of the most rep­utable uni­ver­si­ties in China and a lead­ing in­sti­tu­tion in terms of AI re­search in China. ↩︎

  3. This re­port is au­thored by the China Academy of In­for­ma­tion and Com­mu­ni­ca­tions Tech­nol­ogy (CAICT). While it is an or­gan of the Chi­nese gov­ern­ment and “sub­or­di­nate to the pow­er­ful Ministry of In­dus­try and In­for­ma­tion Tech­nol­ogy (MIIT)” (New Amer­ica), I am not in a po­si­tion to as­sess the ex­tent to which this re­port is shaped by the in­ter­ests of the Chi­nese gov­ern­ment. It is no­table that even though they used a very similar method­ol­ogy to the China AI Devel­op­ment Re­port 2018, they seem to in­clude far fewer pa­pers, and the UK does not seem to make it into the top 5 (con­trary to the China AI Devel­op­ment Re­port 2018). All of this makes me some­what skep­ti­cal about this source. ↩︎

  4. From Wikipe­dia: “The Allen In­sti­tute for Ar­tifi­cial In­tel­li­gence (ab­bre­vi­ated AI2) is a re­search in­sti­tute founded by late Microsoft co-founder Paul Allen. The in­sti­tute seeks to achieve sci­en­tific break­throughs by con­struct­ing AI sys­tems with rea­son­ing, learn­ing, and read­ing ca­pa­bil­ities.” ↩︎

  5. Since I do not know their method­ol­ogy, it is not clear if this in­cludes Chi­nese-Amer­i­cans with Chi­nese names or only au­thors from Chi­nese in­sti­tu­tions, etc. ↩︎

  6. I did this to save time while in­clud­ing the most cut­ting-edge re­search. This, how­ever, might ex­clude pa­pers that were break­throughs at the time but have since been su­per­seded. An­a­lyz­ing all en­tries would solve this is­sue. ↩︎

No comments.