Alternatively, AI academics might be becoming more sociable – i.e. citing their friends’ papers more, and collaborating more on papers. I don’t find either of the explanations particularly convincing.
FWIW, I find this somewhat convincing. I think the collaborating on papers part seems like it could be downstream of the expectations of # of paper produced being higher. My sense is that grad students are expected to write more papers now than they used to. One way to accomplish this is to collaborate more.
I expect if you compared data on the total number of researchers in the AI field and the number of papers, you would see the second rising a little faster than the first (I think I’ve seen this trend, but don’t have the numbers in front of me). If these were rising at the same rate, I think it would basically indicate no change in the difficulty of ideas, because research hours would be scaling with # papers. Again, I expect the trend is actually papers rising faster than people, which would make it seem like ideas are getting easier to find.
I think other explanations, like the norms and culture around research output expectation, collaboration, how many references you have to have, are more to blame.
Overall I don’t find the methodology presented here, of just looking at number of authors and number of references, to be particularly useful for figuring out if ideas are getting harder to find. It’s definitely some evidence, but I think there’s quite a few plausible explanations.
I’m sorry it’s taken me a little while to get back to you.
In hindsight, the way I worded this was overly strong. Cultural explanations are possible, yes.
I guess I see this evidence as a weak update on a fairly strong prior that the burden of knowledge (BOK) is increasing – given the range of other variables (e.g., age of innovation, levels of specialisation), and similar trends within patent data. For example, you couldn’t attribute increasing collaboration on patents to norms within academia.
I’d be interested to compare # researchers with # papers. The ratio of these two growth rates is key, for the returns to research parameter from Bloom et al. Do send me this, if you have remembered in the intervening time.
I don’t have the time right now to find exactly which comparison I am thinking of, but I believe my thought process was basically “the rate of new people getting AI PhDs is relatively slow”; this is of course only one measure for the number of researchers. Maybe I used data similar to that here: https://www.lesswrong.com/s/FaEBwhhe3otzYKGQt/p/AtfQFj8umeyBBkkxa
FWIW, I find this somewhat convincing. I think the collaborating on papers part seems like it could be downstream of the expectations of # of paper produced being higher. My sense is that grad students are expected to write more papers now than they used to. One way to accomplish this is to collaborate more.
I expect if you compared data on the total number of researchers in the AI field and the number of papers, you would see the second rising a little faster than the first (I think I’ve seen this trend, but don’t have the numbers in front of me). If these were rising at the same rate, I think it would basically indicate no change in the difficulty of ideas, because research hours would be scaling with # papers. Again, I expect the trend is actually papers rising faster than people, which would make it seem like ideas are getting easier to find.
I think other explanations, like the norms and culture around research output expectation, collaboration, how many references you have to have, are more to blame.
Overall I don’t find the methodology presented here, of just looking at number of authors and number of references, to be particularly useful for figuring out if ideas are getting harder to find. It’s definitely some evidence, but I think there’s quite a few plausible explanations.
Hi Aaron,
I’m sorry it’s taken me a little while to get back to you.
In hindsight, the way I worded this was overly strong. Cultural explanations are possible, yes.
I guess I see this evidence as a weak update on a fairly strong prior that the burden of knowledge (BOK) is increasing – given the range of other variables (e.g., age of innovation, levels of specialisation), and similar trends within patent data. For example, you couldn’t attribute increasing collaboration on patents to norms within academia.
I’d be interested to compare # researchers with # papers. The ratio of these two growth rates is key, for the returns to research parameter from Bloom et al. Do send me this, if you have remembered in the intervening time.
Charlie
I don’t have the time right now to find exactly which comparison I am thinking of, but I believe my thought process was basically “the rate of new people getting AI PhDs is relatively slow”; this is of course only one measure for the number of researchers. Maybe I used data similar to that here: https://www.lesswrong.com/s/FaEBwhhe3otzYKGQt/p/AtfQFj8umeyBBkkxa