Your research is in all likelihood built upon decades, or perhaps centuries of academic research. Researchers actively browse the academic literature in the areas that they are interested in. If you do not publish your research in a fashion that is noticed by academic databases such as Google Scholar, you will lose a lot of readers. Furthermore, these researchers might have used your research as a building block for their research. In other words, you are actively slowing down future academic research.
The vast majority of academic contributions get read by approximately no one. The average high-effort LessWrong post or EA Forum post likely gets an order of magnitude more readers than they would if the same effort were put into an academic paper.
Almost nobody searches academic databases, even academics and researchers mostly use normal Google search when researching things.
Academia has benefits, but reaching a larger audience is not one of them, as far as I can tell (there are of course exceptions and some publications are much better suited for being published in a journal than a blogpost, but by and large academia does not have a good way of actually driving readership).
Academia has benefits, but reaching a larger audience is not one of them, as far as I can tell (there are of course exceptions and some publications are much better suited for being published in a journal than a blogpost, but by and large academia does not have a good way of actually driving readership).
I agree that most academic research is a bad ROI but I find that a lot of this sort of ‘nobody reads research’ commentary is equating reads with citations which seems completely wrong. By that metric most forum posts would also not be read by anyone.
I do think it is somewhat fair to say that you will miss out on audiences if you only post on a forum because a forum post probably won’t reach certain audiences.
Almost nobody searches academic databases, even academics and researchers mostly use normal Google search when researching things.
This has not been my experience as an academic. For several years my job was to trying to find and synthesise evidence for industry/government decision makers. Those reports involved searching academic databases and largely omitted unpublished literature and never included any forum posts, as far as I recall. I think that this was also true of my colleagues.
Literature reviews are also published and read a lot and these usually do not include unpublished work.
I think we should maybe consider doing some research to test the reach and credibility of different communication modalities to have a better sense of when they are worthwhile. For instance, an experiment to test if people (ideally policymakers or similar demographics) differ greatly in their assessments of content in different formats (e.g., preprint v forum post). Or to see where key decision makers get their information from and what they trust.
Otherwise, we run the risk that a lot of very detailed and helpful content on places like LW and EAF is largely overlooked by people outside those communities, including many important decision makers.
I agree that most academic research is a bad ROI but I find that a lot of this sort of ‘nobody reads research’ commentary is equating reads with citations which seems completely wrong. By that metric most forum posts would also not be read by anyone.
I agree-for one, the studies I’ve seen saying that the median publication is not cited are including conference papers, so if one is talking about the peer-reviewed literature, citations are significantly greater. I’ve estimated the average number of citations per paper is around 30 for the peer-reviewed literature. Furthermore, from what I’ve seen, the number of reads on places like ResearchGate and Academia.edu tend to be one to two orders of magnitude greater than the number of citations. So I think a reasonable expectation for a peer-reviewed paper is hundreds or thousands of reads.
The academic publishing system is deeply flawed and we need to push for change from an EA standpoint.
As of now, we are losing thousands upon thousands of top-grade, peer-reviewed research articles that are stuck in obscure journals or self-published and read by nobody. This is due to the high cost of translation, publication, and open access fees, which are particularly prohibitive for researchers in the third world. We simply can’t afford to publish in high-impact journals unless we do self-funding or beg for foreign funds.
I believe the EA community has compassion for the injustice in the world, and I believe that we can be the push for that societal change. We need to prioritize knowledge and information above the corporate greed of publishers.
I completely agree that academic publishing is completely flawed and incentives are a mess. However, anyone that has actually sought to improve upon that must confess that changing this is difficult and will take time. I think it is an important cause.
Not publishing in academic fashion is not a viable or even working solution to the problem of academic publishing.
I understand, It’s a huge task to replace the 150 year systems set in place, we can’t do it overnight, but I do believe that not enough academics talk about it. I am not sure why.
I agree with you on the second point too, I have never source-cited anything that wasn’t peer reviewed and published in a journal, I know other academics will agree. I was just giving a perspective on why academic writers in general might go through the alternative routes of publishing.
Almost nobody searches academic databases, even academics and researchers mostly use normal Google search when researching things.
As an academic and researcher, I don’t agree with this. I will use regular google or wikipedia if I want the basics of a subject, or am researching something that has no related academic field. If I want high quality research on an established field, I will use google scholar, and from there, look through the citations of relevant papers to find other relevant research.
The vast majority of academic contributions get read by approximately no one. The average high-effort LessWrong post or EA Forum post likely gets an order of magnitude more readers than they would if the same effort were put into an academic paper.
This is the wrong comparison. The counterfactual case here is not “write an academic paper and don’t tell anyone about it”, it’s “write an academic paper, then link it on lesswrong/EAforum”. In the latter case, you get both general public reads and academic searchability.
The other point is that we don’t care about every read equally. If you can get the worlds leading expert in a subject to engage with your research, the result will be orders of magnitude more productive in terms of feedback and future research than the average Lesswrong user who doesn’t really understand what you’re talking about. Very, very few subjects have their leading experts reading this forum.
As an academic and researcher, I don’t agree with this. I will use regular google or wikipedia if I want the basics of a subject, or am researching something that has no related academic field.
Yep, agree that if you are writing things that could straightforwardly fit into an academic field, publication is often pretty decent (though I still expect that people will find the average engagement their paper gets to be highly disappointing, given the levels of effort involved).
My sense is only a very small minority of work on the EA Forum or LessWrong would straightforwardly fit into an existing scientific field.
The counterfactual case here is not “write an academic paper and don’t tell anyone about it”, it’s “write an academic paper, then link it on lesswrong/EAforum”. In the latter case, you get both general public reads and academic searchability.
I agree this helps a bit, though also beware that link posts, especially to long pdfs with bad readability on mobile also tend to lose about 80% of their readership on LW and the EA Forum.
Yeah, I agree with these points. To me the best practice would be:
If the research doesn’t really fit within an established field or journal, write it up as a post.
If the research does fit, do a cost benefit analysis on whether it’s worth the extra effort to make it publication worthy. Remember that the benefits aren’t just in terms of academic readership: the process of making it publication worthy may allow you to spot mistakes or shortcomings, which are also valuable.
If you do try and publish, put it on Arxiv or equivalent and link it on the forum, ideally with a good, readable summary of the findings.
I would say this is largely true, but there are a number of notable exceptions where publishing gets more views and is more influential than writing a forum post. Here are just a couple.
The article is generally interesting with obvious real life application either intellectually or practically, even if the journal isn’t super high impact
For example this article on health mapping in Uganda is published in a pretty low ranked journal, but is cited by an impressive 40 people and is part of the reason why I connected with the author to make our health AIM tool which has huge potential to impact healthcare provision in remote areas.
You are published in a top rated journal. In this case readership will be orders of magnitude higher than a forum like this, and potentially orders of magnitude more influential. Obviously you need to make a seriously good discovery to be published in science, or the lancet, or the new England journal of medicine etc.
Yes, it is true that a forum post may get you more readers, and more importantly perhaps, a select readership. But I am advocating for also publishing your research in a way so that it shows up in a Google Scholar query. You should probably publish it here too, if it is important.
I do a bit of research myself and my impression is that most researchers rarely use a plain Google search, but rather use Google Scholar, Scopus or tools like ResearchRabbit and Elicit. None of these returns non-scholarly articles.
Furthermore, it is not only about reaching a larger audience, but rather making your research accessible to those who search. Publishing to ArXiv allows for someone who is actively searching for your article to find it. Conversely, the article is of most value to these readers. Lastly, ArxiV and other scholarly servers are more likely to be used for decades into the future, increasing the number of readers in the future relative to other options.
The vast majority of academic contributions get read by approximately no one. The average high-effort LessWrong post or EA Forum post likely gets an order of magnitude more readers than they would if the same effort were put into an academic paper.
Almost nobody searches academic databases, even academics and researchers mostly use normal Google search when researching things.
Academia has benefits, but reaching a larger audience is not one of them, as far as I can tell (there are of course exceptions and some publications are much better suited for being published in a journal than a blogpost, but by and large academia does not have a good way of actually driving readership).
I agree that most academic research is a bad ROI but I find that a lot of this sort of ‘nobody reads research’ commentary is equating reads with citations which seems completely wrong. By that metric most forum posts would also not be read by anyone.
I do think it is somewhat fair to say that you will miss out on audiences if you only post on a forum because a forum post probably won’t reach certain audiences.
This has not been my experience as an academic. For several years my job was to trying to find and synthesise evidence for industry/government decision makers. Those reports involved searching academic databases and largely omitted unpublished literature and never included any forum posts, as far as I recall. I think that this was also true of my colleagues.
Literature reviews are also published and read a lot and these usually do not include unpublished work.
I think we should maybe consider doing some research to test the reach and credibility of different communication modalities to have a better sense of when they are worthwhile. For instance, an experiment to test if people (ideally policymakers or similar demographics) differ greatly in their assessments of content in different formats (e.g., preprint v forum post). Or to see where key decision makers get their information from and what they trust.
Otherwise, we run the risk that a lot of very detailed and helpful content on places like LW and EAF is largely overlooked by people outside those communities, including many important decision makers.
I agree-for one, the studies I’ve seen saying that the median publication is not cited are including conference papers, so if one is talking about the peer-reviewed literature, citations are significantly greater. I’ve estimated the average number of citations per paper is around 30 for the peer-reviewed literature. Furthermore, from what I’ve seen, the number of reads on places like ResearchGate and Academia.edu tend to be one to two orders of magnitude greater than the number of citations. So I think a reasonable expectation for a peer-reviewed paper is hundreds or thousands of reads.
The academic publishing system is deeply flawed and we need to push for change from an EA standpoint.
As of now, we are losing thousands upon thousands of top-grade, peer-reviewed research articles that are stuck in obscure journals or self-published and read by nobody. This is due to the high cost of translation, publication, and open access fees, which are particularly prohibitive for researchers in the third world. We simply can’t afford to publish in high-impact journals unless we do self-funding or beg for foreign funds.
I believe the EA community has compassion for the injustice in the world, and I believe that we can be the push for that societal change. We need to prioritize knowledge and information above the corporate greed of publishers.
I completely agree that academic publishing is completely flawed and incentives are a mess. However, anyone that has actually sought to improve upon that must confess that changing this is difficult and will take time. I think it is an important cause.
Not publishing in academic fashion is not a viable or even working solution to the problem of academic publishing.
I understand, It’s a huge task to replace the 150 year systems set in place, we can’t do it overnight, but I do believe that not enough academics talk about it. I am not sure why.
I agree with you on the second point too, I have never source-cited anything that wasn’t peer reviewed and published in a journal, I know other academics will agree. I was just giving a perspective on why academic writers in general might go through the alternative routes of publishing.
Just saw this from @soroushjp—it seems very relevant.
As an academic and researcher, I don’t agree with this. I will use regular google or wikipedia if I want the basics of a subject, or am researching something that has no related academic field. If I want high quality research on an established field, I will use google scholar, and from there, look through the citations of relevant papers to find other relevant research.
This is the wrong comparison. The counterfactual case here is not “write an academic paper and don’t tell anyone about it”, it’s “write an academic paper, then link it on lesswrong/EAforum”. In the latter case, you get both general public reads and academic searchability.
The other point is that we don’t care about every read equally. If you can get the worlds leading expert in a subject to engage with your research, the result will be orders of magnitude more productive in terms of feedback and future research than the average Lesswrong user who doesn’t really understand what you’re talking about. Very, very few subjects have their leading experts reading this forum.
Yep, agree that if you are writing things that could straightforwardly fit into an academic field, publication is often pretty decent (though I still expect that people will find the average engagement their paper gets to be highly disappointing, given the levels of effort involved).
My sense is only a very small minority of work on the EA Forum or LessWrong would straightforwardly fit into an existing scientific field.
I agree this helps a bit, though also beware that link posts, especially to long pdfs with bad readability on mobile also tend to lose about 80% of their readership on LW and the EA Forum.
Yeah, I agree with these points. To me the best practice would be:
If the research doesn’t really fit within an established field or journal, write it up as a post.
If the research does fit, do a cost benefit analysis on whether it’s worth the extra effort to make it publication worthy. Remember that the benefits aren’t just in terms of academic readership: the process of making it publication worthy may allow you to spot mistakes or shortcomings, which are also valuable.
If you do try and publish, put it on Arxiv or equivalent and link it on the forum, ideally with a good, readable summary of the findings.
I would say this is largely true, but there are a number of notable exceptions where publishing gets more views and is more influential than writing a forum post. Here are just a couple.
The article is generally interesting with obvious real life application either intellectually or practically, even if the journal isn’t super high impact
For example this article on health mapping in Uganda is published in a pretty low ranked journal, but is cited by an impressive 40 people and is part of the reason why I connected with the author to make our health AIM tool which has huge potential to impact healthcare provision in remote areas.
https://equityhealthj.biomedcentral.com/articles/10.1186/s12939-020-01371-5
You are published in a top rated journal. In this case readership will be orders of magnitude higher than a forum like this, and potentially orders of magnitude more influential. Obviously you need to make a seriously good discovery to be published in science, or the lancet, or the new England journal of medicine etc.
Yes, it is true that a forum post may get you more readers, and more importantly perhaps, a select readership. But I am advocating for also publishing your research in a way so that it shows up in a Google Scholar query. You should probably publish it here too, if it is important.
I do a bit of research myself and my impression is that most researchers rarely use a plain Google search, but rather use Google Scholar, Scopus or tools like ResearchRabbit and Elicit. None of these returns non-scholarly articles.
Furthermore, it is not only about reaching a larger audience, but rather making your research accessible to those who search. Publishing to ArXiv allows for someone who is actively searching for your article to find it. Conversely, the article is of most value to these readers. Lastly, ArxiV and other scholarly servers are more likely to be used for decades into the future, increasing the number of readers in the future relative to other options.