Great comment David! It made me focus in on the heart of the question here. It is simply this—What is the right counterfactual to EA, the philosophy/academic discipline? OP is comparing EA philosophy/discipline to Econ. But is that fair? When I read your comment this morning, I noticed how I utterly failed to clarify that EA is less rigorous… than what?! It got me thinking empirically and I quickly whipped up some stuff using OpenAlex which is a open-source repository of publications used by bibliometricians. Now this preliminary analysis I show doesn’t resolve the question of what is counterfactual to EA, but it begins to describe the problem better.
If you’re interested, see my GitHub repo with more details on the research design that I imagined and the Python/R code. I wonder if OP will like this because I’m doing stuff that a (design-based) econometrician would do :-)
First up, lets validate what David said makes sense—Are EA publications in top journals?
Yep. David is not wrong. But of course that is not the question! The claim I want to make is that EA publications are more insular than other stuff and the obstacle to making this claim is what the hell is “other stuff”?! This is where OpenAlex topics come in. OpenAlex uses a clsutering+classification pipeline to classify papers as belonging to some topic. Here is a plot showing that:
Now, the next step would be to ask ourselves, “EA has such a spread of topics. But field X has a much wider spread. This is why EA is insular” But what is that field X? EA compared to Deontology? Utilitarianism? These have around for decades—how is that a fair comparison group? What exactly is the benchmark to weigh EA, the discipline, against?
Now maybe the way to do this, is to pull out all the papers in these topics I have plotted above from OpenAlex and compare against those. But I guess a better way to do this would be to pull out abstracts of all publications, clean up, tokenize, cluster and see whats close by and compare against that. Can someone else make this pipeline more concrete?
Great comment David! It made me focus in on the heart of the question here. It is simply this—What is the right counterfactual to EA, the philosophy/academic discipline? OP is comparing EA philosophy/discipline to Econ. But is that fair? When I read your comment this morning, I noticed how I utterly failed to clarify that EA is less rigorous… than what?! It got me thinking empirically and I quickly whipped up some stuff using OpenAlex which is a open-source repository of publications used by bibliometricians. Now this preliminary analysis I show doesn’t resolve the question of what is counterfactual to EA, but it begins to describe the problem better.
If you’re interested, see my GitHub repo with more details on the research design that I imagined and the Python/R code. I wonder if OP will like this because I’m doing stuff that a (design-based) econometrician would do :-)
First up, lets validate what David said makes sense—Are EA publications in top journals?
Yep. David is not wrong. But of course that is not the question! The claim I want to make is that EA publications are more insular than other stuff and the obstacle to making this claim is what the hell is “other stuff”?! This is where OpenAlex topics come in. OpenAlex uses a clsutering+classification pipeline to classify papers as belonging to some topic. Here is a plot showing that:
Now, the next step would be to ask ourselves, “EA has such a spread of topics. But field X has a much wider spread. This is why EA is insular” But what is that field X? EA compared to Deontology? Utilitarianism? These have around for decades—how is that a fair comparison group? What exactly is the benchmark to weigh EA, the discipline, against?
Now maybe the way to do this, is to pull out all the papers in these topics I have plotted above from OpenAlex and compare against those. But I guess a better way to do this would be to pull out abstracts of all publications, clean up, tokenize, cluster and see whats close by and compare against that. Can someone else make this pipeline more concrete?
EDIT: Fixed broken links