[Summary: Most people would probably agree that science benefited greatly from the shift to structured, rigorous empirical analyses over the past century, but some fields still struggle to make progress. I’m curious whether people think that we could/should seek to introduce more structure/sophistication to the way researchers make and engage with theoretical analyses, such as something like “epistemic mapping”]
I just discovered this post, and I was struck by how it echoed some of my independent thoughts and impressions, especially the quote: “But it should temper our enthusiasm about how many insights we can glean by getting some data and doing something sciency to it.”
(What follows is shortform-level caveating and overcomplicating, which is to say, less than I normally would provide, and more about conveying the overall idea/impression)
I’ve had some (perhaps hedgehoggy) “big ideas” about the potential value of what I call “epistemic mapping” for advancing scientific study/inquiry/debate in a variety of fields. One of them relates to the quote above: the “empirical-scientific revolution” of the past ~100-200 years (e.g., the shift to measuring medical treatment effectiveness through inpatient/outpatient data rather than professionals’ impressions) seems to have been crucial in the advancement of a variety of fields.
However, there are still many fields where such empirical/data-heavy methods appear insufficient and where it seems like progress languishes: my impression has been that this especially includes many of the social sciences (e.g., conflict studies, political science, sociology). There are no doubt many possible explanations, but over time I’ve increasingly wondered whether a major set of problems is loosely that the overall complexity of the systems (e.g., human decision making process vs. gravitational constants) + the difficulty of collecting sufficient data for empirical analyses + (a few other factors) leads to a situation of high information lossage between researchers/studies and/or people are incentivized to oversimplify things (e.g., following the elsewhere-effective pattern of regression analyses and p<0.05 = paper). I do not know, but if the answer is yes, that leads to a major question:
How could/should we attempt to solve or mitigate this problem? One of the (hedgehoggy?) questions that keeps bugging me: We have made enormous advances in the past few hundred years when it comes to empirical analyses; in comparison, it seems that we have only fractionally improved the way we do our theoretical analysis… could/should we be doing better?[Very interested to get people’s thoughts about that overall characterization, which even I’ll admit I’m uncertain about]
So, I’m curious if people share similar sentiment about our ability/need to improve our methods of theoretical analysis, including how people engage with the broader literature aside from the traditional (and, IMO, inefficient) paragraph-based literature reviews. If people do share similar sentiment, what do you think about that concept of epistemic mapping as a potential way of advancing some sciences forward? Could it be the key to efficient future progress in some fields? My base rates for such a claim are really low, and I recognize that I’m biased, but I feel like it’s worth posing the question if only to see if it advances the conversation.
(I might make this into an official post if people display enough interest)
[Summary: Most people would probably agree that science benefited greatly from the shift to structured, rigorous empirical analyses over the past century, but some fields still struggle to make progress. I’m curious whether people think that we could/should seek to introduce more structure/sophistication to the way researchers make and engage with theoretical analyses, such as something like “epistemic mapping”]
I just discovered this post, and I was struck by how it echoed some of my independent thoughts and impressions, especially the quote: “But it should temper our enthusiasm about how many insights we can glean by getting some data and doing something sciency to it.”
(What follows is shortform-level caveating and overcomplicating, which is to say, less than I normally would provide, and more about conveying the overall idea/impression)
I’ve had some (perhaps hedgehoggy) “big ideas” about the potential value of what I call “epistemic mapping” for advancing scientific study/inquiry/debate in a variety of fields. One of them relates to the quote above: the “empirical-scientific revolution” of the past ~100-200 years (e.g., the shift to measuring medical treatment effectiveness through inpatient/outpatient data rather than professionals’ impressions) seems to have been crucial in the advancement of a variety of fields.
However, there are still many fields where such empirical/data-heavy methods appear insufficient and where it seems like progress languishes: my impression has been that this especially includes many of the social sciences (e.g., conflict studies, political science, sociology). There are no doubt many possible explanations, but over time I’ve increasingly wondered whether a major set of problems is loosely that the overall complexity of the systems (e.g., human decision making process vs. gravitational constants) + the difficulty of collecting sufficient data for empirical analyses + (a few other factors) leads to a situation of high information lossage between researchers/studies and/or people are incentivized to oversimplify things (e.g., following the elsewhere-effective pattern of regression analyses and p<0.05 = paper). I do not know, but if the answer is yes, that leads to a major question:
How could/should we attempt to solve or mitigate this problem? One of the (hedgehoggy?) questions that keeps bugging me: We have made enormous advances in the past few hundred years when it comes to empirical analyses; in comparison, it seems that we have only fractionally improved the way we do our theoretical analysis… could/should we be doing better? [Very interested to get people’s thoughts about that overall characterization, which even I’ll admit I’m uncertain about]
So, I’m curious if people share similar sentiment about our ability/need to improve our methods of theoretical analysis, including how people engage with the broader literature aside from the traditional (and, IMO, inefficient) paragraph-based literature reviews. If people do share similar sentiment, what do you think about that concept of epistemic mapping as a potential way of advancing some sciences forward? Could it be the key to efficient future progress in some fields? My base rates for such a claim are really low, and I recognize that I’m biased, but I feel like it’s worth posing the question if only to see if it advances the conversation.
(I might make this into an official post if people display enough interest)