Hi Jeff,
Thanks for your comment :) I totally agree with Larissa’s response, and also really liked your example about instrument building.
I’ve been working with Kerry at Leverage to set-up the early stage science research program by doing background reading on the academic literature in history and philosophy of science, so I’m going to follow-up on Larissa’s comment to respond to the question raised in your 3rd bullet point (and the 4th bullet in a later comment, likely tomorrow).
Large hedge: this entire comment reflects my current views—I expect my thinking to change as we keep doing research and become more familiar with the literature.
“How controversial is the idea that early stage science works pretty differently from more established explorations, and that you need pretty different approaches and skills? I don’t know that much history/philosophy of science but I’m having trouble telling from the paper which of the hypotheses in section 4 are ones that you expect people to already agree with, vs ones that you think you’re going to need to demonstrate?”
This is a great question. The history and philosophy of science literature is fairly large and complicated, and we’ve just begun looking at it, but here is my take so far.
I think it’s somewhat controversial that early stage science works pretty differently from more established explorations and that you need pretty different approaches and skills. It’s also a bit hard to measure, because our claims slightly cross-cut the academic debate.
Summary of our position
To make the discussion a little simpler, I’ve distilled down our hypothesis and starting assumptions to four claims to compare to positions in the surrounding literature[1]. Here are the claims:
(1) You can learn[2] about scientific development and methodology via looking at history.
(2) Scientific development in a domain tends to go through phases, with distinct methodologies and patterns of researcher behavior.
(3) There is an early phase of science that uses less well-established and very different methods than the later phases of science (following on claim (2)).
(4) We can study the early phase of science to figure out how these methods work and use them to make methodological recommendations (following on (1) and (3)).
We take (1) as an assumption, whereas (2)-(4) are elements of our starting hypothesis. As a result, we’ll aim to defend (2)-(4) in the course of our research.
I’ve included a super brief overview of my understanding of the literature below and our position with respect to it, but I worry/expect that I didn’t summarize the perspectives enough to be particularly useful for people who haven’t read Kuhn, Popper, or responses to them. As a result, I’ve tried to answer the original question without going into detail on the literature and then gone into more detail below.
Summary of controversiality of our position in the literature
I’d say that claims (1), (2), and (3) are typically relatively uncontroversial to people who agree with Thomas Kuhn’s ideas in The Structure of Scientific Revolutions. Kuhn’s work is accepted by many people in academia and the mainstream, but there are also a lot of people in academia who dispute it. Of the people who disagree with Kuhn, most disagree with (1), and then (2) (and therefore (3) and (4)). I find their objections to (2) more confusing than their objections to (1), and will need to look into it more. Kuhn himself disagrees with claim (4).
We want our research to (at least initially) avoid getting into the debate directly around (1) (the use of historical case studies to learn about scientific development and make methodological conclusions), largely because it’s a bit outside of scope for us. I expect our work to lose some people, including some academics, due to not believing that we can learn about scientific methodology from history. That said, we’re hopeful that as we try to look at history to learn about methodology, we’ll be able to tell if the entire endeavor doesn’t make sense. We’ll try to be clean about distinguishing what our investigation is indicating we can learn from history and whether that should be generalized. If it turns out we need to investigate and discuss (1) more directly, we’ll do that later. So, for now, we will not justify (1) directly.
Due to all the disagreement around (2), (3), and (4), we’re going take those as hypotheses that we’ll need to demonstrate and justify.
Rather than justifying (2) directly off the bat (e.g., taking a random sampling of cases throughout history and comparing them), we’ll build a methodological model of how discovery works in (4) and then see whether that model predicts that there would be different methods used later which would imply phases of scientific development. If so, we’ll then expand our investigation into cases likely to fall into later phases of science, to see if the model’s predictions hold. We’re hoping that this method will supplement Kuhn’s way of investigating phases of scientific development, let us build a narrower (and hopefully more checkable) initial model, and let us focus most on (4), which has had the least academic attention.
Hope that helps!
Details of our initial take on the literature
This section describes my take on the debate that I’ve situated us with respect to above, but (mostly for length) doesn’t go into detail on the researcher positions (e.g., what paradigms are or what Popper argues about falsification). For an initial overview of positions in the space, I recommend checking out: https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions, https://en.wikipedia.org/wiki/Falsifiability, https://www.jstor.org/stable/686690, http://philsci-archive.pitt.edu/15876/1/A%20New%20Functional%20Approach%20to%20Scientific%20Progress%20-%20preprint.pdf(The latter aren’t designed as overviews, but I’ve found them helpful to get a sense of the discussion in philosophy of science)
The specific academic research area most relevant to the work we’re doing that I’ve found so far is a set of research done at the intersection between history of science and philosophy of science, primarily done between 1960 and 1990. A lot of the work in the field since then is still centered on and generated by this debate. Important figures in the debate include Kuhn, Lakatos, Feyerabend, Smart, and Popper. I expect to find discussion in other areas relevant to what we’re doing, but I’ve mostly investigated this debate so far, as it seemed most central.
Of this work, our hypothesis is most similar to Kuhn’s thinking in The Structure of Scientific Revolutions. I read Kuhn as saying and implying[3]: you can learn about scientific development and methodology by investigating history, there are phases of scientific development, and some phases work differently than other phases with respect to the methods used and researcher behavior.
Kuhn’s ontology differs from ours a bit. In Kuhn’s work, the phases he thinks are different than established explorations are “pre-science” and the period after “anomaly” and “crisis.” I expect our early stage science hypothesis to apply at least in what Kuhn calls “pre-science,” but also possibly at other points in the development of fields (e.g., in some cases after “anomaly” and “crisis”). I take this to mean that Kuhn agrees with a narrower claim than (3). It is also possible that examining the different ontologies will turn up further disagreements.
The clearest disagreement is with respect to (4). Kuhn is quoted (both in his book and verbally) as saying that he considered the methodology used in pre-science to be mysterious, possibly not uniform, and (implied) not fruitful to study. I take him to be saying that there is a distinction between the methodology used in phases of science, but we can’t understand how the earliest phases of discovery work mechanistically. This means he disagrees with (4).[4]
So, on this reading, Kuhn agrees with (1), (2), and parts of (3) of our claims above, but (at least importantly) disagrees with (4). A significant portion of academics (and mainstream audiences) agree with and respect Kuhn, so in that sense, a bunch of our hypothesis is less controversial.
However, lots of people disagree with Kuhn. The best way I’ve found to explain this so far centers around Karl Popper and falsifiability.
Specifically, because of work done by Popper around falsifiability, there’s a lot of debate about claim (1) listed above—whether history can be used to make claims about how science should, does, or has worked. Since Kuhn wrote his book, his methodology has become even more controversial due to questions about the falsifiability of his conclusions. There’s a sizeable contingent of people in the history and philosophy of science that thus don’t use historical cases to reach methodological conclusions and argue against methodologies that do so.
Also, Popper’s views on falsification have created object-level skepticism about Kuhn’s view that science works on the basis of paradigms. You could imagine Popper saying about Kuhn: “Normatively speaking, good science must avoid the pitfalls of unfalsifiable claims, and paradigms don’t work that way, therefore Kuhn is wrong in his description of history or historical science wasn’t making scientific progress.” I want to hedge here that I find interpreting Popper and his effect on surrounding science very difficult, so this interpretation is somewhat likely to be wrong[5]. This debate has spawned many other different interpretations of how to measure, describe, and chunk scientific progress (see Laudan’s functionalism and Lakatos’ Research Programmes, as examples).
As a result of the object-level debate on paradigms and the methodological debate about history, some researchers don’t affirm phases of science (either because we can’t make conclusions like that from history or because they disagree on the object-level), and others agree that there are phases but disagree with the validity of using Kuhnian paradigms to demarcate the phases. This means claim (2) isn’t accepted enough that we can take it and run with it. Because claim (3) and (4) rest on claim (2), this means we’ll need to go into (2), (3), and (4).
—
Notes:
[1] Our current hypothesis actually has 6 claims, and doesn’t include starting assumption (1). I’ve found the version I included in this comment simpler to talk about, but less clear and detailed. Check out the paper for the full hypothesis.
[2] This is overly general, sorry about that. The debate on what can be learned from history (descriptive and normative) is complicated, and I want to largely skip it for now. There is a bit more information in the section below on details of the surrounding literature, but not much. We expect to later write a paper that situates us in the surrounding literature, which will clarify.
[3] It’s a bit tricky because he published work in the 1990s that some philosophers think rescinded his previous views and some do not. This might mean that he back-tracked on (2), though I currently don’t interpret his comments this way.
[4] Kuhn also believes in paradigm incommensurability, which we don’t affirm and don’t want to get into off the bat.
[5] A lot of work has gone into trying to square Popper and Kuhn’s perspectives by arguing that people have misinterpreted either Kuhn or Popper. People differ widely in how sophisticated a claim they view Popper as making about falsification, and how ambiguous they take Kuhn’s description of paradigms to be. Lakatosian Research Programmes are a potential example of a way to square paradigms and falsifiability.
“One question that comes to mind is whether there is still early stage science today. Maybe the patterns that you’re seeing are all about what happens if you’re very early in the development of science in general, but now you only get those patterns when people are playing around (like I am above)? So I’d be interested in the most recent cases you can find that you’d consider to be early-stage.”
This is also a great question.
It is totally possible that early stage science occurred only in the past, and science as a whole has developed past it. We talked to a number of people in our network to try to gather plausible alternatives to our hypothesis about early stage science, and this is one of the most common ones we found. I’m currently thinking of this as one of the possible views we’ll need to argue against or refute for our original hypothesis to hold, as opposed to a perspective we’ve already solidly eliminated.
On recent past cases:
If you go back a bit, there are lots of plausible early stage science success cases in the late 1800s and early 1900s. The study of radiation is a potential case in this period with some possible indicators of early stage methods. This period is arguably not recent enough to refute the “science as a whole has moved past early stage exploration” hypothesis, so I want to seek out more recent examples in addition to studying these.
To get a better answer here, I’ll want us to look more specifically at the window between 1940 and 2000, which we haven’t looked at much so far—I expect it will be our best shot at finding early stage discoveries that have already been verified and accepted, while still being recent.
On current cases:
Finding current cases or cases in the more recent past is trickier. For refuting the hypothesis you laid out, we’d be most interested in finding recent applications of early stage methods that produced successful discoveries. Unfortunately, it can be hard to identify these cases, because when the early research is still happening, it’s often still unclear if it’s on track to being successful.
That said, we think it is possible to identify areas that are potentially early stage science. This is a pretty different activity from looking at more confirmed success cases, but it’s something we’re looking into.