I’m a bit confused by your distinction: the question “Did […]
If you can’t find reliable data, that just makes it hard, not theoretical.
The use of “did” vs “would” wasn’t very intentional or precise.
As to the empirical vs. theoretical nature of my hypothesis, it is indeed claiming that certain relationships empirically existed (and, with a lot of caveats, may continue into the future). However, my point was that the research methods I used were much more “theoretical”: I couldn’t do a large-N empirical analysis or controlled experiments to even establish meaningfully-controlled correlation (let alone causation) between the dependent, control, and independent variables, and instead had to rely on lines of reasoning such as:
Hypothetical scenarios (e.g., imagine comparing an ambush where both parties have machine guns vs. one where neither side has machine guns)—which is impractical to clinically/experimentally test (I.e., with high reality fidelity)
More-qualitative (and somewhat subjective) comparison of case studies, using a large amount of argumentation/theoretical reasoning to deal with the many gaps and flaws in the case comparison (given that, as I noted, there didn’t seem to be any good case comparison pair in the historical record)
Agreement with existing theoretical and/or empirical concepts in the literature, such as Biddle’s Modern System.
it basically just seems to say “think about the problem until you can figure out how to test it with traditional empirical methods.”
Well, yeah, what else would you expect? The post describes how you might use argument clashes and oversimplified simulations in thinking about the problem.
Again, perhaps I was being a bit too imprecise with my language? My point is that for some questions (arguably including my thesis), theoretical argumentation has to bear a lot of the analytical burden. This analytical burden can include things like:
Explaining why variables Q, K, and W—none of which you could experimentally control for—probably do or don’t affect the relationship;
Explaining why your very limited sample size can probably be extrapolated to some other cases;
Explaining why some metric is probably a decent proxy for what you actually are trying to measure;
Reasoning about hypothetical scenarios which will not actually empirically occur.
(Caveat: all of those activities can be supported by direct reference to supporting data in some situations, but not always.)
In contrast, it seems that much of the “theoretical” research methods described in this post are basically just “use lots of thinking to figure out how to test this empirically against data [at which point these empirical methods do almost all the legwork.]”
There is perhaps some debate to be had over the meaning of “theoretical” research methods: do mathematical proofs or algorithms count as theory? While I’m not universally opposed to using the term in such a context, I think it is much less helpful to use the term “theory” when you’re trying to juxtapose it with empirical methods. This especially feels true if a major reason you support a mathematical proof or algorithm is based on your determination that “this empirically works every single time.” When teaching research methods, I think it’s important to emphasize the differences that I described previously (e.g., legibility/transparency, reliability/consistency, reputation stake) which, in my view, have tended to make empirical methods so much more effective when they can be used.
The use of “did” vs “would” wasn’t very intentional or precise.
As to the empirical vs. theoretical nature of my hypothesis, it is indeed claiming that certain relationships empirically existed (and, with a lot of caveats, may continue into the future). However, my point was that the research methods I used were much more “theoretical”: I couldn’t do a large-N empirical analysis or controlled experiments to even establish meaningfully-controlled correlation (let alone causation) between the dependent, control, and independent variables, and instead had to rely on lines of reasoning such as:
Hypothetical scenarios (e.g., imagine comparing an ambush where both parties have machine guns vs. one where neither side has machine guns)—which is impractical to clinically/experimentally test (I.e., with high reality fidelity)
More-qualitative (and somewhat subjective) comparison of case studies, using a large amount of argumentation/theoretical reasoning to deal with the many gaps and flaws in the case comparison (given that, as I noted, there didn’t seem to be any good case comparison pair in the historical record)
Agreement with existing theoretical and/or empirical concepts in the literature, such as Biddle’s Modern System.
Again, perhaps I was being a bit too imprecise with my language? My point is that for some questions (arguably including my thesis), theoretical argumentation has to bear a lot of the analytical burden. This analytical burden can include things like:
Explaining why variables Q, K, and W—none of which you could experimentally control for—probably do or don’t affect the relationship;
Explaining why your very limited sample size can probably be extrapolated to some other cases;
Explaining why some metric is probably a decent proxy for what you actually are trying to measure;
Reasoning about hypothetical scenarios which will not actually empirically occur.
(Caveat: all of those activities can be supported by direct reference to supporting data in some situations, but not always.)
In contrast, it seems that much of the “theoretical” research methods described in this post are basically just “use lots of thinking to figure out how to test this empirically against data [at which point these empirical methods do almost all the legwork.]”
There is perhaps some debate to be had over the meaning of “theoretical” research methods: do mathematical proofs or algorithms count as theory? While I’m not universally opposed to using the term in such a context, I think it is much less helpful to use the term “theory” when you’re trying to juxtapose it with empirical methods. This especially feels true if a major reason you support a mathematical proof or algorithm is based on your determination that “this empirically works every single time.” When teaching research methods, I think it’s important to emphasize the differences that I described previously (e.g., legibility/transparency, reliability/consistency, reputation stake) which, in my view, have tended to make empirical methods so much more effective when they can be used.