When performing qualitative research (say, ethnographic analysis from observations and interviews), do experts perform much better than, say, smart and motivated undergrads with a good framework? I guess that designing a good framework is very expertise-dependent, but I am also not sure to what extent and how much this skill is needed (as opposed to there existing enough existing frameworks to choose from adequately).
Or,
Are skills developed while increasing expertise makes one’s result better in the eyes of other expert researchers?
Are expert estimation of the scientific merit of such results in fact indicative of gaining much more valuable information?
Say, psychoanalysis is debatably a related field in which one can become expert in the sense of (1) but perhaps the effects are similar to talking with inexperienced people (so (2) doesn’t hold).
I guess that the answer is a resounding yes, but I don’t really know. Coming from a quantitative background, I don’t feel like I understand it enough.
This question is important to me in order to help someone I’m giving career advice to. Also, I’m generally interested in understanding more about research methods and their use in EA related works.
I think you’re more likely to make simple mistakes or do very naive analysis if you haven’t studied best practices. After doing an MA in Public Policy I spot a lot of mistakes that I wouldn’t have before.
I think this holds true in more traditionally “quantitative” fields, too, because often things can be useful or not depending on how they are framed such that without the proper framing good numbers don’t matter because they are measuring the right thing.
This seems to suggest that a lot of what makes quantitative research successful also makes qualitative research successful, and so we should expect any extent to which expertise matters in quantitative fields to matter in qualitative fields (although I think this mostly points at the quant/qual distinction being a very fuzzy one that is only relevant along certain dimensions).