I consider the issue of identifying knowledge gaps (and the semi-opposite concept: arguments/experiments which are highly influential in a field) as another good potential use case/benefit of epistemic mapping: both the importance of and lack of research on a concept may become more apparent with visualization—especially if the map/graph shows that the claim or assumption is very influential in the literature but it has not received much support or scrutiny. I’d be curious to get your thoughts on the potential usefulness/viability of such a project for such purposes.
Interesting. I think a challenge would be to find the right level of complexity of a map like that—it needs to be simple enough to give a useful overview, but complex enough that it models everything that’s necessary to make it a good tool for decisionmaking.
Who do you imagine would be the main user of such a mappning? And for which decisions would they mainly use it? I think the requirements would be quite different depending on if it’s to be used by non-experts such as policymakers or grantmakers, or if it’s to be used by researchers themselves?
I agree the complexity level question is a tough question, although my impression has been that it could probably be implemented with varying levels of complexity (e.g., just focusing on simpler/more-objective characteristics like “data source used” or “experimental methodology” vs. also including theoretical arguments and assumptions). I think the primary users would tend to be researchers—who might then translate the findings into more-familiar terms or representations for the policymaker, especially if it does not become popular/widespread enough for some policymakers to be familiar with how to use or interpret it (somewhat similar to regression tables and similar analyses). That being said, I also see it as plausible that some policymakers would have enough basic understanding of the system to engage/explore on their own—like how some policymakers may be able to directly evaluate some regression findings.
Ultimately, two examples of the primary use cases I envision are:
Identifying the ripple effects of changes in assumptions/beliefs/datasets/etc. Suppose for example an experimental finding or dataset which influenced dozens of studies is shown to be flawed: it would be helpful to have an initial outline of what claims and assumptions need to be reevaluated in light of the new finding.
Mapping the debate for a somewhat contentious subject (or just anything where the literature is not in agreement), including by identifying if any claims have been left unsupported or unchallenged.
It seems that such insights might be helpful for a researcher trying to decide what to focus on (and/or a grantmaker trying to decide what research to fund).
Cool—my immediate thought is that it would be interesting to see a case study of (1) and/or (2) - do you know of this being done for any specific case?
Perhaps we could schedule a call to talk further—I’ll send you a DM!
I also mention that I think that mapping could be very useful, particularly for onboarding new people getting involved and increasing the overlap of understanding between those already involved. It can also help to pick out key behaviours/actors to target. I am thinking something like this diagram, as an example
I agree that such mapping seems like it could be very useful for catching people up to speed on a complicated/messy topic more quickly, particularly since I see it as a more efficient/natural way of conveying and browsing information about a controversial or complicated question.
As for the diagram you mentioned, I think that something like that might be helpful, but personally I consider that a map for broader academic questions should probably use semantically-richer relationship information than just (+) and (-). For an example of what I mean, see the screenshots of diagrams I posted in my post on the subject.
I consider the issue of identifying knowledge gaps (and the semi-opposite concept: arguments/experiments which are highly influential in a field) as another good potential use case/benefit of epistemic mapping: both the importance of and lack of research on a concept may become more apparent with visualization—especially if the map/graph shows that the claim or assumption is very influential in the literature but it has not received much support or scrutiny. I’d be curious to get your thoughts on the potential usefulness/viability of such a project for such purposes.
Interesting. I think a challenge would be to find the right level of complexity of a map like that—it needs to be simple enough to give a useful overview, but complex enough that it models everything that’s necessary to make it a good tool for decisionmaking.
Who do you imagine would be the main user of such a mappning? And for which decisions would they mainly use it? I think the requirements would be quite different depending on if it’s to be used by non-experts such as policymakers or grantmakers, or if it’s to be used by researchers themselves?
I agree the complexity level question is a tough question, although my impression has been that it could probably be implemented with varying levels of complexity (e.g., just focusing on simpler/more-objective characteristics like “data source used” or “experimental methodology” vs. also including theoretical arguments and assumptions). I think the primary users would tend to be researchers—who might then translate the findings into more-familiar terms or representations for the policymaker, especially if it does not become popular/widespread enough for some policymakers to be familiar with how to use or interpret it (somewhat similar to regression tables and similar analyses). That being said, I also see it as plausible that some policymakers would have enough basic understanding of the system to engage/explore on their own—like how some policymakers may be able to directly evaluate some regression findings.
Ultimately, two examples of the primary use cases I envision are:
Identifying the ripple effects of changes in assumptions/beliefs/datasets/etc. Suppose for example an experimental finding or dataset which influenced dozens of studies is shown to be flawed: it would be helpful to have an initial outline of what claims and assumptions need to be reevaluated in light of the new finding.
Mapping the debate for a somewhat contentious subject (or just anything where the literature is not in agreement), including by identifying if any claims have been left unsupported or unchallenged.
It seems that such insights might be helpful for a researcher trying to decide what to focus on (and/or a grantmaker trying to decide what research to fund).
Cool—my immediate thought is that it would be interesting to see a case study of (1) and/or (2) - do you know of this being done for any specific case? Perhaps we could schedule a call to talk further—I’ll send you a DM!
I also mention that I think that mapping could be very useful, particularly for onboarding new people getting involved and increasing the overlap of understanding between those already involved. It can also help to pick out key behaviours/actors to target. I am thinking something like this diagram, as an example
I agree that such mapping seems like it could be very useful for catching people up to speed on a complicated/messy topic more quickly, particularly since I see it as a more efficient/natural way of conveying and browsing information about a controversial or complicated question.
As for the diagram you mentioned, I think that something like that might be helpful, but personally I consider that a map for broader academic questions should probably use semantically-richer relationship information than just (+) and (-). For an example of what I mean, see the screenshots of diagrams I posted in my post on the subject.