I do not think we have missed a significant portion of primary prevention papers in our time period. Looking at that page, I am seeing some things that had midpoint evaluations in 2018. Looking at this group’s final report (https://www.whatworks.co.za/documents/publications/390-what-works-to-prevent-vawg-final-performance-evaluation-report-mar-2020/file), I do not see anything that qualifies as a primary prevention of sexual violence. We did a painstaking systematic search and I’m reasonably confident we got just about everything that meets our criteria. As to whether we might have chosen different criteria—too late now, but for my own curiosity, what would you suggest?
We have many subgroup analyses in the paper, though for my tastes, I wish we could have done more in terms of grouping studies together by theoretical approach and then analyzing them. This turned out to be really hard in practice because there was so much overlapping content but also so many bespoke delivery mechanisms. This heterogeneity is one reason my next meta-analysis (https://forum.effectivealtruism.org/posts/k9qqGZtmWz3x4yaaA/environmental-and-health-appeals-are-the-most-effective) sets strict quality-related inclusion criteria and then compares theoretical approaches head-to-head.
Definitely, and this is arguably the main limitation of this paper: we’re a few years out of date. Basically what happened was, we started this paper pre-pandemic (the first conversation I was part of was summer 2017!), did our search, and then lost a few years to the pandemic and also a lot of the team worked on a different meta-analysis in the interim (https://www.annualreviews.org/content/journals/10.1146/annurev-psych-071620-030619, on which I was an RA). I still think we’ve got some interesting and general lessons from reviewing 30+ years worth of papers.
Hi Akhil,
Thanks for engaging.
I do not think we have missed a significant portion of primary prevention papers in our time period. Looking at that page, I am seeing some things that had midpoint evaluations in 2018. Looking at this group’s final report (https://www.whatworks.co.za/documents/publications/390-what-works-to-prevent-vawg-final-performance-evaluation-report-mar-2020/file), I do not see anything that qualifies as a primary prevention of sexual violence. We did a painstaking systematic search and I’m reasonably confident we got just about everything that meets our criteria. As to whether we might have chosen different criteria—too late now, but for my own curiosity, what would you suggest?
We have many subgroup analyses in the paper, though for my tastes, I wish we could have done more in terms of grouping studies together by theoretical approach and then analyzing them. This turned out to be really hard in practice because there was so much overlapping content but also so many bespoke delivery mechanisms. This heterogeneity is one reason my next meta-analysis (https://forum.effectivealtruism.org/posts/k9qqGZtmWz3x4yaaA/environmental-and-health-appeals-are-the-most-effective) sets strict quality-related inclusion criteria and then compares theoretical approaches head-to-head.
Definitely, and this is arguably the main limitation of this paper: we’re a few years out of date. Basically what happened was, we started this paper pre-pandemic (the first conversation I was part of was summer 2017!), did our search, and then lost a few years to the pandemic and also a lot of the team worked on a different meta-analysis in the interim (https://www.annualreviews.org/content/journals/10.1146/annurev-psych-071620-030619, on which I was an RA). I still think we’ve got some interesting and general lessons from reviewing 30+ years worth of papers.
DM’d you