Evidence Synthesis in the Social Sciences
What is Evidence Synthesis?
Evidence synthesis involves gathering information from various sources and disciplines to support informed decisions on specific issues. Providing decision-makers with timely, relevant, and reliable evidence is essential for effective policymaking. As a result, delivering an accurate, concise, and impartial synthesis of the evidence is one of the most impactful contributions the research community can make in advancing informed policy and decision-making.
Adapted from: The Royal Society. (n.d.). Evidence Synthesis. https://royalsociety.org/news-resources/projects/evidence-synthesis/
Evidence syntheses are much more time-intensive than traditional literature reviews and require a multi-person research team. Before embarking on an evidence synthesis, it's important to clearly identify your reasons for conducting one.
Typical Components of Evidence Synthesis
- Adherence to methodological guidelines or standards.
- A pre-registered protocol defining the methods used to conduct the review.
- A methodological, comprehensive literature search focused on a well-formulated research question.
- Inclusion and exclusion criteria are defined before conducting the review.
- May include appraisal of the quality of included literature.
Challenges in the Social Sciences
Retrieving information for systematic reviews in the social, behavioral, and educational sciences can be particularly challenging due to the wide distribution and inconsistent categorization of relevant studies. Unlike the medical sciences, these fields often lack structured abstracts and standardized terminology, requiring more sensitive rather than specific searches (Mallett et al., 2012; Petticrew, 2006). Additionally, the review process tends to favor research from the Global North, making studies from the Global South harder to locate.
References: MacDonald, H., Comer, C., Foster, M., Labelle, P. R., Marsalis, S., Nyhan, K., ... & Young, S. (2024). Searching for studies: A guide to information retrieval for Campbell systematic reviews. Campbell Systematic Reviews, 20(3), e1433. https://doi.org/10.1002/cl2.1433
Mallett, R., Hagen‐Zanker, J., Slater, R., & Duvendack, M. (2012). The benefits and challenges of using systematic reviews in international development research. Journal of Development Effectiveness, 4(3), 445–455. https://doi.org/10.1080/19439342.2012.711342
Petticrew, M. (Ed.). (2006). Systematic reviews in the social sciences. Blackwell Publishing Ltd. https://doi.org/10.1002/9780470754887
Why conduct this type of research?
A researcher synthesizes evidence to advance collective knowledge and enhance societal well-being. By systematically collecting and analyzing existing research, they aim to:
- Clarify complex social issues
- Identify effective interventions
- Highlight areas needing attention and resources
Many organizations, including the United Nations, emphasize evidence-based research to guide policymakers and address complex societal challenges outlined in the Sustainable Development Goals, with initiatives like the Global SDG Synthesis Coalition playing a pivotal role.
With the volume of research literature growing at an ever-increasing rate, individual decision-makers find it increasingly challenging to evaluate this extensive body of primary research. The rigorous approach of systematic reviews ensures that decisions impacting communities are grounded in the best available evidence, promoting more equitable, informed policies and practices. Therefore, it is crucial that this research be conducted with the highest integrity and adherence to recommended guidelines.
Sample Questions Answered by Evidence Synthesis
Here are some examples of questions that evidence syntheses have addressed:
Education: What effects does the flipped classroom have on student learning outcomes compared to traditional classrooms and what variables moderate the effects? (https://doi.org/10.1007/s11423-018-9633-7)
Natural resources: Do managed bees have negative effects on wild bees? (https://doi.org/10.1371/journal.pone.0189268)
Business: Does executive compensation predict publicly traded firms’ financial performance or inaccurate financial reporting? (https://doi.org/10.1002/cl2.1064)
Psychology: Are video games an effective treatment (on their own or as an add‐on) for improving the well‐being and functioning of people with schizophrenia or schizoaffective disorder? (https://doi.org/10.1002/14651858.CD012844.pub2)
Source: Kocher, M., Riegelman, A., Kibbee, M., Young, S., Bostrom, M., Marsalis, S., and Premji, Z. Evidence Synthesis for Librarians and Information Specialists. Open Learning Initiative, 2024. Available from https://oli.cmu.edu/courses/evidence-synthesis-for-librarians-and-information-specialists-o-f/