Evidence Synthesis in the Social Sciences

Evidence synthesis involves gathering information from various sources and disciplines to support informed decisions on specific issues. Providing decision-makers with timely, relevant, and reliable evidence is essential for effective policymaking. As a result, delivering an accurate, concise, and impartial synthesis of the evidence is one of the most impactful contributions the research community can make in advancing informed policy and decision-making. 

Adapted from: The Royal Society. (n.d.). Evidence Synthesis. 

Evidence syntheses are much more time-intensive than traditional literature reviews and require a multi-person research team. Before embarking on an evidence synthesis, it's essential to identify your reasons for conducting one.

Typical Components of Evidence Synthesis 

• Adherence to methodological guidelines or standards.
• A pre-registered protocol defining the methods used to conduct the review.
• A methodological, comprehensive literature search focused on a well-formulated research question.
• Inclusion and exclusion criteria are defined before conducting the review.
• May include appraisal of the quality of included literature. 

Challenges in the Social Sciences 

Retrieving information for systematic reviews in the social, behavioral, and educational sciences can be particularly challenging due to the wide distribution and inconsistent categorization of relevant studies. Unlike the medical sciences, these fields often lack structured abstracts and standardized terminology, requiring more sensitive rather than specific searches (Mallett et al., 2012; Petticrew, 2006). Additionally, the review process tends to favor research from the Global North, making it harder for studies from the Global South to be located. 

References:

• MacDonald, H., Comer, C., Foster, M., Labelle, P. R., Marsalis, S., Nyhan, K., ... & Young, S. (2024). Searching for studies: A guide to information retrieval for Campbell systematic reviews. Campbell Systematic Reviews, 20(3), e1433. https://doi.org/10.1002/cl2.1433
• Mallett, R., Hagen‐Zanker, J., Slater, R., & Duvendack, M. (2012). The benefits and challenges of using systematic reviews in international development research. Journal of Development Effectiveness, 4(3), 445–455. https://doi.org/10.1080/19439342.2012.711342
• Petticrew, M. (Ed.). (2006). Systematic reviews in the social sciences. Blackwell Publishing Ltd. https://doi.org/10.1002/9780470754887

A researcher synthesizes evidence to advance collective knowledge and enhance societal well-being. By systematically collecting and analyzing existing research, they aim to: 

• Clarify complex social issues
• Identify effective interventions
• Highlight areas needing attention and resources

Many organizations, including the United Nations, emphasize the importance of evidence-based research in guiding policymakers and addressing the complex societal challenges outlined in the Sustainable Development Goals. Initiatives such as the Global SDG Synthesis Coalition play a pivotal role in this effort.

With the volume of research literature growing at an ever-increasing rate, individual decision-makers are finding it increasingly challenging to evaluate the extensive body of primary research. The rigorous approach of systematic reviews ensures that decisions affecting communities are grounded in the best available evidence, promoting more equitable and informed policies and practices. Therefore, this research must be conducted with the highest integrity and adherence to recommended guidelines.

Here are some examples of questions that evidence syntheses have addressed:

• Education: What effects does the flipped classroom have on student learning outcomes compared to traditional classrooms and what variables moderate the effects?
• Natural resources: Do managed bees have negative effects on wild bees?
• Business: Does executive compensation predict publicly traded firms’ financial performance or inaccurate financial reporting?
• Psychology: Are video games an effective treatment (on their own or as an add‐on) for improving the well‐being and functioning of people with schizophrenia or schizoaffective disorder?

Source: Kocher, M., Riegelman, A., Kibbee, M., Young, S., Bostrom, M., Marsalis, S., and Premji, Z. Evidence Synthesis for Librarians and Information Specialists. Open Learning Initiative, 2024. 

The Right Review tool is designed to provide guidance and supporting materials to reviewers on methods for conducting and reporting knowledge synthesis.