Psychology

A method of research that involves a systematic and comprehensive process of searching, assessing, analyzing, and synthesizing diverse sources of evidence from multiple studies to provide a comprehensive and unbiased summary of a particular research question or topic. Evidence synthesis provides researchers, policymakers, and practitioners with an exhaustive and reliable basis to make informed decisions by considering the totality of evidence rather than relying on individual studies in isolation. 

Steps for conducting reviews: 

1. Specification of research question and development of protocol
   • It is best practice that the protocol be peer-reviewed before conducting your search.
2. Search of literature to identify relevant studies
   • It is best practice to consult with a librarian to assist you in developing your strategy. Searches must be reproducible and transparent. 
3. Screening of the studies
4. Analysis of the evidence
5. Synthesis in the completed review

Your review type should match your intended purpose. 

Created by Cornell University Libraries

You can also use Right Review, which will provide guidance and supporting material on methods for conducting and reporting knowledge synthesis.

Reproduced from Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

How much time do you have to commit to the project? 

PredicTER

"PredicTER is a tool for estimating how long a review will take to complete. The tool calculates the time requirements for various tasks involved in reviewing evidence, from planning and coordination to quantitative synthesis and reporting."

Suggested Readings

• Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements. Health Information & Libraries Journal, 36(3), 202-222.
• Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC medical research methodology, 18, 1-7.
• Littell, J. H. (2018). Conceptual and practical classification of research reviews and other evidence synthesis products. Campbell Systematic Reviews, 14(1), 1.
• Campbell, F., Tricco, A. C., Munn, Z., Pollock, D., Saran, A., Sutton, A., ... & Khalil, H. (2023). Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): the same but different—the “Big Picture” review family. Systematic reviews, 12(1), 45.
• Peters, M. D., Marnie, C., Tricco, A. C., Pollock, D., Munn, Z., Alexander, L., ... & Khalil, H. (2021). Updated methodological guidance for the conduct of scoping reviews. JBI evidence implementation, 19(1), 3-10.
• White, H., Albers, B., Gaarder, M., Kornør, H., Littell, J., Marshall, Z., ... & Welch, V. (2020). Guidance for producing a Campbell evidence and gap map. Campbell Systematic Reviews, 16(4), e1125.
• Sager, M., & Pistone, I. (2019). Mismatches in the production of a scoping review: Highlighting the interplay of (in) formalities. Journal of Evaluation in Clinical Practice, 25(6), 930-937.
• Khalil, H., & Tricco, A. C. (2022). Differentiating between mapping reviews and scoping reviews in the evidence synthesis ecosystem. Journal of Clinical Epidemiology, 149, 175-182.
• Tod, D., Booth, A., & Smith, B. (2022). Critical appraisal. International Review of Sport and Exercise Psychology, 15(1), 52-72.

Review Question: 

• The review question is unclear: You have not shown why some studies were included and why others were excluded. Your decision seems arbitrary. 
• The review question is not precise: You have framed your problem as a topic, not a review question. 

Search: 

• Lack of quality sources: You have not identified/included sufficient peer-reviewed sources. You have included a high proportion of web sources. 

Problem: 

• Glaring omissions: Key items on the topic are omitted. You have not searched widely enough, including reading reviews and following up on references. 

Appraisal: 

• Too descriptive: You have not evaluated the quality of your sources but have taken them at face value. 
• Over-reliant on the work of others: Your ideas or wording are taken unprocessed from the work of others. This may reflect plagiarism or a lack of critical thought about what the original sources are saying. 

Synthesis: 

• Listing included studies: You have described each study one by one instead of looking for patterns and commonalities across the studies
• Random presentation: You have presented studies as you have found or processed them without assigning a meaningful order (e.g., chronological) 

Analysis: 

• Unsupported statements: You have not established a clear link between your statements and the literature supporting them. 
• Over-generalization: You have not identified exceptions to the overall rules. You have rushed your analysis. 

Discussion and Conclusion: 

• Not answering the "So what?" question: You have not clarified how the literature you included answered your review question or the implications for research/practice. 
• Rambling and non-logical order: You have lost sight of the review question you were trying to answer. You are presenting everything you know on the topic. Your tables are not presented in a logical sort order. 

Papaioannou, D., Sutton, A., & Booth, A. (2016). Systematic approaches to a successful literature review. Systematic approaches to a successful literature review, 1-336.