Evidence Synthesis in the Social Sciences
Before beginning a systematic review, researchers should address these questions:
1. Does this review already exist?
Evidence synthesis reviews should only be done if no existing review addresses your research question. Ensure no protocols (statements of planned reviews) or published reviews on your topic exist.
2. Is there enough literature published on the topic to warrant a review?
Systematic reviews are designed to distill the evidence from many studies into actionable insights. Is there a body of evidence available to analyze, or does more primary research need to be done?
3. Can your research question be answered by a systematic review?
Systematic review questions should be specific and clearly defined. Questions that fit the PICO (problem/patient, intervention, comparison, outcome) format are usually well-suited for the systematic review methodology. The research question determines the search strategy, inclusion criteria and data that you extract from the selected studies, so it should be clearly defined at the start of the review process.
4. Do you have a protocol outlining the review plan?
The protocol is the roadmap for the review project. A good protocol outlines the study methodology, includes the rationale for the systematic review, and describes the key question broken into PICO components. It is also a good place to plan out inclusion/exclusion criteria, databases that will be searched, data abstraction and management methods, and how the studies will be assessed for methodological quality.
5. Do you have a team of experts?
A systematic review is a team effort. Having multiple reviewers minimizes bias and strengthens analysis. Teams are often composed of subject experts, two or more literature screeners, a librarian to conduct the search, and a statistician to analyze the data.
6. Do you have the time that it takes to properly conduct a systematic review?
Systematic reviews typically take 12-18 months. Systematic reviews can be very time-intensive (up to 18 months, by some estimates), and other reviews may be more appropriate for you if you have limited time or are working alone. The PredicTER tool can give you an estimate of how much time may be needed for your review.
7. Do you have a method for discerning bias?
There are many types of bias, including selecting, performance, and reporting bias, and assessing the risk of bias in individual studies is an important part of study design. Review the Risk of Bias Assessment page.
8. Can you afford to have articles translated into languages other than English?
You should include all relevant studies in your systematic review, regardless of the language in which they were published, so as to avoid language bias.
Adapted from: University of Michigan's Systematic Review research guide
Additional Considerations
Time
Systematic reviews can be very time-intensive (up to 18 months, by some estimates), and other reviews may be more appropriate for you if you have limited time or are working alone. The PredicTER tool can provide an estimate of how much time may be required for your review.
The Cochrane Handbook for Systematic Reviews of Interventions suggests the following timeline to complete a Cochrane Review:
| Month | Activity |
|---|---|
| 1 - 2 | Preparation of protocol |
| 3 - 8 | Searches for published and unpublished studies |
| 2 - 3 | Pilot test of eligibility criteria |
| 3 - 8 | Inclusion Assessments |
| 3 | Pilot test of ‘Risk of bias’ assessment. |
| 3 - 10 | Validity assessments |
| 3 | Pilot test of data collection |
| 3 - 10 | Data collection |
| 3 - 10 | Data entry |
| 5 - 11 | Follow up on missing information |
| 8 - 10 | Analysis |
| 1 - 11 | Preparation of review report |
| 12 - | Keeping the review up-to-date |