A systematic review summarizes research on a specific topic by using reproducible methods to connect findings from multiple studies. It aims to draw reliable conclusions while addressing potential biases. These reviews are particularly valuable in fields like healthcare, where they guide evidence-based decision-making by providing a comprehensive evaluation of existing research.
To maintain quality and transparency, systematic reviews adhere to established guidelines like PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), which lays out the steps for methodically reporting research. Additionally, tools like AMSTAR and GRADE are used to assess the rigor and quality of the evidence within the review.
Although both systematic and literature reviews involve gathering research, they differ significantly in their methods and goals.
Let's walk through the steps in the process with this literature review example.
Decision tree to assist researchers in choosing a review type. Image source: Cornell university library
A broad term referring to reviews with a wide scope and non-standardized methodology.
Applies systematic review methodology within a time-constrained setting.
Systematically and transparently collects and categorizes existing evidence on a broad topic or set of research questions.
Reviews other systematic reviews on a topic.
A methodical and comprehensive literature synthesis focused on a well-formulated research question.
A statistical technique for combining the findings from disparate quantitative studies.
Steps for conducting a systematic review. Image source: sagepub
DependinThe foundation of a solid review process is a well-defined research question. Start by asking: What topic are you investigating? What specific issue or research gap do you want to address? The clearer your question, the easier it is to build a strong review.
For systematic reviews, frameworks like PICO (Population, Intervention, Comparison, Outcome) can guide you in refining your question, especially in fields like healthcare. Once the question is clear, the next step is to create a protocol, a roadmap for your review. This document should cover your objectives, inclusion criteria, data collection methods, and analysis strategies.
Don’t forget to register your protocol on platforms like PROSPERO. Doing this helps maintain transparency and avoids unnecessary duplication of research efforts.g on the type of data, researchers can perform either:
Once you’ve outlined your protocol, it’s time to dive into the literature. This involves a comprehensive search strategy across multiple sources to ensure no relevant studies are missed. Search through both major academic databases (e.g., PubMed, Cochrane Library) and grey literature, such as conference proceedings and theses.
Keep everything organized by documenting your search terms, databases, and selection criteria. This documentation ensures that others can reproduce your review, which is a key principle of systematic research.
After gathering potential studies, it's time to screen them. Use the inclusion and exclusion criteria you defined in your protocol to decide which studies will be part of your review. To reduce selection bias, it’s best to have two independent reviewers assess the studies. Having a second opinion helps catch mistakes and ensure consistency in the selection process.
Next, you’ll need to extract key data from each selected study. This might include participant characteristics, outcomes, and results. But it’s not just about gathering data, you also need to evaluate the quality of each study. Tools like the Cochrane Risk of Bias tool and GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) can help you assess how reliable and valid the evidence is.
Now comes the core of your review: synthesizing the data. Depending on the type of research, you can take one of two approaches:
It's crucial to address heterogeneity, differences in study design, populations, and outcomes. These variations can affect the reliability of your findings, so make sure to document and explain how you handled them.
Once you've analyzed the data, it's time to present your conclusions. Be thorough, include details about your methods, study characteristics, and results. For systematic reviews, guidelines like PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) ensure transparency and reproducibility.
When interpreting your findings, pay close attention to the strength of the evidence. If some results are inconsistent or limited by study quality, be clear about these limitations in your conclusions.
Systematic reviews are powerful tools for synthesizing evidence, but they come with significant challenges that can complicate the process. From identifying relevant studies to maintaining the currency of findings, each stage requires careful planning, organization, and execution. By proactively addressing these obstacles, you can improve the quality, reliability, and efficiency of your review. The following sections provide key strategies to overcome these challenges and ensure your systematic review meets high standards of rigor and transparency.
Conducting a comprehensive search across multiple databases can be challenging. You may miss important studies due to incomplete searches, limited access to full-text articles, or difficulty finding grey literature such as reports, conference papers, or theses. To ensure all relevant research is identified, you need meticulous planning and a well-defined search strategy.
Systematic reviews require screening and analyzing large volumes of studies, sometimes hundreds or thousands. Without a solid plan, data extraction, organization, and management can easily become overwhelming and cause delay
Striking a balance between sensitivity (capturing all relevant studies) and specificity (avoiding irrelevant results) is crucial. If your search is too sensitive, you'll get flooded with irrelevant results. If it's too specific, you risk missing critical studies.
Selection bias can undermine your review if you don’t consistently apply the inclusion and exclusion criteria. This can happen when certain types of studies are overrepresented or unintentionally excluded.
When studies vary in methodology, outcomes, or populations, synthesizing the data can be difficult. This heterogeneity can prevent direct comparisons or limit the reliability of a meta-analysis. you must carefully consider how to handle these differences to draw meaningful conclusions.
New studies are published frequently, which can quickly make your review outdated. Without regular updates, your conclusions may lose relevance over time.
Litmaps can aid in the search for relevant papers for a systematic review, as well as finding any papers overlooked in traditional search. Additionally, Litmaps Monitor can be used to stay up-to-date on the field throughout the review process.
To use these tools for a systematic review it's important to 1) record the collection methodology, and 2) consult a secondary set of electronic sources. This is true no matter what tool you're using.
Use Litmaps to collect papers for your review. Here's how to search for articles in Litmaps. The platform's search tool works best when you provide multiple input articles, which allows the system to generate more targeted results. Traditional search methods can overlook important studies due to keyword limitations, but Litmaps explores connections between research papers, potentially uncovering hidden or hard-to-find studies.
Key Tip:
To ensure reproducibility, document essential search details such as input articles, lookup date, Filters, and Version information. This metadata will help future researchers replicate your work.
After gathering your initial set of papers, create a new Litmap to visualize related studies. Traditional searches often depend heavily on keywords, which may limit results due to differences in terminology. Litmaps, however, uses a keyword-free approach, suggesting connected papers even if they use different terms or phrasing. This broader perspective helps you find gaps in research and overlooked studies.
Action Step:
Cross-check Litmaps’ suggestions to ensure you haven't missed any relevant research, particularly in areas where new terminology may have emerged.
Systematic reviews can take months to complete, during which time new studies are constantly published. Litmaps Monitor helps you stay on top of recent developments by tracking relevant topics and notifying you weekly of any new studies connected to your research. This feature ensures your review remains current and up-to-date throughout the entire process.
Traditional searches often fail to capture all relevant studies due to variations in terminology. Litmaps improves coverage by discovering papers through connections, reducing the risk of missing critical research. This feature is particularly useful when navigating large fields of study where papers may use different keywords to describe similar concepts.
Systematic reviews are essential for consolidating evidence and supporting evidence-based practice. However, the process can be complex and time-consuming, involving extensive searches and ongoing updates. Tools like Litmaps streamline these tasks by improving search efficiency, organizing research, and keeping you informed of new studies. By combining Litmaps with best practices and established guidelines, you can produce high-quality, impactful reviews that stand up to rigorous scrutiny.
