Overcome common challenges and develop a high-quality systematic review by following these four key steps.
Literature reviews play a pivotal role in synthesizing existing knowledge, identifying gaps and informing new studies. Narrative literature reviews focus on a topic of interest and aim to provide a broad picture of a field. Systematic literature reviews are bound to a more rigorous process. They focus on a specific research question and employ structured, reproducible methods to identify, assess and critically appraise all studies relevant to the question. The results of systematic reviews provide critical insights that guide practitioners and policymakers in their decision-making. For that reason, systematic reviews have gained prestige and the volume of systematic reviews produced has increased each year.[1]
Yet, challenges still exist. Researchers report increasing pressure from too many tasks and too little time in their day-to-day. To help address the pain points of overburdened researchers, many research tools are implementing rapidly advancing artificial intelligence (AI) and machine learning (ML) technologies. For example, Web of Science™ Research Assistant, a generative AI-powered tool forthcoming in September, offers a guided walkthrough for researchers starting a narrative literature review.
Rigorous systematic reviews typically require even more time and effort from researchers, and AI and ML technologies have the potential to significantly reduce that load, especially when it comes to screening studies. On the flip side, AI and ML are no substitute for the human expertise required to ensure the quality and relevance of a systematic literature review, and many organizations are still cautiously evaluating AI’s true potential for this research use case.
In the meantime, there are trusted resources that will streamline the process of developing a high-quality systematic literature review today. In this blog, we’ll explore those resources and provide tips on how to overcome common challenges. Read on to learn the impact of:
- Having a librarian collaborator on your review
- Going beyond keywords for a more comprehensive search
- Avoiding publication bias
- Finding the best citation manager
Following these four steps can help you efficiently complete a rigorous systematic review that delivers useful results to fellow researchers and practitioners.
Want more tips on streamlining and improving systematic reviews? Check out our on-demand webinar.
Talk to your librarian: Learn from the experts
Librarians can play a pivotal role in helping you develop a high-quality, transparent systematic review. In fact, research[2] has shown that involving librarians in the process correlates with the reporting of higher quality search strategies, which also helps improve the reproducibility of your review. Librarians excel at constructing comprehensive search strategies, and they know how to navigate databases effectively. They can also guide you through the development of study selection criteria and risk of bias assessments.
Cast a wide net: Go beyond keyword searching
Research databases and search engines are an integral part of any literature search. It’s important you utilize as many options available through your library as possible. This will help you incorporate relevant studies across disciplines for a thorough systematic review.
Your literature search also needs to use a well-constructed and thorough search strategy. Following a framework like PICO (Problem, Intervention, Comparison, Outcome) or SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) can help you ensure you’re not missing anything.
The Web of Science platform is a particularly useful tool for uncovering relevant research. This multidisciplinary research discovery platform covers a broad range of research types and areas. Papers in the database are interconnected via citations. That means once you get started with your keyword search, you can follow the trail of cited and citing papers to efficiently expand your search and locate additional relevant studies. The platform also offers tools that will help you develop, document, and share your search strategy with collaborators.
Avoid publication bias: Include unpublished works for more complete reporting
Many scholars note a growing need to overcome publication bias with more meticulous and inclusive literature searches. An abundance of quality research is never published in the peer reviewed literature. The well-documented problem of “publication bias” or the “file drawer effect”[3] shows that studies with negative, null, or statistically insignificant results are less likely to appear in the published literature. It’s important to include all relevant evidence in your systematic review, and unpublished works, such as dissertations and theses, are an important source of additional findings.
There are a number of databases that contain unpublished works or grey literature, including ProQuest™ Dissertations & Theses Global (PQDT), Publicly Available Content, preprint servers, and ClinicalTrials.gov. For streamlined searching, the Web of Science recently integrated preprints from several leading repositories and dissertation and thesis metadata records from PQDT Global into the platform. This offers an easy way to locate relevant post-graduate studies for your review and if your subscription allows, direct links to the full text for detailed review. Additionally, dissertations and theses contain rich bibliographies that can lead you to further research.
Find the best citation manager before you start your review
One of the most challenging aspects of a systematic review is keeping your citations, notes and progress organized. Before you begin your search, get established with a citation management tool. Citation managers, such as EndNote™ from Clarivate™, simplify systematic reviews by streamlining the collection and organization of research articles.
Your librarian partner can help you identify a robust and reliable citation management tool. Here’s a checklist of citation management features that will make your life easier and create a more accurate systematic review:
- Integrates with search tools: You want to eliminate manual data entry so look for a tool that enables you to import your findings directly into a library that saves and organizes them.
- Allows you to group references: The ability to create several groups of references you collect will allow you to organize references by topic, study type, review status and more. Make sure the overall project library can support thousands of references if you’re undertaking a large-scale review.
- Offers tagging/categorization: Tagging references with keywords, labels and custom fields will allow quick retrieval during the review process.
- Deduplicates your library: Your review will likely encompass thousands of references that you are pulling in from varied sources, and you’ll need to run multiple checks for duplicates. Bonus if the tool allows you to control which version of a reference is kept by default.
- Simplifies working with PDFs: Being able to annotate PDFs you’ve saved in your library will help you extract relevant information during the review. Another time-saver: automatic population of key metadata, like author names, publication details, and abstracts.
- Streamlines production of your manuscript and ensures more accuracy and consistency with automated citation insertions and bibliography creation. (Make sure it supports your citation style!)
A special note about citation managers and collaboration
Collaborating with your review team can be hard, especially if your coauthors are working across multiple locations. This is another area where citation management tools can improve your process.
A citation manager like EndNote will allow you to work together in real-time, sharing and annotating references in a library that team members can see from anywhere. For large teams, you may want to have various levels of access, with some team members who can read references, and others who can add and delete references and annotate articles.
A note of caution: if you’re giving team members the ability to add, delete and annotate, be sure your citation manager has a version history so you can walk back changes when needed.
No matter what citation manager you choose, remember that effective collaboration also depends on clear communication and agreed-upon conventions among team members.
Click here to get started with EndNote today.
Make an impact with a great systematic review
Systematic reviews are arduous, daunting tasks, but the payoff is rich. Authors who meticulously synthesize the existing literature contribute valuable findings to the broader scientific and scholarly community. Following best practices, aiming for greatness and organizing the process so that manual steps are automated, and errors are minimized can set you and your team up for success.
Want more tips on streamlining and improving systematic reviews? Check out our on-demand webinar.
References:
[1]Smela, B., Toumi, M., Świerk, K., Gawlik, K., Clay, E., & Boyer, L. (2023). Systematic literature reviews over the years. Journal of Market Access & Health Policy, 11(1). https://doi.org/10.1080/20016689.2023.2244305
[2]Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617-626. https://doi.org/10.1016/j.jclinepi.2014.11.025
[3]Nagarajan, P. J., Garla, B. K., Taranath, M., & Nagarajan, I. (2017). The file drawer effect: A call for meticulous methodology and tolerance for non-significant results. Indian journal of anaesthesia, 61(6), 516–517. https://doi.org/10.4103/ija.IJA_280_17
Related insights
The latest news, technologies, and resources from our team.
