Prior to starting your project, you should search the literature to determine if a review has recently been published on your topic of interest. Depending on the quality of a recent review, you may decide your planned review is not needed at this time and/or that your review question should change.
For help with searching the literature for recent reviews, contact your Welch Medical Library Informationist.
A well-thought-out and clearly formulated research question is the foundation for a successful systematic review. It provides the basis for the search terms used to identify evidence and informs the generation of inclusion/exclusion criteria used to select evidence. A research question includes multiple components, which may include a population, an exposure or intervention, a setting or context, a comparison, an outcome, and a timeframe. The inclusion/exclusion criteria specifically address these components and are used to decide which studies will be included in or excluded from the review.
For more information about the process of defining the research question and developing criteria for inclusion see Core Methods section of the Cochrane Handbook.
The PICO (Patient or Problem, Intervention, Comparison, Outcome) framework is a commonly used tool for formulating research questions. Davies (2011) reviews PICO and numerous PICO-derived frameworks for developing evidence based practice questions.
Systematic reviews are specialized research projects that require multiple types of expertise. These include subject matter knowledge, familiarity with systematic review methodology, skills for searching in a variety of databases, and expertise in statistical methods.
The inclusion of a trained information professional on the team has become part of established standards for high quality systematic reviews ( IOM, Cochrane, AHRQ). Contact your Welch Informationist about becoming a part of your team.
A protocol is a key document that outlines how the many steps of a systematic review will be carried out. It provides a roadmap to the review team and shows transparently how the review will be conducted.
It's a good idea to register your protocol. It increases transparency, minimizes bias, and reduces the redundancy of groups working on the same topics (PLoS Medicine Editors, 2011).
The following registries publish protocols of systematic reviews and include information about registering a protocol:
Contact your Welch Informationist with questions about developing your protocol and where to register it.
The following resources list commonly used frameworks for developing research questions:
The following are free tools for analyzing topics and visualizing connections within large sets of documents:
Databases must be searched as comprehensively as possible in order to identify all of the documents relevant to your systematic review. The following components are used in comprehensive searching:
Visit our Expert Searching guide to learn more about these and other searching concepts.
When Welch informationists are on systematic review teams, they develop comprehensive search strategies for multiple databases.
When searching for health and medical information you are expected to search PubMed, which includes the MEDLINE database of indexed citations. PubMed contains a very large collection of literature, but to be comprehensive you will need to supplement your PubMed search with additional databases. Always check the requirements or recommendations specified in guidelines for the type of review you're conducting (See this guide's "Standards and Guidelines" page for a list of guidelines).
Things to consider when selecting additional databases include
The Welch Medical Library website offers a searchable and browsable list of Hopkins-subscribed databases. This list covers topics including biomedicine, health, engineering, business, economics, criminal justice, public policy, etc. In addition, the Hopkins Sheridan Libraries maintain a collection of subject guides that link to subject-specific databases.
Search filters, also known as "search hedges," are pre-developed search strategies for topics and study types. We recommend only using validated filters that have been tested for reliability and accuracy.
Do not confuse filters with pre-set limits in databases, even when these limits are called filters (as they are in PubMed). Pre-set limits such as "Article types," "Humans," and "Ages" will limit the search to controlled vocabulary terms, and exclude articles that would be found with the combination of controlled vocabulary and keywords used by validated filters.
See this guide's "Expert Searching Tips" page for more information about search filters, including examples of validated filters for finding randomized controlled trials. Contact your Welch Informationist for questions about the appropriate use of search filters.
Grey literature is information produced by government agencies, academic institutions, and the for-profit sector that is not made available by commercial publishers. This kind of literature can be more difficult to find than the journal literature, but you may wish to include it because it may contain unique and relevant evidence related to your review.
Examples of grey literature document types have been compiled by GreyNet International and include clinical trials, reports, proceedings, dissertations and theses, white papers, newsletters, and patents. Some of these types of documents can be found along with the journal literature in some databases, but grey literature searches often require the use of other specialized sources.
Contact your Welch Informationist for questions about grey literature and how to search for it.
Handsearching involves the page by page examination of key journals or conference proceedings for relevant documents.
Handsearching may identify documents that are missed by database searching because of how articles are described, or indexed, in databases such as PubMed or Embase. Gaps in indexing may result from whole journals not being indexed, portions of journals not being indexed, lack of appropriate indexing terms, inconsistencies in indexing, or poorly reported research that does not allow accurate indexing (Hopewell, Clarke, Lefebvre, & Scherer, 2007).
Citation searching involves manually exploring the citation networks of selected relevant articles as another way of identifying documents that may have been missed through other means of searching. Forward searching identifies later articles that have cited articles of interest. Backward searching identifies earlier articles from the reference lists of articles of interest. The databases Scopus and Web of Science have excellent features for citation searching.
Contact your Welch Informationist with questions about best practices for handsearching and citation searching.
The following are free tools for identifying MeSH terms and keywords from text or articles:
A citation management tool is needed to organize the results of searches and to remove and store duplicates. Typically, records are then exported from the citation management tool to a screening tool.
A pre-established set of inclusion/exclusion criteria are employed when screening studies.
In systematic reviews, the selection of studies is a two-step process that includes a title/abstract review followed by a full-text review. Studies are reviewed according to the inclusion/exclusion criteria by two reviewers, who must agree to include or exclude. Any conflicts are resolved by a third reviewer.
EndNote is the preferred tool for large systematic reviews because of the following features:
RefWorks is sometimes used for smaller reviews for the following reasons:
Information about these and other citation management tools are available from the library's Citation Management guide.
Additional tools can be found by searching the Systematic Review Toolbox (SR Toolbox).
Data abstraction is the systematic collection of data elements from included studies in a review. Data elements could include study design, intervention, outcomes measured, results, etc. This is best achieved through a standardized form derived from your research question and inclusion/exclusion criteria. Pilot the use of this form before full implementation.
Study appraisal systematically examines factors such as
Tools that Have Robust Data Abstraction Features:
Additional tools can be found by searching the Systematic Review Toolbox (SR Toolbox).
The synthesis of evidence includes a qualitative synthesis of included studies which describes study methodology, strengths and limitations, patterns across studies, potential bias in study design, and the relevance of studies to the populations, comparisons, cointerventions, settings, and outcomes or measures of interest.
The synthesis of evidence may also include a meta-analysis that pools data from included studies. A meta-analysis will address the heterogeneity among study effects, statistical uncertainty, and the sensitivity of conclusions to changes in the protocol, assumptions, and study selection (Institute of Medicine, 2011).
Additional tools can be found by searching the Systematic Review Toolbox (SR Toolbox).
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an established guideline for reporting systematic reviews.
See the Standards and Guidelines page of this guide for other evidence synthesis reporting guidelines.