Skip to Main Content

Systematic Reviews: Step 1: Complete Pre-Review Tasks

Created by Health Science Librarians

About Step 1: Complete Pre-Review Tasks

It takes an average of 160 hours for a systematic review team to complete pre'review tasks

This step will help you prepare to conduct your systematic review. You will:

  1. Develop your research question.
  2. Look at literature to decide if you need to do a systematic review.
  3. Build your research team.
  4. Decide which citation manager and systematic review software you will use.

This page has information about research questions and systematic review teams. Librarians can help you edit your research question based on the literature.

Click an item below to see how it applies to Step 1: Pre-Review Tasks.

The PRISMA (Preferred Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines detail what you should report about your systematic review methods.

For this step of the process, you can review the PRISMA checklist and flow diagram and download the PRISMA flow diagram template that matches your review type and sources, then list out the databases and other sources you plan to search.

Covidence is the tool we use at UNC to help manage the systematic review screening process. For this step, you can create a Covidence account, set up a review, add reviewers, list inclusion and exclusion criteria, and edit other review settings.

Before you begin conducting a systematic review, a librarian can help you:

  • Develop and refine your research question framework 
  • Determine if any prior reviews have been published on the same or similar topics
  • Determine how much literature might be available on your topic

 

 

Watch Systematic Review Workshop videos

The Introduction to Conducting a Systematic Review workshop, offered in October 2020, covered recommended standards, methods, and tools for completing a systematized, scoping, or systematic review at UNC. This workshop recording is available as a series of short videos on the process of conducting a review. It is recommended for those who have not yet conducted such a review, but are planning to do so. 

In summer 2023, the Health Sciences Library hosted an 8-session Systematic Review Summer Workshop Series covering all the steps needed to confidently navigate the systematic review process and get familiar with helpful tools. Slides from the sessions are combined into a single pdf below.

There are also a number of free systematic review methods courses you can take online.

Recruit team members

Keep these guidelines in mind when establishing your systematic review team:Review team members can include a principal investigator, 3 reviewers, a librarian, a statistician, and others.

  • Have a minimum of 3 reviewers, although a higher number will expedite the screening process.
    • Odd number simplifies tie-breaking process
    • Depending on the size of the literature, you may want to add additional team members. A team of up to ten or twelve people is not unusual for a large systematic review.
  • Choose experts in the field.
    • Conscientiously create a diverse team to ensure different voices and perspectives are represented.
  • Collaborate with a librarian to develop a search strategy.
  • Work with a statistician if conducting a meta-analysis.
  • Define roles and expectations early in the review process.

Choose review tools

Citation manager softwareGuide comparing features of Endnote, Zotero, and Sciwheel

Citation managers are recommended to collect citations, remove duplicates, and manage your systematic review citations.  UNC offers Sciwheel for free and Endnote Desktop at a discount. Endnote Basic and Zotero are free for anyone to use.

While citation managers are not required to complete a systematic review, we highly recommend using one, as they can assist with organizing citations and screening levels, deduplication, and finding PDFs of articles for full text screening.

 

  EndNote
(Desktop)
Zotero Sciwheel (formerly F1000Workspace)
HSL Online Guides EndNote Zotero Sciwheel (formerly F1000Workspace)
Classes at HSL EndNote Classes   Sciwheel Classes
Cost Under $100 through UNC-CH
 
Zotero is free Free through UNC-CH
Notes See EndNote Basic guide for further details on the free online version   Create an account.  Select No, I'm a new user then select password.
How is it used? Computer
+ Web
Computer
+ Web
Web
Major citation styles?
Annotation of PDFs
Locate full-text using UNC-CH subscriptions
Instructions for EndNote

Instructions for Zotero

Instructions for Sciwheel
Adds citation from a PDF
Sharing options X7 and above users have sharing options / email compressed libraries See group options here
 
Share unlimited projects and manuscripts
Free online storage X7 and above: unlimited 700+ papers
(more space can be purchased)
Unlimited
Word Processor Microsoft Word Microsoft Word
LibreOffice (all)
Microsoft Word
Google Docs
Manuscripts for Mac

Systematic review software

Systematic review software is recommended for conducting parts of the systematic review process iCovidence Logoncluding screening, quality assessment, data extraction, and reporting.

There are many tools that can be useful for organizing the screening process including Covidence, Rayyan, Abstrackr, and HAWC.   

UNC currently has an institutional subscription to Covidence making it available for free to UNC-affiliated users. HSL can provide classes and support for Covidence. To learn more visit the Covidence LibGuide.

HSL does not currently offer in-house support for screening tools other than Covidence.

 

The table compares Covidence, Rayyan, Abstrackr, and HAWC (from left to right) based on a number of features.
Feature: Covidence Rayyan Abstrackr HAWC
Cost UNC has a subscription to Covidence Free Free Free
Support available at UNC HSL Yes No No No
Quality Assessment Yes Yes No Yes
Data Extraction Yes  Yes No Yes
Allows Multiple User Support Yes Yes No No
Reference Allocation Yes Yes Yes No
Multiple Screening Phases Yes No No No
Discrepancy Resolving Yes Yes Yes No
Show Project Progress Yes Yes Yes No
Attaching Comments Yes No Yes No
Attaching PDFs Yes Yes No Yes
Keyword Highlighting Yes Yes Yes No
Deduplication Yes Yes No No
PRISMA Diagram Yes No No Yes
Interrater Reliability Yes Yes No No
Mobile Friendly Version Yes Yes No No
Export Results File Types RIS, CSV, TXT RIS, BibTex, CSV, EndNote RIS, CSV, XML XLSX

If the tools above don't meet your needs, you can also try this Excel tool called the VonVille Method.  

Information about screening tools and features: Van der Mierden, S., Tsaioun, K., Bleich, A., & Leenaars, C. H. C. (2019). Software tools for literature screening in systematic reviews in biomedical research. ALTEX : Alternativen zu Tierexperimenten, 36(3), 508-517.

 

Screening tool feature definitions

Quality Assessment: The tool supports risk of bias and quality assessment. 

Data Extraction: The tool supports data extraction. 

Allows Multiple User Support: It is possible for two or more users to work on the same project at the same time, without seeing how others have voted.

Reference Allocation: It is possible to assign references to reviewers. 

Discrepancy Resolving: There is official support for resolving conflict between reviewers. 

Show Project Progress: the tool can display the progress of each reviewer and the overall project.

Attaching Comments: It is possible to add comments to a reference while screening.

Attaching PDFs: It is possible to upload PDFs for full-text screening. 

Keyword Highlighting: It is possible to display highlighted keywords during screening.

Deduplication: The tool will identify and remove duplicate citations. 

PRISMA Diagram: The tool can automatically generate a PRISMA flow diagram. 

Import Multiple File Types: It is possible to import formatted references and the tool supports multiple file types. 

Interrater Reliability: The tool can calculate and display interrater reliability.

Mobile Friendly Version: It is possible to screen on a mobile device. 

Export Results File Types: References can be exported from the screening tool into the listed file types. 

 

Develop and refine research question

Systematic reviews aim to answer a specific research question. There are frameworks to help in question development and identification of search terms. PICO is the most popular framework utilized for clinical research topics.

PICO

The PICO question framework is useful for quantitative research topics. PICO questions identify four concepts: population, intervention, comparison, and outcome.

Elements, Definitions and Examples for the PICO Question framework
Element Definition Example
Population / Patient Who is the focus of my question? infants diagnosed with necrotizing enterocolitis (NEC)
Intervention What is the proposed, new intervention? early enteral re-feeding
Comparison What is the current or alternative intervention? late enteral re-feeding
Outcome What measurable outcome is affected? NEC recurrence

Research question: In infants diagnosed with necrotizing enterocolitis (NEC), what is the effect of early enteral re-feeding on NEC recurrence compared to late enteral re-feeding?

Did you know?

Did you know there are at least 25 other question frameworks besides variations of PICO?  Frameworks like PEO, SPIDER, SPICE, ECLIPSE, and others can help you formulate a focused research question. The table and example below were created by the Medical University of South Carolina (MUSC) Libraries.

Click on a framework below to learn more and see an example of its use.

The PEO question framework is useful for qualitative research topics. PEO questions identify three concepts: population, exposure, and outcome.

Elements, Definitions and Examples for the PEO Question framework
Element Definition Example
Population Who is the focus of my question? mothers
Exposure What issue interests me? postnatal depression
Outcome What, in relation to the issue, do I want to examine? daily living experiences

Research question: What are the daily living experiences of mothers with postnatal depression?

 

The SPIDER question framework is useful for qualitative or mixed methods research topics focused on "samples" rather than populations.

SPIDER questions identify five concepts: sample, phenomenon of interest, design, evaluation, and research type.

Elements, Definitions and Examples for the SPIDER Question framework
Element Definition Example
Sample Who is the group of people being studied? young parents
Phenomenon of Interest What are the reasons for behavior and decisions? attendance at antenatal education classes
Design How has the research been collected (e.g., interview, survey)? interviews
Evaluation What is the outcome being impacted?

experiences

Research type What type of research (qualitative or mixed methods)? qualitative studies

Research question: What are the experiences of young parents in attendance at antenatal education classes?

 

The SPICE question framework is useful for qualitative research topics evaluating the outcomes of a service, project, or intervention. SPICE questions identify five concepts: setting, perspective, intervention/exposure/interest, comparison, and evaluation.

Elements, Definitions and Examples for the SPICE Question framework
Element Definition Example
Setting the context for the question (where) South Carolina
Perspective the users, potential users, or stakeholders of the service (for whom) teenagers
Intervention / Interest / Exposure the action taken for the users, potential users, or stakeholders (what) provision of Quit Kits to support smoking cessation
Comparison the alternative actions or outcomes (compared to what)

no support or "cold turkey"

Evaluation the result or measurement that will determine the success of the intervention (what is the result, how well) number of successful attempts to give up smoking with Quit Kits compared to number of successful attempts with no support

Research question: For teenagers in South Carolina, what is the effect of provision of Quit Kits to support smoking cessation on number of successful attempts to give up smoking compared to no support ("cold turkey")?

 

The ECLIPSE framework is useful for qualitative research topics investigating the outcomes of a policy or service. ECLIPSE questions identify six concepts: expectation, client group, location, impact, professionals, and service.

Elements, Definitions and Examples for the ECLIPSE Question framework
Element Definition Example
Expectation What are you trying to improve or change? How is the information going to be used? to increase access to wireless internet in the hospital
Client group Who is expected to benefit from the service or policy? patients and families
Location Where is the service or policy located? hospitals
Impact What is the change in service or policy that the researcher is investigating? clients have easy access to free internet
Professionals Who is involved in providing or improving the service or policy? information technology staff, hospital administration
Service What kind of service or policy is this? provision of free wireless internet to patients

Research question: How can I increase access to wireless internet for hospital patients?

 

Specify eligibility criteria

In order to reduce bias, eligibility criteria (also known as inclusion and exclusion criteria) refer to what you plan to include and exclude from your systematic review. These criteria are decided after the research question is developed but before searches are completed. Below are examples of criteria that may be used to determine inclusion or exclusion.

Table providing examples of different types of eligibility criteria
Type of criteria Example
Type of participants May be limited to specific groups of people or age ranges
Study design May include specific study designs and exclude others based on which best answer the research question
Intervention of interest Includes interventions of interest and excludes any others
Outcomes of interest Includes outcomes of interest and may exclude studies reporting outcomes not of interest
Setting May be limited to a specific setting like inpatient, ambulatory, classroom, etc.
Type of publication Reviews, editorials, commentaries, and letters are often excluded
Date of publication Date ranges may be applied when updating a systematic review or when specific to an intervention or therapy
Language of publication Non-English articles are often excluded from reviews; however, this may allow language bias to affect the quality of your review