Click an item below to see how it applies to Step 1: Pre-Review Tasks.
The PRISMA (Preferred Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines detail what you should report about your systematic review methods.
For this step of the process, you can review the PRISMA checklist and flow diagram and download the PRISMA flow diagram template that matches your review type and sources, then list out the databases and other sources you plan to search.
Covidence is the tool we use at UNC to help manage the systematic review screening process. For this step, you can create a Covidence account, set up a review, add reviewers, list inclusion and exclusion criteria, and edit other review settings.
The Introduction to Conducting a Systematic Review workshop, offered in October 2020, covered recommended standards, methods, and tools for completing a systematized, scoping, or systematic review at UNC. This workshop recording is available as a series of short videos on the process of conducting a review. It is recommended for those who have not yet conducted such a review, but are planning to do so.
Keep these guidelines in mind when establishing your systematic review team:
Citation managers are recommended to collect citations, remove duplicates, and manage your systematic review citations. UNC offers Sciwheel for free and Endnote Desktop at a discount. Endnote Basic and Zotero are free for anyone to use.
While citation managers are not required to complete a systematic review, we highly recommend using one, as they can assist with organizing citations and screening levels, deduplication, and finding PDFs of articles for full text screening.
HSL Online Guides
Sciwheel (formerly F1000Workspace)
Classes at HSL
Under $100 through UNC-CH
Zotero is free
Free through UNC-CH
See EndNote Basic guide for further details on the free online version
Use this link to create account. Select No, I'm a new user then select password.
How is it used?
Major citation styles?
Annotation of PDFs
Locate full-text using UNC-CH subscriptions
Instructions for EndNote
Instructions for Zotero
Instructions for Sciwheel
Adds citation from a PDF
X7 and above users have sharing options / email compressed libraries
See group options here
Share unlimited projects and manuscripts
Free online storage
X7 and above: unlimited
(more space can be purchased)
Manuscripts for Mac
There are many tools that can be useful for organizing the screening process including Covidence, Rayyan, Abstrackr, and HAWC.
UNC currently has an institutional subscription to Covidence making it available for free to UNC-affiliated users. HSL can provide classes and support for Covidence. To learn more visit the Covidence LibGuide.
HSL does not currently offer in-house support for screening tools other than Covidence.
|Cost||UNC has a subscription to Covidence||Free||Free||Free|
|Support available at UNC HSL||Yes||No||No||No|
|Allows Multiple User Support||Yes||Yes||No||No|
|Multiple Screening Phases||Yes||No||No||No|
|Show Project Progress||Yes||Yes||Yes||No|
|Mobile Friendly Version||Yes||Yes||No||No|
|Export Results File Types||RIS, CSV, TXT||RIS, BibTex, CSV, EndNote||RIS, CSV, XML||XLSX|
If the tools above don't meet your needs, you can also try this Excel tool called the VonVille Method.
Information about screening tools and features: Van der Mierden, S., Tsaioun, K., Bleich, A., & Leenaars, C. H. C. (2019). Software tools for literature screening in systematic reviews in biomedical research. ALTEX : Alternativen zu Tierexperimenten, 36(3), 508-517.
Quality Assessment: The tool supports risk of bias and quality assessment.
Data Extraction: The tool supports data extraction.
Allows Multiple User Support: It is possible for two or more users to work on the same project at the same time, without seeing how others have voted.
Reference Allocation: It is possible to assign references to reviewers.
Discrepancy Resolving: There is official support for resolving conflict between reviewers.
Show Project Progress: the tool can display the progress of each reviewer and the overall project.
Attaching Comments: It is possible to add comments to a reference while screening.
Attaching PDFs: It is possible to upload PDFs for full-text screening.
Keyword Highlighting: It is possible to display highlighted keywords during screening.
Deduplication: The tool will identify and remove duplicate citations.
PRISMA Diagram: The tool can automatically generate a PRISMA flow diagram.
Import Multiple File Types: It is possible to import formatted references and the tool supports multiple file types.
Interrater Reliability: The tool can calculate and display interrater reliability.
Mobile Friendly Version: It is possible to screen on a mobile device.
Export Results File Types: References can be exported from the screening tool into the listed file types.
Systematic reviews aim to answer a specific research question. There are frameworks to help in question development and identification of search terms. PICO is the most popular framework utilized for clinical research topics.
The PICO question framework is useful for quantitative research topics. PICO questions identify four concepts: population, intervention, comparison, and outcome.
|Population / Patient||Who is the focus of my question?||infants diagnosed with necrotizing enterocolitis (NEC)|
|Intervention||What is the proposed, new intervention?||early enteral re-feeding|
|Comparison||What is the current or alternative intervention?||late enteral re-feeding|
|Outcome||What measurable outcome is affected?||NEC recurrence|
Research question: In infants diagnosed with necrotizing enterocolitis (NEC), what is the effect of early enteral re-feeding on NEC recurrence compared to late enteral re-feeding?
Did you know there are at least 25 other question frameworks besides variations of PICO? Frameworks like PEO, SPIDER, SPICE, ECLIPSE, and others can help you formulate a focused research question. The table and example below were created by the Medical University of South Carolina (MUSC) Libraries.
Click on a framework below to learn more and see an example of its use.
The PEO question framework is useful for qualitative research topics. PEO questions identify three concepts: population, exposure, and outcome.
|Population||Who is the focus of my question?||mothers|
|Exposure||What issue interests me?||postnatal depression|
|Outcome||What, in relation to the issue, do I want to examine?||daily living experiences|
Research question: What are the daily living experiences of mothers with postnatal depression?
The SPIDER question framework is useful for qualitative or mixed methods research topics focused on "samples" rather than populations.
SPIDER questions identify five concepts: sample, phenomenon of interest, design, evaluation, and research type.
|Sample||Who is the group of people being studied?||young parents|
|Phenomenon of Interest||What are the reasons for behavior and decisions?||attendance at antenatal education classes|
|Design||How has the research been collected (e.g., interview, survey)?||interviews|
|Evaluation||What is the outcome being impacted?||
|Research type||What type of research (qualitative or mixed methods)?||qualitative studies|
Research question: What are the experiences of young parents in attendance at antenatal education classes?
The SPICE question framework is useful for qualitative research topics evaluating the outcomes of a service, project, or intervention. SPICE questions identify five concepts: setting, perspective, intervention/exposure/interest, comparison, and evaluation.
|Setting||the context for the question (where)||South Carolina|
|Perspective||the users, potential users, or stakeholders of the service (for whom)||teenagers|
|Intervention / Interest / Exposure||the action taken for the users, potential users, or stakeholders (what)||provision of Quit Kits to support smoking cessation|
|Comparison||the alternative actions or outcomes (compared to what)||
no support or "cold turkey"
|Evaluation||the result or measurement that will determine the success of the intervention (what is the result, how well)||number of successful attempts to give up smoking with Quit Kits compared to number of successful attempts with no support|
Research question: For teenagers in South Carolina, what is the effect of provision of Quit Kits to support smoking cessation on number of successful attempts to give up smoking compared to no support ("cold turkey")?
The ECLIPSE framework is useful for qualitative research topics investigating the outcomes of a policy or service. ECLIPSE questions identify six concepts: expectation, client group, location, impact, professionals, and service.
|Expectation||What are you trying to improve or change? How is the information going to be used?||to increase access to wireless internet in the hospital|
|Client group||Who is expected to benefit from the service or policy?||patients and families|
|Location||Where is the service or policy located?||hospitals|
|Impact||What is the change in service or policy that the researcher is investigating?||clients have easy access to free internet|
|Professionals||Who is involved in providing or improving the service or policy?||information technology staff, hospital administration|
|Service||What kind of service or policy is this?||provision of free wireless internet to patients|
Research question: How can I increase access to wireless internet for hospital patients?
In order to reduce bias, eligibility criteria (also known as inclusion and exclusion criteria) refer to what you plan to include and exclude from your systematic review. These criteria are decided after the research question is developed but before searches are completed. Below are examples of criteria that may be used to determine inclusion or exclusion.
|Type of criteria||Example|
|Type of participants||May be limited to specific groups of people or age ranges|
|Study design||May include specific study designs and exclude others based on which best answer the research question|
|Intervention of interest||Includes interventions of interest and excludes any others|
|Outcomes of interest||Includes outcomes of interest and may exclude studies reporting outcomes not of interest|
|Setting||May be limited to a specific setting like inpatient, ambulatory, classroom, etc.|
|Type of publication||Reviews, editorials, commentaries, and letters are often excluded|
|Date of publication||Date ranges may be applied when updating a systematic review or when specific to an intervention or therapy|
|Language of publication||Non-English articles are often excluded from reviews; however, this may allow language bias to affect the quality of your review|