Skip to Main Content

Systematic Reviews: Step 5: Screen Citations

Created by Health Science Librarians

About Step 5: Screen Studies

It takes an average of 252 hours for a systematic review team to screen citations

In Step 5 you will screen all the articles you found to decide if they are relevant to your research question. You will:

  1. Use a systematic review tool like Covidence to organize articles.
  2. Read the title and abstract of every article and vote to include or exclude them.
  3. Resolve disputes with your teammates.
  4. Find the full text of all included articles.
  5. Read the full text of included articles to decide if they are relevant.

This page has information about using the systematic review tool Covidence to screen articles. Librarians can help you set up your project in Covidence.

Click an item below to see how it applies to Step 5: Screen Studies.

For your PRISMA flow diagram, you must track the number of articles you exclude at the title/abstract level and the number of articles and reasons for exclusion at the full text level. For information on how to document title/abstract and full text screening on your PRISMA flow diagram, visit our FAQ "How do I document title/abstract and full text screening on my PRISMA flow diagram?"

In Covidence, you can

  • Conduct title-abstract screening
  • Conduct full text screening
  • Tag references
  • Track review / reviewer progress
  • Resolve conflicts / ensure consensus between reviewers
  • Link full text to references



A librarian can advise you on study selection for your systematic review, including:

  • How to use Covidence to screen citations and manage the screening process
  • How to apply your review inclusion and exclusion criteria to the screening process
  • How to create a randomized sample of citations for a pilot test



Choose a tool for screening

There are many tools that can be useful for organizing the screening process, including Covidence, Covidence LogoRayyan, Abstrackr, and HAWC.   

UNC currently has an institutional subscription to Covidence making it available for free to UNC-affiliated users. HSL can provide classes and support for Covidence. To learn more visit the Covidence LibGuide.

HSL does not offer in-house support for other screening tools.  

All of these tools support reference importing of multiple file types (.ris & txt), in-/excluding references, exporting results, and tagging with keywords. Additionally, they are all web-based and therefore can be used on multiple operating systems, such as MacOS, Windows, and Linux. They cannot assist with protocol development or automated searching. The chart below provides more details to help you compare and choose a screening tool.


HSL does not offer in-house support for screening tools other than Covidence. The chart below can help you choose and compare screening tools. Links to training materials for these tools are provided below the table.
Feature: Covidence Rayyan Abstrackr HAWC
Cost UNC has a subscription to Covidence Free Free Free
Support available at UNC HSL Yes No No No
Quality Assessment Yes Yes No Yes
Data Extraction Yes  Yes No Yes
Allows Multiple User Support Yes Yes No No
Reference Allocation Yes Yes Yes No
Multiple Screening Phases Yes No No No
Discrepancy Resolving Yes Yes Yes No
Show Project Progress Yes Yes Yes No
Attaching Comments Yes No Yes No
Attaching PDFs Yes Yes No Yes
Keyword Highlighting Yes Yes Yes No
Deduplication Yes Yes No No
PRISMA Diagram Yes No No Yes
Interrater Reliability Yes Yes No No
Mobile Friendly Version Yes Yes No No
Export Results File Types RIS, CSV, TXT RIS, BibTex, CSV, EndNote RIS, CSV, XML XLSX

If the tools above don't meet your needs, you can also try this Excel tool called the VonVille Method.  

Information about screening tools and features: Van der Mierden, S., Tsaioun, K., Bleich, A., & Leenaars, C. H. C. (2019). Software tools for literature screening in systematic reviews in biomedical research. ALTEX : Alternativen zu Tierexperimenten, 36(3), 508-517.


Screening tool training resources


Screening tool feature definitions

Quality Assessment: The tool supports risk of bias and quality assessment. 

Data Extraction: The tool supports data extraction. 

Allows Multiple User Support: It is possible for two or more users to work on the same project at the same time, without seeing how others have voted.

Reference Allocation: It is possible to assign references to reviewers. 

Discrepancy Resolving: There is official support for resolving conflict between reviewers. 

Show Project Progress: the tool can display the progress of each reviewer and the overall project.

Attaching Comments: It is possible to add comments to a reference while screening.

Attaching PDFs: It is possible to upload PDFs for full-text screening. 

Keyword Highlighting: It is possible to display highlighted keywords during screening.

Deduplication: The tool will identify and remove duplicate citations. 

PRISMA Diagram: The tool can automatically generate a PRISMA flow diagram. 

Import Multiple File Types: It is possible to import formatted references and the tool supports multiple file types. 

Interrater Reliability: The tool can calculate and display interrater reliability.

Mobile Friendly Version: It is possible to screen on a mobile device. 

Export Results File Types: References can be exported from the screening tool into the listed file types. 


Import citations into Covidence

For information on how to sign up for a Covidence account, create a review, or customize your review settings, visit our Covidence guide.

Once you have created a review for your project in Covidence:

  1. Go to your The import button in a Covidence reviewReview Summary page and click Import Studies.
  2. In the dropdown menu below "Import in to", select the folder where you would like the citations to go.
  3. Browse to and attach your citation file in XML, CRS, or RIS text format.
  4. Then click "Import". You should see your citations appear in your systematic review on Covidence within the next few minutes. Large files will take longer to import.

Screen results

Once you have completed literature searching and gathered all citations, it is time to screen the results. The purpose of screening is to remove studies that do not meet your inclusion criteria. Two independent reviewers should screen all studies, starting with a title and abstract screening, followed by a full-text screening. A third reviewer or the review team should resolve any conflicts.

Remember, all voting should be blinded, meaning team members should be unable to see how others cast their votes.

During the title/abstract screening, for each reference, each reviewer should read the title and abstract and make aNo Maybe and Yes buttons in Covidence decision:

  • No: This reference does not meet inclusion criteria and should not be included in the systematic review.
  • Maybe: There is not enough information in the title/abstract to make a decision (move to full-text screening stage).
  • Yes: This reference appears to meet inclusion criteria and should move to the full-text screening stage.

After you complete title/abstract screening, you will need to review any conflicts between reviewers and decide if those citations should move forward to full text screening or be marked as irrelevant.

During the full-text screening, for each reference, read the full-text and make a decision:List of potential reasons for exclusion in CovidenceInclude and exclude buttons in Covidence

  • Include: This reference meets inclusion criteria and should be included in the systematic review.
  • Exclude: This reference does not meet inclusion criteria and should not be included in the systematic review.

After full-text screening, you will need to review any citations where there are different opinions about inclusion/exclusion, as well as different reasons for excluding.

Note: The example images we use are from Covidence.  Other systematic review screening tools may have different appearances and features.

Helpful tip - Pilot testing and screening

Lightbulb- Helpful Tip

Test screening or pilot screening usually involves choosing a random sample of citations from your results, then having all reviewers screen that sample to ensure consistency across responses.  It can be performed in Covidence or Excel. While pilot screening may take some extra time early on, it may make the screening process faster overall because screeners will be more likely to apply inclusion and exclusion criteria properly, leading to fewer errors and conflicts.

Reporting Standards for Systematic Reviews (PRISMA)

"PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses. The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram." (From )

In your review, you will need to track the number of results from each database you search, the number of duplicates you remove, the decisions you make on each article, and in the full text level, the reasons why you exclude articles. More information and a step-by-step guide can be found on the Using PRISMA for Reporting page on this guide.

PRISMA and Covidence:

Covidence will track numbers of citations screened at each stage and how many are ultimately included in the review. These numbers can be copied and pasted into a PRISMA diagram template. You will still need to record the number of results from each databases search and the number of duplicates removed from your set of articles. 

Need Covidence training?

Covidence Webinars
Covidence demonstrates their features for new users in monthly webinars.  You can also listen to a recording of a recent Covidence webinar.

Training Sessions
The HSL holds regular Covidence training sessions throughout the semester. To sign up for a Covidence training session, please visit our Classes Page. 

Individual or Group Consultations
If you are unable to attend one of the group training sessions or have additional questions after a training session, you can request a Covidence consultation on the HSL website at any time.