Skip to Main Content

Systematic Reviews: Step 5: Screen Citations

Created by Health Science Librarians

About Step 5: Screen Studies

It takes an average of 252 hours for a systematic review team to screen citations

Click an item below to see how it applies to Step 5: Screen Studies.

For your PRISMA flow diagram, you must track the number of articles you exclude at the title/abstract level and the number of articles and reasons for exclusion at the full text level. For information on how to document title/abstract and full text screening on your PRISMA flow diagram, visit our FAQ "How do I document title/abstract and full text screening on my PRISMA flow diagram?"

In Covidence, you can

  • Conduct title-abstract screening
  • Conduct full text screening
  • Tag references
  • Track review / reviewer progress
  • Resolve conflicts / ensure consensus between reviewers
  • Link full text to references

 

 

A librarian can advise you on study selection for your systematic review, including:

  • How to use Covidence to screen citations and manage the screening process
  • How to apply your review inclusion and exclusion criteria to the screening process
  • How to create a randomized sample of citations for a pilot test

 

 

Choose a tool for screening

There are many tools that can be useful for organizing the screening process, including Covidence, Covidence LogoRayyan, Abstrackr, and HAWC.   

UNC currently has an institutional subscription to Covidence making it available for free to UNC-affiliated users. HSL can provide classes and support for Covidence. To learn more visit the Covidence LibGuide.

HSL does not currently offer in-house support for other screening tools.  

All of these tools support reference importing of multiple file types (.ris & txt), in-/excluding references, exporting results, and tagging with keywords. Additionally, they are all web-based and therefore can be used on multiple operating systems, such as MacOS, Windows, and Linux. They cannot assist with protocol development or automated searching. The chart below provides more details to help you compare and choose a screening tool.

 

The table compares Covidence, Rayyan, Abstrackr, and HAWC (from left to right) based on a number of features. HSL does not currently offer in-house support for other screening tools; however, links to training materials are provided below the table.
Feature: Covidence Rayyan Abstrackr HAWC
Cost UNC has a subscription to Covidence Free Free Free
Support available at UNC HSL Yes No No No
Quality Assessment Yes Yes No Yes
Data Extraction Yes  Yes No Yes
Allows Multiple User Support Yes Yes No No
Reference Allocation Yes Yes Yes No
Multiple Screening Phases Yes No No No
Discrepancy Resolving Yes Yes Yes No
Show Project Progress Yes Yes Yes No
Attaching Comments Yes No Yes No
Attaching PDFs Yes Yes No Yes
Keyword Highlighting Yes Yes Yes No
Deduplication Yes Yes No No
PRISMA Diagram Yes No No Yes
Interrater Reliability Yes Yes No No
Mobile Friendly Version Yes Yes No No
Export Results File Types RIS, CSV, TXT RIS, BibTex, CSV, EndNote RIS, CSV, XML XLSX

If the tools above don't meet your needs, you can also try this Excel tool called the VonVille Method.  

Information about screening tools and features: Van der Mierden, S., Tsaioun, K., Bleich, A., & Leenaars, C. H. C. (2019). Software tools for literature screening in systematic reviews in biomedical research. ALTEX : Alternativen zu Tierexperimenten, 36(3), 508-517.

 

Screening tool training resources

 

Screening tool feature definitions

Quality Assessment: The tool supports risk of bias and quality assessment. 

Data Extraction: The tool supports data extraction. 

Allows Multiple User Support: It is possible for two or more users to work on the same project at the same time, without seeing how others have voted.

Reference Allocation: It is possible to assign references to reviewers. 

Discrepancy Resolving: There is official support for resolving conflict between reviewers. 

Show Project Progress: the tool can display the progress of each reviewer and the overall project.

Attaching Comments: It is possible to add comments to a reference while screening.

Attaching PDFs: It is possible to upload PDFs for full-text screening. 

Keyword Highlighting: It is possible to display highlighted keywords during screening.

Deduplication: The tool will identify and remove duplicate citations. 

PRISMA Diagram: The tool can automatically generate a PRISMA flow diagram. 

Import Multiple File Types: It is possible to import formatted references and the tool supports multiple file types. 

Interrater Reliability: The tool can calculate and display interrater reliability.

Mobile Friendly Version: It is possible to screen on a mobile device. 

Export Results File Types: References can be exported from the screening tool into the listed file types. 

 

Import citations into Covidence

For information on how to sign up for a Covidence account, create a review, or customize your review settings, visit our Covidence guide.

Once you have created a review for your project in Covidence:

  1. Navigate to your The import button in a Covidence reviewReview Summary page and click Import Studies.
     
  2. In the dropdown menu below Import in to, select the folder where you would like the citations to go (Screen).
     
  3. Browse to and attach your citation file in XML, CRS, or RIS text format.
     
  4. Then click Import. It may take a couple of minutes, but you should see your citations appear in your systematic review on Covidence within the next few minutes.

Screen results

Once you have completed literature searching and compiled all citations, it is time to screen the results. The purpose of screening is to eliminate studies that do not meet your inclusion criteria. Two independent reviewers should screen all studies, starting with a title and abstract screening, followed by a full-text screening. A third reviewer or the review team should resolve any conflicts.

Remember, all voting should be blinded, meaning team members should be unable to see how others cast their votes.


During the title/abstract screening, for each reference, each reviewer should read the title and abstract and make aNo Maybe and Yes buttons in Covidence decision:

  • No: This reference does not meet inclusion criteria and should not be included in the systematic review.
  • Maybe: There is not enough information in the title/abstract to make a decision (move to full-text screening stage).
  • Yes: This reference appears to meet inclusion criteria and should move to the full-text screening stage.

After you complete title/abstract screening, you will need to review any conflicts between reviewers and resolve if references should move forward to full text screening or be marked as irrelevant.


During the full-text screening, for each reference, read the full-text and make a decision:List of potential reasons for exclusion in CovidenceInclude and exclude buttons in Covidence

  • Include: This reference meets inclusion criteria and should be included in the systematic review.
  • Exclude: This reference does not meet inclusion criteria and should not be included in the systematic review.

After full-text screening, you will need to review any conflicting decisions, as well as conflicting reasons for exclusion.

 

UNC currently has an institutional subscription to Covidence making it available for free to UNC-affiliated users. HSL can provide classes and support for Covidence. To learn more visit the Covidence LibGuide.

Note: The example images we use are from Covidence.  Other systematic review screening tools may have different appearances and features.

Helpful tip - Preliminary testing and screening

Lightbulb- Helpful Tip

Test screening or pilot screening usually involves choosing a random sample of citations from your results, then having all reviewers screen that sample to ensure consistency across responses.  It can be performed in Covidence or Excel. While adding this step may add some time initially, it will most likely expedite both screening phases as screeners will be more comfortable applying the inclusion and exclusion criteria properly and most likely result in less error or conflicts.

Reporting Standards for Systematic Reviews (PRISMA)

"PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses. The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram." (From prisma-statement.org )

In your review, you will need to track the number of results from each database you search, the number of duplicates you remove, the decisions you make on each article, and in the full text level, the reasons why you exclude articles. More information and a step-by-step guide can be found on the Using PRISMA for Reporting page on this guide.


PRISMA and Covidence:

Covidence will track numbers of citations screened at each stage and how many are ultimately included in the review. These numbers can be copied and pasted into a PRISMA diagram template. You will still need to record the number of results from each databases search and the number of duplicates removed from your set of articles. 

Need Covidence training?

Covidence Webinars
Covidence demonstrates their features for new users in monthly webinars.  You can also listen to a recording of a recent Covidence webinar.

Training Sessions
The HSL holds regular Covidence training sessions throughout the semester. To sign up for a Covidence training session, please visit our Classes Page. 

Individual or Group Consultations
If you are unable to attend one of the group training sessions or have additional questions after a training session, you can request a Covidence consultation on the HSL website at any time.