Covidence: 10 Tips for Using Covidence Effectively

When added, inclusion and exclusion reasons are viewable while screening citations. To add:

1. 1. Click the “add criteria” button at the top of the screening page. This will redirect you to the review settings page, where inclusion and exclusion reasons can be added or edited.
   2. Once you've added inclusion and exclusion criteria, you'll see the “show criteria” button at the top of the screening page, to make them viewable while screening.

More information on managing inclusion and exclusion criteria can be found here: https://support.covidence.org/help/creating-and-viewing-inclusion-criteria.

Unlike in the title and abstract screening phase, during the full-text screening phase, a reason will need to be selected each time an article is excluded. Covidence can help manage these reasons, as well as keep up with exclusion counts for the PRISMA flow chart.

1. 1. Click the “settings” button in the top, right corner of the review dashboard.
   2. Click the “criteria & exclusion reasons” tab.
   3. Click the “manage criteria for screening and full text review” tab.
   4. Then, click “manage exclusion reasons for full-text review” to add your list. Delete any default reasons that are not applicable to your review. Number or arrange exclusion reasons in a hierarchy or priority order, if applicable. It is important to note that even if two reviewers both vote to exclude, if they pick differing exclusion reasons, a conflict will still be noted and need to be resolved. To save time, it is helpful to pick a priority order for selecting these reasons. For instance, it can be the first reason encountered when reading the article from start to finish, provided reviewers read in that order. Or reasons can be ordered from broadest (not in English language, abstract only, etc.) to narrowest, or vice versa.
   5. Alternatively, exclusion reasons can be added while performing full-text review if needed. To do this, first click the “exclude” button for the study you want to exclude. This action will reveal a dropdown menu asking you to specify your reason for excluding the study. Select the “edit this list” option from the dropdown. You'll then see a pop-up prompting you to add an exclusion reason. Enter your exclusion reason in the text box provided and click the “add” button. Your new exclusion reason will appear at the top of the list.

More information on managing exclusion reasons can be found here: https://support.covidence.org/help/customising-reasons-for-exclusion.

Covidence allows users to enter key words or phrases, which will be highlighted in green and red, to indicate potential inclusion or exclusion. Highlighting words of interest can make the screening process faster. To add highlights:

1. Click the “settings” button in the top right corner of the review dashboard.
2. Click the “criteria & exclusion reasons” tab.
3. Click “manage highlights”.
4. Add specific keywords or phrases for inclusion and exclusion criteria.
5. On the screening page, click “show highlights” to see these terms highlighted in green and red.

More information on adding highlights can be found here: https://support.covidence.org/help/highlighting-keywords.

Covidence offers the ability to create customizable tags that can be added to citations. Tags can be added for any reason, but often are added to note where the citation may be useful to cite in your review (i.e., introduction, discussion, etc.). Additionally, tags are also useful in assigning a topic or specific reviewer with an expertise to complete data extraction later in the review process. Two tags are available by default, “awaiting classification” and “ongoing study tags”, and cannot be deleted.

To add a customizable tag:

1. 1. Click the checkbox next to the citation you want to tag.
   2. Click the “tags” dropdown in the header of the screening page. Click a listed tag, or type a new reason in the box under “tag with:”. Tags can also be added from the review settings page under the “study tags” tab.

Use the filter function to limit to only citations with specific tags. Keep in mind, only citations in your current list with tags will display (i.e., if a study had a tag but has been excluded, it will no longer appear when filtered).

More information on tagging citations can be found here: https://support.covidence.org/help/tagging.

Covidence categorizes a review’s citations into six lists: screening, review, irrelevant, included, excluded, and those without full text. Each of these can be exported in CSV format, along with any associated tags and notes. To export a list:

1. Click the “export” button in the top right corner of the Review Summary page.
2. In the first dropdown menu in the “references” box (labelled “options”), select the list you’d like to export.
3. In the second dropdown menu in the “references” box (labelled “format”), select CSV. Click “export”.
4. Wait for the file to prepare, then click “download”.
5. Save and open your file.

Lists can also be exported in RIS format, which can be used in many citation managers, including EndNote, Sciwheel, Zotero, and Mendeley. To export lists this way, select RIS in the second dropdown menu in the “references” box.

More information on exporting citations can be found here: https://support.covidence.org/help/exporting-study-lists-to-your-reference-manager.

Before commencing formal screening, it is beneficial for reviewers to pilot screen a subset of citations to ensure all have the same understanding and interpretation of inclusion and exclusion criteria. Teams may wish to conduct pilot screening until reviewers reach a certain percentage of agreement. Covidence can assist with this process by reporting inter-rater reliability data, including Cohen’s Kappa coefficient, of screeners.

1. Once screening, in either the title and abstract or full-text phase, has been completed, click the “export” button in the top, right of the review dashboard.
2. In the box labeled “inter-rater reliability”, select the stage for which you’d like to evaluate, and then click “export CSV”.

This file will include reviewer comparison data, as well as various inter-rater calculations and Cohen’s Kappa coefficient. Keep in mind that resolving conflicts does not affect inter-rater reliability scores, “maybe” votes are calculated as “yes” votes, and that only final votes are included in these calculations (i.e., votes that are undone and re-cast are not calculated).

More information on exporting inter-rater reliability data can be found here: https://support.covidence.org/help/exporting-inter-rater-reliability-data.

If you need to change your own vote after it has been cast but before a subsequent vote moves it to another step, you can change your own vote from the “awaiting other reviewer” tab of the current step. Your current vote is displayed in dark grey. Votes can be changed by selecting the light grey “change vote to ___” option(s).

If a reference has been moved beyond the screening phase but needs to be moved back and voted on again, click the "move to ___" button below the abstract pane. If it’s in the full text review phase, this will return it to the screening phase. If it has progressed to included/excluded references, it will be returned to full text review.

More information on undoing a vote can be found here: https://support.covidence.org/help/can-i-undo-a-vote.

Covidence does not allow for a subset of citations to be assigned, or allocated, to specific reviewers for screening. This design is intentional and designed in this manner in order to reduce potential bias.

To divide the screening burden among reviewers, Covidence suggests having each reviewer track their overall contribution using the “team settings” button under the team progress bar of each stage of screening.

More information on allocating studies to specific reviewers can be found here: https://support.covidence.org/help/can-i-allocate-a-set-of-studies-to-a-specific-reviewer-for-screening.

A provisional PRISMA diagram of your review can be accessed at any time from the top, right of the review dashboard, by clicking the “PRISMA” button. Covidence will the progress of citations in accordance with the PRISMA flow chart; however, only counts can be exported from Covidence, not a usable PRISMA flow chart.

Covidence counts citations as they are imported and continuously updates this information throughout the review process. The box that lists exclusion numbers, and reasons during the full-text review stage. Current PRISMA numbers (counts), can be exported from Covidence as text by clicking “export data” at the top right of the diagram. Then, you can manually enter them in the official PRISMA flow chart available at https://www.prisma-statement.org/prisma-2020-flow-diagram.  

More information on using Covidence to track citation counts can be found here: https://support.covidence.org/help/export-prisma.

References identified by snowballing or other hand searching methods may be tracked with Covidence. They can be imported to Covidence in the usual way. In order to separate them from studies obtained via the review’s database search strategy, only import snowballed citations after the first round of screening is complete. To identify these accurately for later reporting, do the following:

1. In the “study tags” tab of your review settings, add a tag reading “Snowballing” to your review.
2. Import the snowballed references into the Title and abstract screening stage in the usual way.
3. Click “all” at the top to select all newly imported references.
4. Click “tags” and select “Snowballing”.
5. You should now see all new references have been tagged with “Snowballing”.

When screening is complete for this set, check the “full text review” stage to see how many snowballed references made it through. Note this number for your records. After reviewing is complete, conduct an export as follows:

1. Click the “export” button in the top right corner of the Review Summary page.
2. In the first dropdown menu in the “references” box (labelled “options”), select “excluded”.
3. In the second dropdown menu in the “references” box (labelled “format”), select CSV. Click “export”.
4. Once you open the downloaded file, note any “Snowballing” tagged studies and the reasons for their exclusion.

More information on using Covidence to track “snowballed” citations can be found here: https://support.covidence.org/help/tracking-and-reporting-snowballed-references.