Scoping Reviews Legacy Guide: Data Extraction

In this step of the scoping review, you will develop your evidence tables, which give detailed information for each study (perhaps using a PICO framework as a guide), and summary tables, which give a high-level overview of the findings of your review. You can create evidence and summary tables to describe study characteristics, results, or both. These tables will help you determine which studies, if any, are eligible for quantitative synthesis.

"In scoping reviews, the results and discussion focus on charting concepts, themes, and types of evidence available rather than strength of evidence and quantitative findings." - Chang 2018

Data extraction requires a lot of planning.  We will review some of the tools you can use for data extraction, the types of information you will want to extract, and the options available in the review software used here at UNC, Covidence.

How many people should extract data?

The Cochrane Handbook and other studies strongly suggest at least two reviewers and extractors to reduce the number of errors.

Data extraction tips

1. Look for an existing extraction form or tool to help guide you.  Use existing scoping reviews on similar topic to identify what information to collect if you are not sure what to do.
2. Train the review team on the extraction categories and what type of data would be expected.  A manual or guide may help your team establish standards.
3. Pilot the extraction / coding form to ensure data extractors are recording similar data. Revise the extraction form if needed.
4. Discuss any discrepancies in coding throughout the process.
5. Document any changes to the process or the form.  Keep track of the decisions the team makes and the reasoning behind them.

Screening & Extraction Software

Most review software tools have data extraction functionality that can save you time and effort.  Here at UNC, we use a review software called Covidence. You can see a more complete list of options in the Systematic Review Toolbox.

Covidence allows you to

• Create and publish a data extraction template with text fields, single-choice items, section headings and section subheadings.
• Create and publish a quality assessment template with domains.
• View your list of included studies.
• Merge included studies.
• Perform dual and single reviewer data extraction and quality assessment.
• Review extractions for consensus.
• Update a study’s extraction form once it has been opened.
• Export data extraction and quality assessment to a CSV with each item in a column and each study in a row.

Covidence's new data extraction features are detailed in the "Data Extraction in Covidence"  box at the end of this page.

Spreadsheet or Database Software

You can also use spreadsheet or database software to create custom extraction forms. Spreadsheet functions such as drop-down menus and range checks can speed up the process and help prevent data entry errors. Relational databases (such as Microsoft Access) can help you extract information from different categories like citation details, demographics, participant selection, intervention, outcomes, etc.

Form or Survey Software

Survey or form tools can help you create custom forms with many different question types, such as multiple choice, drop downs, ranking, and more.  Content from these tools can often be exported to spreadsheet or database software as well.  Here at UNC we have access to the survey/form software Qualtrics.

Electronic Documents or Paper & Pencil

In the past, people often used paper and pencil to record the data they extracted from articles. Handwritten extraction is less popular now due to widespread electronic tools.  You can record extracted data in electronic tables or forms created in Microsoft Word or other word processing programs, but this process may take longer than many of our previously listed methods.  If chosen, the electronic document or paper-and-pencil extraction methods should only be used for small reviews, as larger sets of articles may become unwieldy. These methods may also be more prone to errors in data entry than some of the more automated methods.

• Elamin MB, Flynn DN, Bassler D, et al. Choice of data extraction tools for systematic reviews depends on resources and review complexity. Journal of Clinical Epidemiology. 2009;62(5):506-510. 10.1016/j.jclinepi.2008.10.016

You should plan to extract data that is relevant to answering the questions posed in your scoping review.  As mentioned previously, it may help to consult other similar scoping reviews to identify what data to collect.  You should use your key questions and your eligibility criteria as a starting point.  It can also help to think about your question in a framework such as PICO.

Helpful data may include:

• Information about the study (author(s), year of publication, title, DOI)
• Demographics (age, sex, ethnicity, diseases / conditions, other characteristics related to the intervention / outcome)
• Methodology (study type, participant recruitment / selection / allocation, level of evidence, study quality)
• Intervention (quantity, dosage, route of administration, format, duration, time frame, setting)
• Outcomes (quantitative and / or qualitative)

If you plan to synthesize data, you will want to collect additional information such as sample sizes, effect sizes, dependent variables, reliability measures, pre-test data, post-test data, follow-up data, and statistical tests used.