Skip to Main Content

Scoping Reviews: Step 7: Extract Data from Included Studies

Created by Health Science Librarians

About Step 7: Extract Data from Included Studies

It takes an average of 88 hours for a systematic review team to extract data from included studies

Click an item below to see how it applies to Step 7: Extract Data from Included Studies.

If you reach the data extraction step and choose to exclude articles for any reason, update the number of included and excluded studies in your PRISMA flow diagram.

Covidence allows you to assemble a custom data extraction template, have two reviewers conduct extraction, then send their extractions for consensus.

 

A librarian can advise you on data extraction for your systematic review, including: 

  • What the data extraction stage of the review entails
  • Finding examples in the literature of similar reviews and their completed data tables
  • How to choose what data to extract from your included articles 
  • How to create a randomized sample of citations for a pilot test
  • Best practices for reporting your included studies and their important data in your review

 

About data extraction (charting)

In this step of the scoping review, you will develop your evidence tables, which give detailed information for each study (perhaps using a PICO framework as a guide), and summary tables, which give a high-level overview of the findings of your review. You can create evidence and summary tables to describe study characteristics, results, or both. These tables will help you determine which studies, if any, are eligible for quantitative synthesis.

Data extraction (charting) requires a lot of planning.  We will review some of the tools you can use for data extraction (charting), the types of information you will want to extract, and the options available in the scoping review software used here at UNC, Covidence.

"‘Charting’ (Ritchie and Spencer 1994) describes a technique for synthesising and interpreting qualitative data by sifting, charting and sorting material according to key issues and themes, a similar process to the one we adopted hence we have borrowed the term. In a systematic review, this process would be called ‘data extraction’ and, in the case of meta-analysis, might involve specific statistical techniques."¹ -p.15

How many people should extract data?

The Cochrane Handbook and other studies strongly suggest at least two reviewers and extractors to reduce the number of errors.

Select a data extraction (charting) tool

Click on a type of data extraction (charting) tool below to see some more information about using that type of tool and what UNC has to offer.

Most systematic review software tools have data extraction functionality that can save you time and effort.  Here at UNC, we use a systematic review software called Covidence. You can see a more complete list of options in the Systematic Review Toolbox.

Covidence allows you to create and publish a data extraction template with text fields, single-choice items, section headings and section subheadings; perform dual and single reviewer data extraction; review extractions for consensus; and export data extraction and quality assessment to a CSV with each item in a column and each study in a row.

 

You can also use spreadsheet or database software to create custom extraction forms. Spreadsheet software (such as Microsoft Excel) has functions such as drop-down menus and range checks can speed up the process and help prevent data entry errors. Relational databases (such as Microsoft Access) can help you extract information from different categories like citation details, demographics, participant selection, intervention, outcomes, etc.

 

RevMan offers collection forms for descriptive information on population, interventions, and outcomes, and quality assessments, as well as for data for analysis and forest plots. The form elements may not be changed, and data must be entered manually. RevMan is a free software download.

 

Survey or form tools can help you create custom forms with many different question types, such as multiple choice, drop downs, ranking, and more. Content from these tools can often be exported to spreadsheet or database software as well. Here at UNC we have access to the survey/form software Qualtrics & Poll Everywhere.

 

In the past, people often used paper and pencil to record the data they extracted from articles. Handwritten extraction is less popular now due to widespread electronic tools. You can record extracted data in electronic tables or forms created in Microsoft Word or other word processing programs, but this process may take longer than many of our previously listed methods. If chosen, the electronic document or paper-and-pencil extraction methods should only be used for small reviews, as larger sets of articles may become unwieldy. These methods may also be more prone to errors in data entry than some of the more automated methods.

 

There are benefits and limitations to each method of data extraction (charting).  You will want to consider:

  • The cost of the software / tool
  • Shareability / versioning
  • Existing versus custom data extraction (charting) forms
  • The data entry process
  • Interrater reliability

For example, in Covidence you may spend more time building your data extraction (charting) form, but save time later in the extraction process as Covidence can automatically highlight discrepancies for review and resolution between different extractors. Excel may require less time investment to create an extraction form, but it may take longer for you to match and compare data between extractors. More in-depth comparison of the benefits and limitations of each extraction tool can be found in the table below.

Benefits and Limitations of Data Extraction (Charting) Tools
Tool Benefits Limitations
Scoping Review Software (Covidence)
  • Free for UNC affiliates through UNC Libraries' subscription
  • Review elements are housed in single system
  • Discrepancies are automatically highlighted for resolution
  • Can calculate interrater reliability
  • Better assurance of blinding during the extraction process
  • Read PDFs of articles and extract data in side-by-side panel 
  • Subscription-based to create more than 3 reviews (provided by Libraries for UNC affiliates)
  • Steeper learning curve to create and customize extraction forms
Spreadsheets (Excel, Google Sheets)
  • Free options available
  • Easy to learn and use (i.e., extractors will be able to begin quickly compared to using other software)
  • Easy to customize extraction fields
  • Manually review, find, and resolve discrepancies
  • Increase in potential bias if all extractors are using or have access to the same file (e.g., issues with blinding data extracted)
  • Potential for more errors and less accuracy due to manual data entry and review
Cochrane Revman
  • Free
  • Compatible with Covidence
  • Capabilities to write the entire review using this software
  • Steeper learning curve to learn new software
Survey or Form Software (Poll Everywhere, Qualtrics, etc.)
  • Free (limited) and paid versions
  • Better assurance of blinding during the extraction process
  • Extractors may be more familiar with using this interface compared to scoping review software (Covidence)
  • Free versions may have limited question or response options
  • Need to ensure data is downloadable or able to be exported in a useable format
Electronic documents (Word, Google Docs)
  • Free options available
  • Easy to learn and use (i.e., extractors will be able to begin quickly compared to using other software)
  • Easy to customize extraction fields
  • Manually review, find, and resolve discrepancies
  • Increase in potential bias if all extractors are using or have access to the same file (e.g., issues with blinding data extracted)
  • Potential for more errors and less accuracy due to manual data entry and review

 

What should I extract?

You should plan to extract data that is relevant to Sample information to include in an extraction tableanswering the question posed in your scoping review. 

It may help to consult other similar scoping reviews to identify what data to collect or to think about your question in a framework such as PICO.

Helpful data for an intervention question may include:

  • Information about the article
    (author(s), year of publication, title, DOI)
  • Information about the study
    (study type, participant recruitment / selection / allocation, level of evidence, study quality)
  • Patient demographics
    (age, sex, ethnicity, diseases / conditions, other characteristics related to the intervention / outcome)
  • Intervention
    (quantity, dosage, route of administration, format, duration, time frame, setting)
  • Outcomes
    (quantitative and / or qualitative)

If you plan to synthesize data, you will want to collect additional information such as sample sizes, effect sizes, dependent variables, reliability measures, pre-test data, post-test data, follow-up data, and statistical tests used.

Extraction templates and approaches should be determined by the needs of the specific review.   For example, if you are extracting qualitative data, you will want to extract data such as theoretical framework, data collection method, or role of the researcher and their potential bias.

Helpful tip- Data extraction (charting)

Lightbulb- Helpful Tip

  • Look for an existing extraction form or tool to help guide you.  Use existing scoping reviews on your topic to identify what information to collect if you are not sure what to do.
  • Train the review team on the extraction categories and what type of data would be expected.  A manual or guide may help your team establish standards.
  • Pilot the extraction / coding form to ensure data extractors are recording similar data. Revise the extraction form if needed.
  • Discuss any discrepancies in coding throughout the process.
  • Document any changes to the process or the form.  Keep track of the decisions the team makes and the reasoning behind them.