Scoping Reviews Legacy Guide: Quality Assessment
Created by Health Science Librarians
Role of the librarian in this stage
A librarian can advise you on quality assessment for your scoping review, including:
- What the quality assessment or risk of bias stage of the review entails
- How to choose an appropriate quality assessment tool
- Best practices for reporting quality assessment results in your review
Contact HSL About Your Review
Ready to start a scoping review? HSL Librarians can help!
Fill out the Systematic/Scoping Review Request Form and the best-suited librarian will get back to you promptly. Our systematic/ scoping review service is only available to faculty, staff, students, and others who are affiliated with UNC Chapel Hill.
Scoping Reviews and Quality Assessment
One of the main differences between systematic and scoping reviews is found in the quality assessment stage. Quality assessment, or risk of bias, occurs when researchers assess individual studies, and the overall body of literature, for elements of bias. While necessary when undertaking a systematic review, quality assessment is not a required element of scoping reviews.
If you decide quality assessment is a necessary component to meet your review's objectives, clearly state those objectives and rationale for completing in your review. When picking a quality assessment tool, make sure the study design it evaluates matches that/those of your included studies. Several examples of quality assessment tools are listed below and grouped by study design.
Using Covidence for Quality Assessment
Covidence uses Cochrane Risk of Bias (which is designed for rating randomized controlled trials and cannot be used for other study types) as the default tool for quality assessment of included studies. You can opt to manually customize the quality assessment template and use a different tool better suited to your review. More information about quality assessment using Covidence, including how to customize the quality assessment template, can be found below. If you decide to customize the quality assessment template, you cannot switch back to using the Cochrane Risk of Bias template.
- Covidence FAQs on Quality AssessmentCommonly asked questions about quality assessment using Covidence
- Covidence YouTube ChannelA collection of Covidence-created videos
Systematic Reviews and Meta-Analyses
- JBI Critical Appraisal Checklist for Systematic Reviews and Research SynthesesAn 11-item checklist for evaluating systematic reviews
- AMSTAR ChecklistA 16-question measurement tool to assess systematic reviews
- AHRQ Methods Guide for Effectiveness and Comparative Effectiveness ReviewsThe AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews, provides detailed information on selecting eligibility criteria, searching the literature, data extraction, quality assessment, and other steps in the creation of a systematic review.
- CASP - Systematic ReviewA checklist for quality assessment of systematic review from the Critical Appraisal Skills Programme
- Scottish Intercollegiate Guidelines Network (SIGN)Checklists for quality assessment
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Clinical Practice Guidelines
- National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) InstrumentA 15-item instrument using a scale of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trust worth guidelines
- AGREE-II Appraisal of Guidelines for Research and EvaluationThe Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument evaluates the process of practice guideline development and the quality of reporting
Randomized Controlled Trials (RCTs)
- Cochrane Risk of Bias (ROB) 2.0 ToolTemplates are tailored to randomized parallel-group trials, cluster-randomized parallel-group trails (including stepped-wedge designs), and randomized cross-over trails and other matched designs.
- CASP- Randomized Controlled Trial Appraisal ToolA checklist for RCTs created by the Critical Appraisal Skills Program (CASP)
- The Jadad ScaleThe Jadad scale assesses the quality of published clinical trials based methods relevant to random assignment, double blinding, and the flow of patients.
- CEBM-RCTA critical appraisal tool for RCTs from the Centre for Evidence Based Medicine (CEBM)
- Checklist for Randomized Controlled Trials (JBI)A critical appraisal checklist from the Joanna Briggs Institute (JBI)
- Scottish Intercollegiate Guidelines Network (SIGN)Checklists for quality assessment
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Cohort Studies
- CASP- Cohort StudiesA checklist created by the Critical Appraisal Skills Programme (CASP) to assess key criteria relevant to cohort studies
- Checklist for Cohort Studies (JBI)A checklist for cohort studies from the Joanna Briggs Institute
- The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analysesValidated tool for assessing case-control and cohort studies.
- Scottish Intercollegiate Guidelines Network (SIGN)Checklists for quality assessment
- STROBE ChecklistA checklist for quality assessment of case-control, cohort, and cross-sectional studies
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Case-Control Studies
- CASP- Case Control StudyA checklist created by the Critical Appraisal Skills Programme (CASP) to assess key criteria relevant to case-control studies
- STROBE ChecklistA checklist for quality assessment of case-control, cohort, and cross-sectional studies
- Tool to Assess Risk of Bias in Case Control Studies by the CLARITY Group at McMaster UniversityA quality assessment tool for case-control studies from the CLARITY Group at McMaster University
- Scottish Intercollegiate Guidelines Network (SIGN)Checklists for quality assessment
- The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analysesValidated tool for assessing case-control and cohort studies.
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
- JBI Checklist for Case-Control StudiesA checklist created by the Joanna Briggs Institute
Cross-Sectional Studies
- STROBE ChecklistA checklist for quality assessment of case-control, cohort, and cross-sectional studies
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Diagnostic Studies
- CASP- Diagnostic StudiesA checklist for diagnostic studies created by the Critical Appraisal Skills Program (CASP)
- QUADAS-2A quality assessment tool developed by a team at the Bristol Medical School: Population Health Sciences at the University of Bristol
- Critical Appraisal Checklist for Diagnostic Test Accuracy Studies (JBI)A checklist for quality assessment of diagnostic studies developed by the Joanna Briggs Institute
- Scottish Intercollegiate Guidelines Network (SIGN)Checklists for quality assessment
Economic Studies
- Consensus Health Economic Criteria (CHEC) List19 yes-or-no questions, one for each category to assess economic evaluations
- CASP- Economic EvaluationA checklist for quality assessment of economic studies by the Critical Appraisal Skills Programme
- Scottish Intercollegiate Guidelines Network (SIGN)Checklists for quality assessment
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Mixed Methods
- McGill Mixed Methods Appraisal Tool (MMAT) 2018 User GuideSee full site (public wiki link above) for additional information, including FAQ's, references and resources, earlier versions, and more.
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Qualitative Studies
- CASP- Qualitative Studies10 questions to help assess qualitative research from the Critical Appraisal Skills Programme
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
Connect with us
