Systematic Reviews- Legacy Guide

Why is this information important?

• Researchers in health science fields are increasingly recognizing the value of including qualitative studies in systematic reviews.
• Because qualitative and quantitative studies can be so different, however, it can be hard to know how to integrate them productively.

On this page you will find the following helpful resources:

• Articles and chapters that discuss methods for integrating qualitative studies into systematic reviews
• Examples of systematic reviews that include qualitative studies
• Selected books on qualitative studies and systematic reviews that are owned by UNC Libraries
• A short list of helpful websites and tutorials on the topic

See also:  Assessing Qualitative Research

The following articles and chapters offer some advice on how to include qualitative research in systematic reviews, as well as some examples of reviews that have done so.

Methods for Including Qualitative Research in Systematic Reviews

Butler, Ashleigh, Helen Hall, Beverley Copnell. (2016). A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care. Worldviews on Evidence-Based Nursing 13(3): 241-249.

Cochrane Review Handbook, Chapter 20: Qualitative Research and Cochrane reviews. 2011.

Cochrane Handbook for Systematic Reviews of Interventions Chapter QQ v1.02a June 14, 2013.Qualitative and Implementation Evidence and Cochrane Reviews. Authors: Jane Noyes, Karin Hannes, Andrew Booth, Janet Harris, Angela Harden, Jennie Popay, Alan Pearson, Margaret Cargo, and Tomas Pantoja on behalf of the Cochrane Qualitative and Implementation Methods Group.

See Journal of Clinical Epidemiology, Volume 97, May 2018, Cochrane Qualitative and Implementation Methods Group Guidance Series. UNC Chapel Hill users direct link.

• This supplemental guidance is intended to be used in conjunction with the current edition of the Cochrane Handbook.  
• Note:  the Cochrane Handbook for Systematic Reviews of Interventions, Version 6, has been completed. A second edition of the book version is now in press and will be published by Wiley in October 2019. All the chapters contained in the printed version will be freely available online in browsable form; the online version will provide additional chapters and supplementary material mostly specific to the Cochrane context. The chapters from the Version 6 book are all now available as PDF chapters but only to registered Cochrane contributors (Archie login required).The printed copy and publicly accessible web version will be available in October 2019.

  ---

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B. How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research 2006; 6: 27-44.

Dixon-Woods, Mary, Shona Argawal, David Jones, Bridget Young, and Alex Sutton. Synthesizing qualitative and quantitative evidence: a review of possible methods.  J Health Serv Res Policy January 1, 2005 vol. 10 no. 1 45-53B

Harden, Angela. Mixed-Methods Systematic Reviews: Integrating Quantitative and Qualitative Findings. FOCUS 25, 2010.

Joanna Briggs Institute Reviewer's Manual Chapter 2 (Systematic Reviews of Qualitative Evidence): Lockwood C, Porrit K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer's Manual, 4th edition. The Joanna Briggs Institute, 2017. Available from https://reviewersmanual.joannabriggs.org/  

Ludvigsen, Mette S., Elizabeth O.C. Hall, Gabriele Meyer, Liv Fegran, Hanne Aagaard, Lisbeth Uhrenfeldt. (2015) Using Sandelowski and Barroso’s Meta-Synthesis Method in Advancing Qualitative Evidence. Qualitative Health Research 26(3).PMID:25794523 DOI:10.1177/1049732315576493

• UNC Chapel Hill users:  link to article in SAGE Journals Qualitative Health.  

Examples of Systematic Reviews Incorporating Qualitative Research

Finfgeld-Connett, Deborah. Intimate partner abuse among older women: qualitative systematic review. Clinical Nursing Research 2014, 23:6 664-683.

Lucas, Patricia, Janis Baird, Lisa Arai, Catherine Law, and Helen M. Roberts. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Medical Research Methodology 2007, 7:4  doi:10.1186/1471-2288-7-4

Added Value of Qualitative Research with Randomized Clinical Trials

O'Cathain, Alicia, Jackie Goode, Sarah J. Drabble, Kate J. Thomas, Anne Rudolph, and Jenny Hewison.  Getting added value from using qualitative research wtih randomized controlled trials:  a qualitative interview study.  Trials 2014, 15 (June): 215.  doi:10.1186/1745-6215-15-215 

Snowden, Claire (Trials) and David Gough (Systematic Reviews) (eds) Qualitative Methods, Trials, and Systematic Reviews. Joint Publication, Trials and Systematic Reviews. 

• Snowden, Claire (2015). Trials editorial about the special joint publication: Qualitative and mixed methods research in trials. Trials 16: 558. 
• Gough, David (2015).  Systematic Reviews editorial about the special joint publication: Qualitative and mixed methods in systematic reviews. Systemaatic Reviews 4:81.
• The two sister-journals, Trials and Systematic Reviews, have, on the face of it, different readerships and deal with different issues. In both journals there is, however, a common and growing interest in the contribution of qualitative methods. We are seeing an expansion of the use and application of a range of techniques with entry into novel research areas and pursuit of new lines of inquiry. Our contributors are working within specific methods, with mixed methods, and across paradigms.  This special issue covers these innovative and challenging areas, with the aim of sharing methodological practice, findings and reflections to drive forward and further the respective fields.

Systematic Review of Qualitative Research (Meta-Synthesis)

Korhonen, Anne, Tuovi Hakulinen-Viitanen, Virpi Jylha, Arja Holopainen.  Meta-synthesis and evidence-based heath care:  a method for systemic review.  Scandinavian Journal of Caring Sciences 2013, 27:4, 1027-1034.  doi: 10.1111/scs.12003

GRADE-CERQUAL Approach

PLOS Medicine Staff. (2016). Correction: Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual). PLoS Medicine, 13(6), e1002065. doi:10.1371/journal.pmed.1002065 PMID:27284910 PMCID: PMC4902189

• Original, uncorrected version:  Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C. J., Gülmezoglu, M., … Rashidian, A. (2015). Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Medicine, 12(10), e1001895. doi:10.1371/journal.pmed.1001895 PMID: 26506244 PMCID: PMC4624425

See additional information in Website/Tutorial box, below. 

Cochrane Methods:  Qualitative & Implementation - Core Library of Qualitative Synthesis Methodology

• Core Library.  NB: items represent key methodology resources. No endorsement of individual methods is implied by inclusion in this list.  See Supplemental Handbook Guidance.   

Meta-ethnography is generally considered an interpretative (vs. aggrevative) qualitative synthesis approach.  

eMERGe Project

Funded by the National Institute for Health Research of the National Health Service (NHS) in the UK,  The aim of the eMERGe project, which ran from June 2015 to May 2017, was to develop a guideline to improve the way researchers report meta-ethnographies. The website includes many resources and publications.  From the website: 

• Meta-ethnography is an interpretive qualitative synthesis approach developed by George W. Noblit and R. Dwight Hare, in the field of education, in the 1980s. They designed the approach to address the inability of an aggregative synthesis of five ethnographic studies to explain the failure of racial desegregation in schools. In a meta-ethnography, the reviewers conducting the meta-ethnography aim to produce new interpretations that transcend the findings of individual studies, rather than simply to aggregate findings. Noblit and Hare described it as ‘making a whole into something more than the parts alone imply’ (Noblit & Hare, 1988, p. 28), i.e. going beyond the findings of any individual study.

Meta-ethnography differs from other qualitative evidence synthesis approaches in its underpinning theory, use of the authors’ interpretations (e.g. concepts, themes) from primary qualitative studies as data, and creation of new interpretations through its unique analytic synthesis process. Researchers select, analyse and interpret qualitative studies to answer focused questions on a specific topic (e.g. people’s experiences of having and being treated for arthritis) to come up with new insights and conclusions. The aim of the eMERGe project was to develop a guideline to improve the way researchers report meta-ethnographies.

Note from the Core Library: The following items represent key methodology resources. No endorsement of individual methods is implied by inclusion in this list.  See Supplemental Handbook Guidance.   

• Campbell R, Pound P, Morgan M, Daker-White G, Britten N, Pill R, Yardley L, Pope C, Donovan J. Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess. 2011 Dec;15(43):1-164.
• France, E. F., Ring, N., Thomas, R., Noyes, J., Maxwell, M., & Jepson, R. (2014). A methodological systematic review of what's wrong with meta-ethnography reporting. BMC medical research methodology, 14(1), 119.
• France, E. F., Wells, M., Lang, H., & Williams, B. (2016). Why, when and how to update a meta-ethnography qualitative synthesis. Systematic Reviews, 5(1). doi:10.1186/s13643-016-0218-4
• Toye, F., Seers, K., Allcock, N., Briggs, M., Carr, E., Andrews, J., & Barker, K. (2013). 'Trying to pin down jelly'-exploring intuitive processes in quality assessment for meta-ethnography. BMC Medical Research Methodology, 13(1), 46.

Some additional meta-ethnography resources and examples:

• SAGE Research Methods (Qualitative) contains many resources for meta-ethnography.  "Meta-ethnography" can be searched from the general Sage Methods page. 
• To see some examples of published meta-ethnographies in Scopus (which indexes both social science and health-related literature), you can run a quick and dirty search such as "public health AND meta-ethnography" or "public health AND (policy OR management) and meta-ethnography)" .  

The Books section of this page contains a small selection of the books available on including qualitative research in systematic reviews. To find more, click the links below to search the UNC library catalog.

• Catalog Search--Review Literature as Topic
• Catalog Search--Systematic Reviews(Medical Research)
• Catalog Search--Meta-Analysis as Topic

Cochrane Qualitative and Implementation Methods Group

Joanna Briggs Institute Online Reviewer's Manual Chapter 2 (Systematic Reviews of Qualitative Evidence): Lockwood C, Porrit K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer's Manual, 4th edition. The Joanna Briggs Institute, 2017. Available from https://reviewersmanual.joannabriggs.org/  

Canadian Cochrane Center YouTube Tutorial on Qualitative Evidence Synthesis, published Dec 2, 2013

This recording is of the first webinar of the 2013-2014 Different Evidence, Different Syntheses Series. Jane Noyes of the Cochrane Qualitative and Implementation Methods Group for the discussion was the presenter for the webinar held on November 28, 2013.This webinar explored:
a) when to consider undertaking a synthesis of qualitative evidence;
b) some frequently used methods and examples of developing methods for synthesising qualitative evidence; and
c) approaches for integrating qualitative and quantitative findings.

Seminar on CerQual: a new approach to qualitative evidence synthesis analysis Oct 13, 2014

Ready to start a systematic review? HSL Librarians can help!

Fill out the Systematic Review Request Form and the best-suited librarian will get back to you promptly.