Systematic Review Automation: MLA Bibliometric Analysis
Created by Health Science Librarians
Abstract
Automating the Systematic Review Process: A Bibliometric Analysis
Objectives : To perform a bibliometric analysis on the body of literature discussing the use of automation tools to facilitate systematic reviews.
Methods : A comprehensive search was performed to identify articles describing the automation of systematic reviews using artificial intelligence (e.g. machine learning, text mining, algorithms, etc.). Searches were conducted in biomedical, library & information science, and engineering databases. Results were aggregated and de-duplicated. The number of results indicated that machine learning was an appropriate technique for reviewing such a large body of literature. A random sample of records was screened to identify relevant papers and create a training data set. After the machine learning process was employed to complete screening on the remaining records, final results were uploaded into a visualization & network mapping tool.
Results : Our bibliometric analysis found considerable collaboration around a core of prolific authors based in countries where evidence-based medicine has been widely adopted. Over time, representation in publications has moved from computer science journals and conference proceedings to domain-specific application with many newer enhancements focused on health sciences. Additionally, publications have moved from theory to problem-solving in specific parts of the systematic review process. Results will be delivered as a poster at the MLA 2019 Annual Meeting with visualizations including: size and scope of literature, key journals and conferences, collaboration networks, and author demographics including affiliation and role.
Conclusions : The field of systematic review automation has grown substantially in recent years, both in the addition of new authors and the expansion of collaboration networks. While adoption has increased in systematic review-focused publications and groups, more work must be done to gain acceptance in biomedical publications, including validations of algorithms and tools, guidelines for use in systematic reviews, and education of researchers. As key partners in systematic review performance and educators in systematic review methods, librarians are poised to play an important role in the adoption and acceptance of this technology.
Top Cited Articles
- Aphinyanaphongs, Y., Tsamardinos, I., Statnikov, A., Hardin, D., & Aliferis, C. F. (2005). Text categorization models for high-quality article retrieval in internal medicine. Journal of the American Medical Informatics Association, 12(2), 207-216. doi:10.1197/jamia.M1641
- Bastian, H., Glasziou, P., & Chalmers, I. (2010). Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Medicine, 7(9), e1000326.
- Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P. Y. (2006). Reducing workload in systematic review preparation using automated citation classification. Journal of the American Medical Informatics Association, 13, 206-219. doi:10.1197/jamia.M1929 10.1197/jamia.M1929. Epub 2005 Dec 15.']
- McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. J Med Libr Assoc, 93(1), 74-80.
- Miwa, M., Thomas, J., O'Mara-Eves, A., & Ananiadou, S. (2014). Reducing systematic review workload through certainty-based screening. Journal of Biomedical Informatics, 51, 242-253. doi:10.1016/j.jbi.2014.06.005
- O'Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic reviews, 4, 5. doi:10.1186/2046-4053-4-5 10.1186/2046-4053-4-5.']
- Staples, M., & Niazi, M. (2007). Experiences using systematic review guidelines. Journal of Systems and Software, 80(9), 1425-1437. doi:https://doi.org/10.1016/j.jss.2006.09.046
- Thomas, J., McNaught, J., & Ananiadou, S. (2011). Applications of text mining within systematic reviews. Research Synthesis Methods, 2(1), 1-14.
- Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014). Systematic review automation technologies. Systematic reviews, 3, 74. doi:10.1186/2046-4053-3-74
- Wallace, B. C., Trikalinos, T. A., Lau, J., Brodley, C., & Schmid, C. H. (2010). Semi-automated screening of biomedical citations for systematic reviews. BMC bioinformatics, 11(1), 55.
Top Sources for New SR Automation Articles
- Systematic Reviews
- Journal of Clinical Epidemiology
- Journal of Biomedical Informatics
- ACM International Conference Proceedings Series
- AMIA Symposium Proceedings
- Research Synthesis Methods
- Journal of the American Medical Informatics Association
- Lecture Notes in Computer Science
- PLOS ONE
- BMC Bioinformatics
Tools Mentioned Frequently in Literature
Many of these tools are included in the Systematic Review Toolbox at systematicreviewtools.com
You can search by part of review or AI approach and see a list of tools and accompanying articles.
- Abstrackr
- Cadima
- Colander
- DistillerSR
- EPPI-Reviewer
- Litsearchr
- RobotAnalyst
- RobotReviewer
- Swift-Review
Download the Poster
This data is part of a poster that was presented at the 2019 Medical Library Association annual meeting. The poster is downloadable below.
