Review Article| Volume 21, P23-39, August 2018

Do Simulation Studies Measure up? A Simulation Study Quality Review


      Simulation-based education has become a ubiquitous teaching approach in nursing. However, ensuring the quality of simulation research is critical. We reviewed the methodological quality of 26 quantitative studies published in Clinical Simulation in Nursing, 2017. The Medical Education Research Study Quality Instrument and Simulation Research Evaluation Rubric showed that nearly all studies were of moderate to high quality (rated ≥50%). Correlation coefficients showed that interrater agreement was high overall (≥0.94). In conclusion, this was a valid approach for examining simulation study quality. Although most included studies were of high quality, some elements of study reporting can be improved upon.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Clinical Simulation In Nursing
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Adamson K.A.
        • Kardong-Edgren S.
        • Willhaus J.
        An updated review of published simulation evaluation instruments.
        Clinical Simulation in Nursing. 2013; 9: e393-e400
        • Andrea J.
        • Kotowski P.
        Using standardized patients in an undergraduate nursing health assessment class.
        Clinical Simulation in Nursing. 2017; 13: 309-313
        • Beaird G.
        • Nye C.
        • Thacker L.R.
        The use of video recording and standardized patient feedback to improve communication performance in undergraduate nursing students.
        Clinical Simulation in Nursing. 2017; 13: 176-185
        • Boling B.
        • Hardin-Pierce M.
        • Jensen L.
        • Hassan Z.-U.
        Implementing simulation training for new cardiothoracic intensive care unit nurses.
        Clinical Simulation in Nursing. 2017; 13: 33-38.e12
        • Booth R.G.
        • Sinclair B.
        • Strudwick G.
        • Brennan L.
        • Tong J.
        • Relouw H.
        • Vlasic W.
        Identifying error types made by nursing students using eMAR technology.
        Clinical Simulation in Nursing. 2017; 13: 492-500
        • Braga M.
        • Tyler M.D.
        • Rhoads J.M.
        • Cacchio M.P.
        • Auerbach M.
        • Nishisaki A.
        • Larson R.J.
        Effect of just-in-time simulation training on provider performance and patient outcomes for clinical procedures: A systematic review.
        BMJ Simulation and Technology Enhanced Learning. 2015; 1: 94-102
        • Breymier T.L.
        • Rutherford-Hemming T.
        • Horsley T.L.
        • Atz T.
        • Smith L.G.
        • Badowski D.
        • Connor K.
        Substitution of clinical experience with simulation in prelicensure nursing programs: A national survey in the United States.
        Clinical Simulation in Nursing. 2015; 11: 472-478
        • Cant R.P.
        • Cooper S.J.
        The value of simulation-based learning in pre-licensure nurse education: A state-of-the-art review and meta-analysis.
        Nurse Education in Practice. 2017; 27: 45-62
        • Cantrell M.A.
        • Franklin A.
        • Leighton K.
        • Carlson A.
        The evidence in simulation-based learning experiences in nursing education and practice: An umbrella review.
        Clinical Simulation in Nursing. 2017; 13: 634-667
        • Cason M.
        • Atz T.
        • Horton L.F.
        New nursing graduates' self-efficacy ratings and urinary catheterization skills in a high-fidelity simulation scenario.
        Clinical Simulation in Nursing. 2017; 13: 71-77
        • Cheng A.
        • Kessler D.
        • Mackinnon R.
        • Chang T.P.
        • Nadkarni V.M.
        • Hunt E.A.
        • International Network for Simulation-based Pediatric Innovation, Research & Education (INSPIRE) Reporting Guidelines Investigators
        Reporting guidelines for health care simulation research: Extensions to the CONSORT and STROBE statements.
        Advances in Simulation. 2016; 1: 25
        • Cook D.A.
        • Reed D.A.
        Appraising the quality of medical education research methods: The medical education research study quality instrument and the Newcastle–Ottawa scale-education.
        Academic Medicine. 2015; 90: 1067-1076
        • Cooper S.J.
        • Hopmans R.
        • Cant R.P.
        • Bogossian F.
        • Giannis A.
        • King R.
        Deteriorating patients: Global reach and impact of an e-simulation program.
        Clinical Simulation in Nursing. 2017; 13: 562-572
        • IBM Corp
        IBM SPSS Statistics for Windows, Version 23.0.
        IBM Corp, Armonk, NY2016
        • DeCoste-Lopez J.
        • Madhok J.
        • Harman S.
        Curricular innovations for medical students in palliative and end-of-life care: A systematic review and assessment of study quality.
        Journal of Palliative Medicine. 2015; 18: 338-349
        • Doolen J.
        • Mariani B.
        • Atz T.
        • Horsley T.L.
        • Rourke J.O.
        • McAfee K.
        • Cross C.L.
        High-fidelity simulation in undergraduate nursing education: A review of simulation reviews.
        Clinical Simulation in Nursing. 2017; 12: 290-302
        • Fey M.K.
        • Gloe D.
        • Mariani B.
        Assessing the quality of simulation-based research articles: A rating rubric.
        Clinical Simulation in Nursing. 2015; 11: 496-504
        • Grant M.J.
        • Booth A.
        A typology of reviews: An analysis of 14 review types and associated methodologies.
        Health Information & Libraries Journal. 2009; 26: 91-108
        • Gu Y.
        • Zou Z.
        • Chen X.
        The effects of vSIM for Nursing™ as a teaching strategy on fundamentals of nursing education in undergraduates.
        Clinical Simulation in Nursing. 2017; 13: 194-197
        • Haley B.
        • Heo S.
        • Wright P.
        • Barone C.
        • Rettigantid M.R.
        • Anders M.
        Effects of using an advancing care excellence for seniors simulation scenario on nursing student empathy: A randomized controlled trial.
        Clinical Simulation in Nursing. 2017; 13: 511-519
        • Hayden J.
        • Smiley R.A.
        • Alexander M.
        • Kardong-Edgren S.
        • Jeffries P.R.
        The NCSBN national simulation study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
        Journal of Nursing Regulation. 2014; 5: 1-64
        • Hegland P.A.
        • Aarlie H.
        • Strømme H.
        • Jamtvedt G.
        Simulation-based training for nurses: Systematic review and meta-analysis.
        Nurse Education Today. 2017; 54: 6-20
        • Hsieh M.-J.
        • Bhanji F.
        • Chiang W.-C.
        • Yang C.-W.
        • Chien K.-L.
        • Ma M.H.-M.
        Comparing the effect of self-instruction with that of traditional instruction in basic life support courses—a systematic review.
        Resuscitation. 2016; 108: 8-19
        • Ignacio J.
        • Scherpbier A.
        • Dolmans D.
        • Rethans J.-J.
        • Liaw S.Y.
        Mental rehearsal strategy for stress management and performance in simulations.
        Clinical Simulation in Nursing. 2017; 13: 295-302
        • Ilgen J.S.
        • Sherbino J.
        • Cook D.A.
        Technology-enhanced simulation in emergency medicine: A systematic review and meta-analysis.
        Academic Emergency Medicine. 2013; 20: 117-127
        • INACSL
        INACSL Standards of best practice: SimulationSM debriefing.
        Clinical Simulation in Nursing. 2016; 12: S21-S25
        • Kaplan B.
        • Murihead L.
        • Zhang W.
        Leveraging partnerships: Nursing student veteran-centered simulation in situ.
        Clinical Simulation in Nursing. 2017; 13: 258-263
        • Khalaila R.
        Simulation in nursing education: An evaluation of students' outcomes at their first clinical practice combined with simulations.
        Nurse Education Today. 2014; 34: 252-258
        • Kim Y.-J.
        • Noh G.-O.
        • Im Y.-S.
        Effect of step-based prebriefing activities on flow and clinical competency of nursing students in simulation-based education.
        Clinical Simulation in Nursing. 2017; 13: 544-551
        • Kubin L.
        • Wilson C.E.
        Effects of community volunteer children on student pediatric assessment behaviors.
        Clinical Simulation in Nursing. 2017; 13: 303-308
        • Labant A.L.
        • Palmer E.A.
        Teaching neonatal abstinence syndrome using simulation.
        Clinical Simulation in Nursing. 2017; 13: 254-257
        • Lee A.H.
        • Kelley C.
        • Alfes C.M.
        • Bennington L.K.
        • Dolansky M.A.
        High-fidelity patient simulation to evaluate student nurse patient safety competency.
        Clinical Simulation in Nursing. 2017; 13: 628-633
        • Letcher D.C.
        • Roth S.J.
        • Varenhorst L.J.
        Simulation-based learning: Improving knowledge and clinical judgment within the NICU.
        Clinical Simulation in Nursing. 2017; 13: 284-290
        • Levett-Jones T.
        • Lapkin S.
        A systematic review of the effectiveness of simulation debriefing in health professional education.
        Nurse Education Today. 2014; 34: e58-e63
        • Liaw S.Y.
        • Scherpbier A.
        • Rethans J.-J.
        • Klainin-Yobas P.
        Assessment for simulation learning outcomes: A comparison of knowledge and self-reported confidence with observed clinical performance.
        Nurse Education Today. 2012; 32: e35-e39
        • Lioce L.
        • Meakim C.H.
        • Fey M.K.
        • Chmil J.V.
        • Mariani B.
        • Alinier G.
        Standards of best practice: Simulation standard IX: Simulation design.
        Clinical Simulation in Nursing. 2015; 11: 309-315
        • Lubbers J.
        • Rossman C.
        Satisfaction and self-confidence with nursing clinical simulation: Novice learners, medium-fidelity, and community settings.
        Nurse Education Today. 2017; 48: 140-144
        • Luctkar-Flude M.
        • Wilson-Keates B.
        • Tyerman J.
        • Larocque M.
        • Brown C.A.
        Comparing instructor-led versus student-led simulation facilitation methods for novice nursing students.
        Clinical Simulation in Nursing. 2017; 13: 264-269
        • Maharaj T.
        Live-model simulation: Improving nursing students' attitudes and knowledge of Alzheimer's disease.
        Clinical Simulation in Nursing. 2017; 13: 446-451
        • Mariani B.
        • Ross J.G.
        • Paparella S.
        • Allen L.R.
        Medication safety simulation to assess student knowledge and competence.
        Clinical Simulation in Nursing. 2017; 13: 210-216
        • McKinney J.
        • Cook D.A.
        • Wood D.
        • Hatala R.
        Simulation-based training for cardiac auscultation skills: Systematic review and meta-analysis.
        Journal of General Internal Medicine. 2013; 28: 283
        • McWilliams L.A.
        • Malecha A.
        • Langford R.
        • Clutter P.
        Comparisons of cooperative-based versus independent learning while using a Haptic Intravenous Simulator.
        Clinical Simulation in Nursing. 2017; 13: 154-160
        • Motola I.
        • Devine L.
        • Chung H.
        • Sullivan J.
        • Issenberg B.
        Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82.
        Medical Teacher. 2013; 35: e1511-e1530
        • Murphy L.J.
        • Janisse L.
        Optimizing transition to practice through orientation: A quality improvement initiative.
        Clinical Simulation in Nursing. 2017; 13: 583-590
        • National Council of State Boards of Nursing (NCSBN)
        Simulation Use in Prelicensure Nursing Programs.
        NCSBN, Chicago, IL2010
        • Nursing and Midwifery Council
        Standards for Preregistration Nursing Education.
        NMC, London2010
        • Pallant J.
        SPSS Survival Manual; A Step by Step Guide to Data Analysis Using SPSS.
        5th ed. Allen and Unwin, Sydney2013
        • Polit D.
        • Beck C.
        Nursing Research- Generating and Assessing Evidence for Nursing Practice.
        9th ed. Wolters Kluwer, Philadelphia2012
        • Reed D.A.
        • Beckman T.J.
        • Wright S.M.
        An assessment of the methodologic quality of medical education research studies published in the American Journal of Surgery.
        The American Journal of Surgery. 2009; 198: 442-444
        • Reed D.A.
        • Cook D.A.
        • Beckman T.J.
        • Levine R.B.
        • Kern D.E.
        • Wright S.M.
        Association between funding and quality of published medical education research.
        JAMA. 2007; 298: 1002-1009
        • Ross J.G.
        • Carney H.
        The effect of formative capstone simulation scenarios on novice nursing students' anxiety and self-confidence related to initial clinical practicum.
        Clinical Simulation in Nursing. 2017; 13: 116-120
        • Rutherford-Hemming T.
        • Alfes C.M.
        The use of hospital-based simulation in nursing education—a systematic review.
        Clinical Simulation in Nursing. 2017; 13: 78-89
        • Sapyta Y.Y.
        • Eiger C.
        Improving pediatric nurses' knowledge, accuracy, and confidence through code documentation simulation.
        Clinical Simulation in Nursing. 2017; 13: 278-283
        • Skinner H.M.
        Simulation: Preparing nursing students to work with community-dwelling older adults.
        Clinical Simulation in Nursing. 2017; 13: 520-523
        • Turkelson C.
        • Keiser M.
        Using checklists and repetitive simulation to improve patient safety: A pilot project with the Impella® Left Ventricular assist Device.
        Clinical Simulation in Nursing. 2017; 13: 53-63
        • Verkuyl M.
        • Romaniuk D.
        • Atack L.
        • Mastrilli P.
        Virtual Gaming simulation for nursing education: An experiment.
        Clinical Simulation in Nursing. 2017; 13: 238-244
        • Walters B.
        • Potetz J.
        • Fedesco H.N.
        Simulations in the classroom: An innovative active learning experience.
        Clinical Simulation in Nursing. 2017; 13: 609-615