Highlights
- •Simulation evaluation frameworks are discussed in relation to the management of deteriorating patients.
- •The impact of face to face and virtual simulation modalities are described.
- •The impact of simulation programs on clinical practice and patient safety should be considered.
Abstract
Background
Measuring the impact of patient safety interventions is challenging. This article
aims to illustrate a “gold standard” model of program evaluation incorporating examples
from two patient deterioration programs.
Methods
Australian nurses were trained in primary responses to emergencies in four hospitals
using either face-to-face (F2F) or screen-based simulation versions of a simulation
program. Evaluation outcomes were measured using Kirkpatrick's evaluation hierarchy
covering participant ‘reaction’, ‘learning’, ‘behaviour’ change and ‘results’—based
on 1,564 pre–post intervention vital signs chart reviews.
Results
Seventy-four nurses participated. Reaction—participant confidence/competence ratings
and Learning improved significantly in both modalities (p < .001). Behaviour—oxygen delivery systems were used more appropriately after training
in the F2F group (p = .037). Applicable recording of oxygen saturation (SpO2) improved significantly (p ≤ .008) in both modalities. Results—at least a two-fold increase in the overall initiation
of a clinical review after training, in both modalities (p < .001; effect: d = 0.41 F2F and d = 0.35 screen-based simulation).
Conclusions
Kirkpatrick's evaluation model enables a suitable template for gold standard education
evaluations.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Clinical Simulation In NursingAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- NSQHS Standards.2nd ed. 2017 (Retrieved from)
- A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence.Evaluation and Program Plannin. 2004; 27: 341-347
- Undergraduate nursing students’ performance in recognising and responding to sudden patient deterioration in high fidelity simulated environments: Quantitative results from an Australian multi-centre study.Nurse Education Today. 2013; 34: 691-696https://doi.org/10.1016/j.nedt.2013.09.015
- A trial of e-simulation of sudden patient deterioration (FIRST2ACTWEB™) on student learning.Nurse Education Today. 2015; 35: 36-42
- The benefits of debriefing as formative feedback in nurse education.Australian Journal of Advanced Nursing. 2011; 29: 37-47
- FIRST2 ACT: Educating nurses to identify patient deterioration – a theory-based model for best practice simulation education.Nurse Education Today. 2011; 31: 687-693https://doi.org/10.1016/j.nedt.2011.03.006
- Simulation-based learning in nurse education: Systematic review.Journal of Advanced Nursing. 2009; 66: 3-15
- The evidence in simulation-based learning experiences in nursing education and practice: An umbrella review.Clinical Simulation In Nursing. 2017; 13: 634-667
- The fundamental standards.2017 (Retrieved from)
- The educational impact of web-based and face-to-face patient deterioration simulation programs: An interventional trial.Nurse Education Today. 2018; 64: 94-98https://doi.org/10.1016/j.nedt.2018.01.037
- The effectiveness of education in the recognition and management of deteriorating patients: A systematic review.Nurse Education Today. 2016; 44: 133-145https://doi.org/10.1016/j.nedt.2016.1006.1
- Technology-enhanced simulation for health professions education a systematic review and meta-analysis.JAMA. 2011; 306: 978-988
- Doing the right thing at the right time: Assessing responses to patient deterioration in electronic simulation scenarios using Course-of-Action analysis.Computers, Informatics, Nursing. 2015; 33: 199-207
- Patient deterioration education: Evaluation of face-to-face simulation and e-simulation approaches.Clinical Simulation in Nursing. 2015; 11: 97-105https://doi.org/10.1016/j.ecns.2014.10.010
- Simulation versus lecture? Measuring educational impact: Considerations for best practice.Evidence-Based Nursing. 2016; 19: 55
- Deteriorating patients: Global reach and impact of an E-simulation program.Clinical Simulation in Nursing. 2017; 3: 562-572
- The impact of web-based and face-to-face simulation on patient deterioration and patient safety: Protocol for a multi-site multi-method design.BMC Health Services Research. 2016; 16: 475https://doi.org/10.1186/s12913-016-1683-0
- The impact of face-to-face and web-based simulation on patient deterioration and patient safety. [final report].(Retrieved from)
- FIRST2ACT.2019 (Retrieved from)
- Training for shoulder dystocia: A trial of simulation using low-fidelity and high-fidelity mannequins.Obstetrics & Gynecology. 2006; 108: 1477-1485
- Improving neonatal outcome through practical shoulder dystocia training.Obstetrics & Gynecology. 2008; 112: 14-20
- Simulation training in obstetrics.Clinical Obstetrics and Gynecology. 2017; 60: 802-810
- Evaluating training programs: Development and correlates of the questionnaire for professional training evaluation.International Journal of Training and Development. 2013; 17: 2
- What is a case study?.Evidence-Based Nursing. 2018; 21 (Retrieved from): 7-8
- Simulation-based training for nurses: Systematic review and meta-analysis.Nurse Education Today. 2017; 54: 6-20
- IBM SPSS Statistics for Windows, Version 24.0.IBM Corp, Armonk, NY2016
- Publications.(Retrieved from)
- Control and validity in medical education research.Medical Education. 2001; 35: 920-921
- Striving for higher levels of evaluation in simulation.Clinical Simulation in Nursing. 2010; 6: e203-e204
- Seven keys to unlock the four levels of evaluation.Performance Improvement. 2006; 45: 7
- Evaluating Training Programs.Tata McGraw-Hill Education, Chicago1975
- Learning styles and learning spaces: Enhancing experiential learning in higher education.Academy of Management Learning & Education. 2005; 4: 193-212
- Courses and Learning: vSim for Nursing.2019 (Retrieved from)
- The impact of a web-based educational program on the recognition and management of deteriorating patients.Journal of Clinical Nursing. 2017; 26: 4848-4856
- Effectiveness of a web-based simulation in improving nurses’ workplace practice with deteriorating ward patients: A pre-and post-intervention study.Journal of Medical Internet Research,. 2016; 18: e37
- A critical review of simulation-based mastery learning with translational outcomes.Medical Education. 2014; 48: 375-385
- What is the impact of multidisciplinary team simulation training on team performance and efficiency of patient care? An integrative review.Australasian Emergency Nursing Journal. 2016; 19: 44-53
- Using Research in Healthcare Practice.1st ed. Lippincott Williams & Wilkins Pty Ltd, New South Wales, Australia2010
- Standard instruction versus simulation: Educating registered nurses in the early recognition of patient deterioration in paediatric critical care.Nurse Education Today. 2015; 36: 287-292
- Operating room team training with simulation: A systematic review.Journal of Laparoendoscopic & Advanced Surgical Techniques. 2017; 27: 475-480
- An integrative review: Instructional strategies to improve nurses' retention of cardiopulmonary resuscitation priorities.International Journal of Nursing Education Scholarship. 2015; 12: 37-43
- Norwegian nursing students’ evaluation of vSim for Nursing.Advances in Simulation. 2018; 3: 10
- Developing a national patient safety education framework for Australia.BMJ Quality & Safety. 2006; 15: 437-442
- Training evaluation: Knowing more than is practiced.Advances in Developing Human Resources. 2006; 8: 528-539
- Kirkpatrick’s levels and education ‘evidence’.Medical Education. 2012; 46: 97-106
Article info
Publication history
Published online: June 30, 2020
Identification
Copyright
© 2020 International Nursing Association for Clinical Simulation and Learning. Published by Elsevier Inc. All rights reserved.
