Advertisement

Healthcare Simulation Standards of Best PracticeTM Evaluation of Learning and Performance

      Abstract

      KEYWORDS

      As the science of simulation continues to evolve, so does the need for additions and revisions to the Healthcare Simulation Standards of Best Practice. Therefore, the Healthcare Simulation Standards of Best PracticeTM are living documents.

      Standard

      Simulation-based experiences may include evaluation of the learner.

      Background

      Simulation-based experiences (SBEs) support evaluation of the learner's knowledge, skills, attitudes, and behaviors demonstrated in the cognitive, psychomotor, and/or affective domains of learning.
      • Alexander M.
      • Durham C.
      • Hooper J.
      • Jeffries P.
      • Goldman N.
      • Kardong-Edgren S.
      • Tillman C.
      NCSBN simulation guidelines for prelicensure nursing programs.
      Formative evaluation of the learner is meant to foster development and assist in progression toward achieving objectives or outcomes. Summative evaluation focuses on the measurement of outcomes or achievement of objectives at a discrete moment in time, for example, at the end of a program of study.
      • Billings D.
      • Halstead J.
      Teaching in nursing: A guide for faculty.
      High-stakes evaluation refers to an assessment that has major implications or consequences based on the result or the outcome, such as merit pay, progression or grades. Research has identified learning benefits to the observer as a learner in the simulation experience.
      • Johnson B.K.
      Simulation observers learn the same as participants: The evidence.
      If the learner is in an observer role in the SBE, the facilitator may consider evaluating the observer.
      • O'Regan S.
      • Molloy E.
      • Watterson L.
      • Nestel D.
      Observer roles that optimize learning in healthcare simulation education: A systematic review.
      ,
      • Johnson B.K.
      Simulation observers learn the same as participants: The evidence.
      Evaluation of the learner(s) using SBEs includes the following elements:
      • (a)
        determine the type of evaluation for the SBE
      • (b)
        design the SBE to include timing of the evaluation
      • (c)
        use of a valid and reliable evaluation tool
      • (d)
        evaluator training
      • (e)
        completion of the evaluation, interpretation of the results and provision of feedback to the learner(s).
        • Huang Y.
        • Rice J.
        • Spain A.
        • Palaganas J.
        Terms of reference.
      Potential consequences of not following this standard are learner dissatisfaction with the SBE, failure to achieve learning outcomes, inaccurate evaluation, and assessment bias.

      Criteria Necessary to Meet This Standard

      • 1
        Determine the method of learner evaluation before the SBE.
      • 2
        SBEs may be selected for formative evaluation.
      • 3
        SBEs may be selected for summative evaluation.
      • 4
        SBEs may be selected for high-stakes evaluation.
      Criterion 1: Determine the method of learner evaluation before the SBE.
      Required Elements:
      Learner evaluation is:
      • directed by the objectives, outcome and/or the level of the learner.
      • guided by the type: formative, summative, or high- stakes evaluation.
      Criterion 2: SBEs may be selected for formative evaluation.
      Required Elements:
      Formative evaluation is conducted:
      • with the purpose of:
        • facilitating teaching and learning.
        • identifying and closing gaps in knowledge, skills, and attitudes.
        • monitoring progress toward achieving outcomes.
        • developing learner's clinical competencies.
        • providing ongoing formative feedback.
          • Adamson K.
          Evaluating simulation effectiveness.
          ,
          • Adamson K.
          Evaluation tools and metrics for simulations.
          assessing readiness for entry to the clinical setting.
      • after adequate evaluation training for facilitators, evaluators and standardized patients. (Follow the Healthcare Simulation Standards of Best PracticeTM (HSSOBPTM) Facilitation).
      • using the most appropriate evaluation tool.
        • Adamson K.
        Evaluation tools and metrics for simulations.
        ,
        • Houston D.
        • Thompson J.
        Blending formative and summative assessment in a Capstone subject: ‘It's not your tools, it's how you use them.
      • using an appropriate group ratio to optimize learning. The ideal ratio of facilitator to learner will vary based on each SBE.
        • Alexander M.
        • Durham C.
        • Hooper J.
        • Jeffries P.
        • Goldman N.
        • Kardong-Edgren S.
        • Tillman C.
        NCSBN simulation guidelines for prelicensure nursing programs.
        ,
        • Billings D.
        • Halstead J.
        Teaching in nursing: A guide for faculty.
        ,

        Arizona State Board of Nursing. (2016). Advisory opinion; Education use of simulation in approved RN/LPN programs. Retrieved from https://azbn.gov/sites/default/files/2020-04/Simulation%20in%20Approved%20RN-LPN%20Programs-AO%2011-2019.pdf

        ,
        • Levett-Jones T.
        • Anderson P.
        • Reid-Searl K.
        • Guinea S.
        • McAllister M.
        • Lapkin S.
        • Palmer L.
        • Niddrie M.
        Tag team simulation: An innovative approach for promoting active engagement of learners and observers during group simulations.
        ,
        • Guinea S.
        • Andersen P.
        • Reid-Searl K.
        • Levett-Jones T.
        • Dwyer T.
        • Heaton L.
        • Flenady T.
        • Applegarth J.
        • Bickell P.
        Simulation-based learning for patient safety: The development of the Tag Team Patient Safety Simulation methodology for nursing education.
      Criterion 3: Simulation-based experiences may be selected for summative evaluation.
      Required Elements:
      Summative evaluation is conducted:
      • after learners are informed of the evaluation process.
        • Alexander M.
        • Durham C.
        • Hooper J.
        • Jeffries P.
        • Goldman N.
        • Kardong-Edgren S.
        • Tillman C.
        NCSBN simulation guidelines for prelicensure nursing programs.
      • to evaluate learning, skill acquisition, and academic achievement at the.
      Criterion 4: Simulation-based experiences may be selected for high-stakes evaluation.
      Required Elements:
      • High-stakes evaluation is conducted:
      • to determine competence, gaps in knowledge, skills, behaviors and/or to identify safety issues.
      • based on specific learner objectives.
      • after the potential implications have been explained to the learners.
      • with predetermined learner actions that would result in the conclusion of the SBE.
      • after the SBE has been pilot tested.
      • by formally trained evaluator(s).
      • after the learner has had the opportunity for multiple exposures to various SBEs including those with summative evaluations.
        • Rizzolo M.
        Developing and using simulation for high-stakes assessment.
        ,
        • Boulet J.
        • Murray D.
        Simulation-based assessment in anesthesiology: Requirements for practical application.
      • use an evaluation instrument previously tested with similar and/or comparable populations.
      • if using an observation-based instrument, consider using more than one rater or evaluator for each learner, either directly observed or through a video recording.
        • Ravert P.
        Curriculum integration of clinical simulation.

      Original INACSL Standard

      The INACSL Board of Directors. (2011). Standard VII: Evaluation of expected outcomes. Clinical Simulation in Nursing, 7, S18-S19.

      Subsequent INACSL Standard

      Sando, C., Coggins, R., Meakim, C., Franklin, A., Gloe, D., Boese, T., & Borum, J. (2013). Standards of best practice: Simulation standard VII: Participant assessment and evaluation. Clinical Simulation in Nursing, 9(6S), S30-S32. http://dx.doi.org/10/1016/j.ecns.2013.04.007.

      INACSL Standards Committee (2016, December). INACSL standards of best practice

      SimulationSM Participant evaluation. Clinical Simulation in Nursing, 12(S), S26-S29. http://dx.doi.org/10.1016/ j.ecns.2016.09.009.

      About the International Nursing Association for Clinical Simulation and Learning (INACSL)

      The International Nursing Association for Clinical Simulation and Learning (INACSL) is the global leader in transforming practice to improve patient safety through excellence in health care simulation. INACSL is a community of practice for simulation where members can network with simulation leaders, educators, researchers, and industry partners. INACSL also provided the original living documents INACSL Standards of Best Practice: SimulationSM, an evidence-based framework to guide simulation design, implementation, debriefing, evaluation, and research. The Healthcare Simulation Standards of Best PracticeTM are provided with the support and input of the international community and sponsored by INACSL.

      References

        • Alexander M.
        • Durham C.
        • Hooper J.
        • Jeffries P.
        • Goldman N.
        • Kardong-Edgren S.
        • Tillman C.
        NCSBN simulation guidelines for prelicensure nursing programs.
        Journal of Nursing Regulation. 2015; 6: 39-42
        • Billings D.
        • Halstead J.
        Teaching in nursing: A guide for faculty.
        6th ed. Elsevier, 2019
        • O'Regan S.
        • Molloy E.
        • Watterson L.
        • Nestel D.
        Observer roles that optimize learning in healthcare simulation education: A systematic review.
        Advances in Simulation. 2016; 1 (.doi)https://doi.org/10.1186/s41077-015-0004-8
        • Johnson B.K.
        Simulation observers learn the same as participants: The evidence.
        Clinical Simulation in Nursing. 2019; 33: 26-34https://doi.org/10.1016/j.ecns.2019.04.006
        • Huang Y.
        • Rice J.
        • Spain A.
        • Palaganas J.
        Terms of reference.
        in: Palaganas J. Maxworthy J. Epps C. Mancini M. Defining excellence in simulation programs. Wolters Kluwer, 2015: xxi-xxxiii
        • Adamson K.
        Evaluating simulation effectiveness.
        in: Ulrich B. Mancini B. Mastering simulation: A handbook for success. 2014: 145-163
        • Adamson K.
        Evaluation tools and metrics for simulations.
        in: Jeffries P. Clinical simulations in nursing education: Advanced concepts, trends, and opportunities. National League for Nursing, Wolters Kluwer Health, 2014: 145-163
        • Houston D.
        • Thompson J.
        Blending formative and summative assessment in a Capstone subject: ‘It's not your tools, it's how you use them.
        Journal of University Teaching & Learning Practice. 2017; 14
      1. Arizona State Board of Nursing. (2016). Advisory opinion; Education use of simulation in approved RN/LPN programs. Retrieved from https://azbn.gov/sites/default/files/2020-04/Simulation%20in%20Approved%20RN-LPN%20Programs-AO%2011-2019.pdf

        • Van Der Vleuten C.P.M.
        • Schuwirth L.W.T.
        • Driessen E.W.
        • Govaerts M.J.B.
        • Heeneman S.
        Twelve tips for programmatic assessment.
        Medical Teacher. 2015; 37: 641-646https://doi.org/10.3109/0142159X.2014.973388
        • Shaughnessy S.M.
        • Joyce P.
        Summative and formative assessment in medicine: The experience of an anesthesia trainee.
        International Journal of Higher Education. 2015; 4: 198-206
        • Eva K.W.
        • Bordage G.
        • Campbell C.
        • Gallbraith R.
        • Ginsburg S.
        • Holmboe E.
        • Regehr G.
        Towards a program of assessment for health care professionals: From training into practice.
        Advances in Health Sciences Education. 2016; 21: 897-913
        • Sook Jung K.
        • Hae Young M.
        Psychological safety in nursing simulation.
        Nurse Educator. 2019; 44: E6-E9https://doi.org/10.1097/NNE.0000000000000571
        • Oermann M.H.
        Using simulation for summative evaluation in nursing.
        Nurse Educator. 2016; 41: 133https://doi.org/10.1097/NNE.0000000000000266
        • Rizzolo M.
        Developing and using simulation for high-stakes assessment.
        in: Jeffries P. Clinical simulations in nursing education: Advanced concepts, trends, and opportunities. Wolter Kluwer Health, 2014: 113-121
        • Boulet J.
        • Murray D.
        Simulation-based assessment in anesthesiology: Requirements for practical application.
        Anesthesiology. 2010; 112: 1041-1052
        • Ravert P.
        Curriculum integration of clinical simulation.
        in: Jeffries P. Simulation in nursing education: From conceptualization to evaluation. 2nd ed. National League for Nursing, 2012: 77-90
        • Levett-Jones T.
        • Anderson P.
        • Reid-Searl K.
        • Guinea S.
        • McAllister M.
        • Lapkin S.
        • Palmer L.
        • Niddrie M.
        Tag team simulation: An innovative approach for promoting active engagement of learners and observers during group simulations.
        Nurse Education in Practice. 2015; 15: 345-352
        • Guinea S.
        • Andersen P.
        • Reid-Searl K.
        • Levett-Jones T.
        • Dwyer T.
        • Heaton L.
        • Flenady T.
        • Applegarth J.
        • Bickell P.
        Simulation-based learning for patient safety: The development of the Tag Team Patient Safety Simulation methodology for nursing education.
        Collegian. 2019; 26: 392-398