Keywords
As the science of simulation continues to evolve, so does the need for additions and revisions to the INACSL Standards of Best Practice: SimulationSM. Therefore, the INACSL Standards of Best Practice: Simulation are living documents.
Standard
All simulation-based experiences begin with the development of measureable objectives designed to achieve expected outcomes.
Background
Outcomes
Outcomes are an integral component of instructional and research design. Educators, clinicians, and researchers utilize outcome measures to determine the impact of simulation-based experiences. The Kirkpatrick Model is a commonly used ranking model that evaluates training programs and transfer of learning outcomes.
1
This model depicts four sequential levels of evaluation: (a) Reaction—measures participant's satisfaction with training, (b) Learning—measures knowledge, skills, and attitudes (KSAs) gained from training, (c) Behavior—measures changes that occurred as a result of training, and (d) Results—improving quality and safety; increased return on investment following training such as productivity, revenue, and employee retention.Objectives
Once the simulation-based experience outcome measures have been determined, the next step is to develop objectives. Objectives are the guiding tools to facilitate achievement of simulation-based outcomes and the hallmark of sound educational design. Objectives may be broad or specific as a blueprint for simulation design. Bloom's Taxonomy
2
provides a framework for developing and leveling objectives to meet expected outcomes. The taxonomy classifies three domains of learning: cognitive, psychomotor, and affective. Each learning domain has a hierarchical taxonomy applicable to simulation activities. The revised Bloom's Taxonomy3
hierarchy progresses from the lower level objectives, remember and understand to the higher level objectives, apply, analyze, evaluate, and create. These verbs provide structure and communicate the KSAs the participant is intended to achieve as a result of participating in a simulation activity.To have achievable outcomes, clearly defined, measurable objectives are necessary. In the field of corporate management, Doran
4
created the acronym S.M.A.R.T. (specific, measurable, assignable, realistic, and time related) as a framework to develop meaningful, measurable objectives. Organizations have adapted the criteria with differing, yet similar criteria. The S.M.A.R.T framework is used to write objectives that focus on the desired KSAs that simulation participants should demonstrate on completion of simulation-based experiences.The Center for Disease Control
5
provides academia and the health care industry with the following S.M.A.R.T. criteria for writing objectives:- ○Specific: What exactly are we going to do for whom?
- ○Measurable: Is it quantifiable and can we measure it?
- ○Achievable: Can we get it done in the proposed time frame with the resources and support we have available?
- ○Realistic: Will it have an effect on the desired goal or outcome?
- ○Time phased: When will this objective be accomplished?
Potential consequences of not following this standard can lead to ambiguity, unintended outcomes, and failure to meet objectives of the simulation-based experience. This may include skewed assessment and evaluation results; decreased participant satisfaction; failure to achieve desired KSAs; and/or lack of change in quality and safety indicators.
Criteria Necessary to Meet This Standard
- 1.Determine expected outcomes for simulation-based activities and/or programs.
- 2.Construct S.M.A.R.T. objectives based on expected outcomes.
Criterion 1: Determine expected outcomes for simulation-based activities and/or programs.
Required elements:
- •Expected Outcomes are:
- ○Consistent with an organization's, mission, vision, and program outcomes.
- ○Driven by the objectives and concepts within program curricula.6
- ○Represent the multiple cultures and diversity of patients as seen in health care practice.7
- ○Threaded throughout a program or course.
- ○Based on a needs assessment or an area of interest.
- ○Addressed by one or more level of evaluation that may include1:
- ▪Individual and aggregate outcomes.
- ▪Intended KSAs.
- ▪Changes in behavior/performance.
- ▪Return on investment.
- ▪Participant satisfaction.
- ▪
- ○Communicated to participants before the simulation-based experience.
- ○Revised as necessary.
- ○Follow INACSL Standard: Simulation Design.
- ○
Criterion 2: Construct Specific, Measurable, Achievable, Realistic, Time-phased objectives based on expected outcomes.
Required elements:
- •Specific objectives
- ○Identify participants, scenario, fidelity, facilitation, debriefing, assessment, and evaluation methods.
- ○Encompass cognitive (knowledge), affective (attitude), and psychomotor (skills) domains of learning.
- ○Clearly identify the targeted learning domain.
- ○Address multiple domains of learning.
- ○Utilize Bloom's Taxonomy2hierarchical classification of learning domains to level objectives from simple to complex.
- ○Level the objectives based on the participant's KSAs.
- ○Select one action verb for each objective.
- ○Avoid verbs with vague meanings.
- ○Recognize specificity has greater measurability.
- ○Consider “what” will change for “whom” and “how.”
- ▪Identify “what” will be accomplished.
- ▪Determine “who” will be involved.
- ▪Consider “how” the objective will be measured.
- ▪
- ○
- •Measurable objectives
- ○Essential for formative, summative, and high-stakes evaluation (see INACSL Standard: Participant Evaluation).
- ○Establish a baseline as a reference point to quantify change.
- ○Determine evaluation criteria.
- ○Assess the outcome via a method of measurement or an instrument that is reliable, valid, and feasible to obtain.
- ○
- •Achievable objectives
- ○Leveled to participant's knowledge, experience, and skill level.
- ○Feasible within a reasonable time frame.
- ○Resources are available to attain expected outcomes participants.
- ○
- •Realistic objectives
- ○Consistent with an organization's, mission, vision, and program outcomes.
- ○Links the objectives to the expected outcomes.
- ○Appropriate to the KSAs of the participant.
- ○Aligned with current evidence-based practice, guidelines, standards, and literature.
- ○
- •Time-phased objectives
- ○Determine a specific time frame to accomplish the objective (i.e., minutes, hours, days).
- ○Use the specific time frame to plan, implement, and evaluate outcomes.
- ○
About the International Nursing Association for Clinical Simulation and Learning
The International Nursing Association for Clinical Simulation and Learning (INACSL) is the global leader in transforming practice to improve patient safety through excellence in health care simulation. INACSL is a community of practice for simulation where members can network with simulation leaders, educators, researchers, and industry partners. INACSL also provides the INACSL Standards of Best Practice: SimulationSM, an evidence-based framework to guide simulation design, implementation, debriefing, evaluation, and research.
References
- Evaluating training programs: The four levels.Berrett-Koehler Publishers, Inc, San Francisco, CA1994
- Bloom B.S. Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. Longman, New York1956
- Anderson L.W. Krathwohl D.R. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Allyn & Bacon, Boston, MA2001
- There's a S.M.A.R.T. way to write management's goals and objectives.Management Review. 1981; 70: 35-36
- Evaluation briefs: Writing SMART objectives.2009 (Retrieved from)
- Faculty development when initiating simulation programs: Lessons learned from the national simulation study.Journal of Nursing Regulation. 2015; 5: 17-23
- Cultural humility: A concept analysis.Journal of Transcultural Nursing. 2016; 27: 210-217https://doi.org/10.1177/1043659615592677
Article info
Identification
Copyright
© 2016 International Nursing Association for Clinical Simulation and Learning. Published by Elsevier Inc. All rights reserved.
