Advertisement

Virtual Simulation for Last-Year Nursing Graduate Students in Times of Covid-19: A Quasi-Experimental Study

  • I Zaragoza-García
    Affiliations
    Department of Nursing. Faculty of Nursing, Physiotherapy and Podology. University Complutense of Madrid, Madrid, 28041 Spain

    Instituto de Investigación Sanitaria Hospital 12 de Octubre (imas12). Centro de Actividades Ambulatorias, 6ª Planta Bloque D Avda. de Córdoba, s/n 28041 Madrid, Spain.
    Search for articles by this author
  • I Ortuño-Soriano
    Correspondence
    Corresponding author:
    Affiliations
    Department of Nursing. Faculty of Nursing, Physiotherapy and Podology. University Complutense of Madrid, Madrid, 28041 Spain

    FIBHCSC, Instituto de Investigación Sanitaria Hospital Clínico San Carlos (IdISSC). Address: Calle del Prof Martín Lagos, s/n, 28040, Madrid, Spain
    Search for articles by this author
  • P Posada-Moreno
    Affiliations
    Department of Nursing. Faculty of Nursing, Physiotherapy and Podology. University Complutense of Madrid, Madrid, 28041 Spain

    FIBHCSC, Instituto de Investigación Sanitaria Hospital Clínico San Carlos (IdISSC). Address: Calle del Prof Martín Lagos, s/n, 28040, Madrid, Spain
    Search for articles by this author
  • R Sánchez-Gómez
    Affiliations
    Department of Nursing. Faculty of Nursing, Physiotherapy and Podology. University Complutense of Madrid, Madrid, 28041 Spain
    Search for articles by this author
  • M Raurell-Torredà
    Affiliations
    Department of Fundamental and Medical Surgical Nursing, Faculty of Medicine and Health Sciences, University of Barcelona, 08907 Spain
    Search for articles by this author
Open AccessPublished:July 27, 2021DOI:https://doi.org/10.1016/j.ecns.2021.07.003

      Highlights

      • Despite the confinement, nursing students could receive virtual hands-on training.
      • Virtual training can increase the knowledge and self-confidence of nursing students.
      • vSim® may be a useful tool for developing skills during periods of confinement.
      • Students expressed a high level of satisfaction with the virtual simulation platform.

      Abstract

      Background

      The COVID-19 pandemic has made it necessary to adapt university health-education. Virtual simulation has been proposed to be a suitable tool.

      Methods

      A quasi-experimental study was conducted on nursing students in the final year. The virtual simulation platform vSim® was used. Improvements in knowledge, skills during simulation, satisfaction and selfconfidence obtained through the training provided were analyzed, as well as satisfaction with the platform.

      Results

      Prepost training knowledge improved. Skill acquisition improved between the first and last attempts in all cases. The levels of selfconfidence and satisfaction with the training and the platform used were high.

      Conclusions

      The vSim® was a useful solution during the pandemic. Knowledge improved and high selfconfidence was obtained.

      Keywords

      Key Points
      • It is suggested that the use of a virtual simulation platform may be a useful option for training nursing students during periods of confinement such as the one produced by COVID-19.
      • Knowledge and skills of nursing students are improved through the use of virtual simulation. In addition, students show high levels of self-confidence and satisfaction with learning and with the use of the platform, which they recommend for future courses.
      • A virtual simulation platform that is specific to nursing can be effectively used in confinement situations. It would be desirable to create cases adapted to all areas of the academic curriculum and to elaborate distance activities that develop technical skills.

      Introduction

      The global pandemic of coronavirus disease 2019 (COVID-19) has brought about a change in our way of life, including traditional forms of health care education. As of March 2020, social distancing and/or confinement policies have been implemented worldwide. These measures have forced us to change our educational model, and carrying out hands-on health care activities, including practical courses for nursing students, is particularly problematic (
      • Rose S.
      Medical student education in the time of COVID-19.
      ).
      While it is true that face-to-face training will continue to be an essential component of the clinical curriculum, distance learning may prove to be an effective and flexible alternative or complement (
      • Newman N.A.
      • Lattouf O.M.
      Coalition for medical education-A call to action: a proposition to adapt clinical medical education to meet the needs of students and other healthcare learners during COVID-19.
      ). The use of new technologies is a key element in this, as they allow health students and professionals to increase their knowledge and improve their skill acquisition. Technological developments in recent years have allowed simulation to take center stage in clinical teaching, and it has been shown to be an effective learning methodology as it allows for teaching technical skills and nontechnical skills (decision-making, leadership, teamwork and communication, situational awareness, resource management, safe practice and reduction of adverse events) (
      • Cant R.P.
      • Cooper S.J.
      Use of simulation-based learning in undergraduate nurse education: an umbrella systematic review.
      ).
      Simulation allows students to link theory and practice (
      • Alamrani M.H.
      • Alammar K.A.
      • Alqahtani S.S.
      • Salem O.A.
      Comparing the effects of simulation-based and traditional teaching methods on the critical thinking abilities and self-confidence of nursing students.
      ). Its use as a pedagogical method is based on reproducing routine or special clinical situations in a controlled and safe way for the students. The objective of simulation is to create a better understanding and handling of essential aspects when these occur in a real-life context. As a cognitive and behavioral education strategy, simulation promotes high levels of selfesteem and selfconfidence, since it allows the internalization of information and satisfaction with the learning process (
      • Cant R.P.
      • Cooper S.J.
      Use of simulation-based learning in undergraduate nurse education: an umbrella systematic review.
      ).
      In spite of all this, clinical simulation, even when carried out in small groups, is usually of a face-to-face nature, which can be a problematic matter during a situation of confinement and/or social distancing (
      • Almarzooq Z.I.
      • Lopes M.
      • Kochar A.
      Virtual learning during the COVID-19 pandemic: a disruptive technology in graduate medical education.
      ). Given that it is not known how long this situation will last, or if we will have similar situations in the future, we must find effective and feasible solutions for the clinical training of our students (
      • Rose S.
      Medical student education in the time of COVID-19.
      ). Education experts propose the use of virtual teaching, including web-based virtual simulation (VS), since it complies with social distancing and it allows students to choose the time and place for learning (
      • Rose S.
      Medical student education in the time of COVID-19.
      ). Moreover, VS offers a high level of interactivity and realism to the student (
      • Gu Y.
      • Zou Z.
      • Chen X.
      The effects of vSIM for NursingTM as a teaching strategy on fundamentals of nursing education in undergraduates.
      ).
      In this scenario, in addition to implementing innovations that allow appropriate training of future professionals, we must evaluate the impact of VS will have on the student. Knowledge as well as satisfaction are considered a strong indicators of quality in educational institutions, mainly due to the fact that they indicate the effectiveness of the teaching strategies used (
      • Almeida R.G.
      • Mazzo A.
      • Martins J.C.
      • Baptista R.C.
      • Girão F.B.
      • Mendes I.A.
      Validation to Portuguese of the scale of student satisfaction and self-confidence in learning.
      ). Furthermore, trust and selfconfidence are important aspects of the cognitive component of an individual's self-efficacy. Self-confidence can be understood as the conviction that an individual is capable of doing or achieving something. In the clinical setting, it can be translated into the personal capability of individuals to achieve their own goals (
      • Almeida R.G.
      • Mazzo A.
      • Martins J.C.
      • Baptista R.C.
      • Girão F.B.
      • Mendes I.A.
      Validation to Portuguese of the scale of student satisfaction and self-confidence in learning.
      ). The purpose of this study was to evaluate if a based-web VS platform is a useful tool, in terms of knowledge, satisfaction and self-confidence in learning in times of COVID-19.

      Methodology

      Study Design, Setting and Participants

      A quasi-experimental study, prepost intervention, with control group for knowledge was conducted during May and July 2020 involving last-year undergraduate nursing students in confinement due to the COVID-19 pandemic.

      Intervention Group

      Students who had not completed 50% of their practical clinical training periods in health centers during the final year of their nursing degree studies.

      Control Group

      Students who decided to compensate their lack of practical training through a health-care assistant contract. The possibility of carrying out these activities was offered by the university and the health authorities.

      Ethics

      The study was approved by the Ethics Committee of our reference center (Ref: C.I.20/418-E_COVID). All participants were informed of the purpose of the study and agreed to participate in it. The result of the scores did not affect their final grades of any subject; they only had to perform the exercise. When data collection was completed, all participants in the control group were given voluntary and free access to the platform.

      Description of the Intervention

      The intervention group received training through the vSim® for Nursing platform. The platform was developed through a collaboration between Wolters Kluwer, Laerdal and the National League for Nursing (NLN). vSim® has peer-reviewed clinical cases and the simulations are based on high-fidelity manikins adapted for use in a virtual environment. This web-based VS measures the students' actions in terms of low, moderate and high risk for patient harm. In addition, it calculates a score based on correctly performed nursing activities.
      The intervention process consisted of three phases and five steps:
      Phase 1. Checking and preparatory
      • 1)
        Pretraining knowledge evaluation: before starting the training with the virtual platform, the intervention group was given a test on basic concepts of pharmacology, pathophysiology and nursing interventions. The intention was to establish the baseline in terms of knowledge of the topics to be treated in the clinical cases. The control group did not receive training during the study period. Instead, they conducted supervised health care activities in health care centers.
      • 2)
        Familiarization with the platform: since none of the students had used vSim®, all of them received training on the use of the platform through video conferencing by a vSim® instructor. For this purpose, a case was used that was not included in the training afterwards. All commands of the simulator and the operation of the system were explained. A document was provided with all commands adjusted to the native language of the participants, so as to facilitate navigation through the platform.
      Phase 2. Experimental
      • 3)
        Clinical cases: each student was given individually five clinical scenarios: two fundamental nursing cases (hip fracture and pneumonia) and three medical-surgical cases (asthma, diabetes and intestinal obstruction). Before carrying out each case, the students had to review the case and study its contents provided by the platform. These contents are based on various Wolters Kluwers publications. The platform provides different external links so that the student can access updated content. They then proceeded to perform the simulation for a maximum of 30 minutes. Finally, the program gave feedback on those aspects that were done properly and on those that could be improved. The targets were: each case had to be repeated until a score of at least 80% was achieved and the students were not allowed to have any error classified as high or moderate risk to the patient. Each case could be carried out for six days.
      • 4)
        Online debriefing: these were programmed on the seventh day of each case by the simulation instructor. The debriefing format proposed by the platform was used: opening questions, scenario analysis questions and concluding questions. Each session lasted approximately 90 minutes.
      Phase 3. Evaluation.
      • 5)
        Post-training evaluation: a post-training knowledge test was done, the same test as given at the pretraining to the two study groups, control and intervention. We did not consider it necessary to carry out a preintervention knowledge test in the control group, since the training of all participants up to the time of the intervention was the same. In addition, selfconfidence and satisfaction with the platform used were assessed in the intervention group.

      Main Objective

      • To compare the level of knowledge prepost intervention in the experimental group and with the control group.

      Secondary Objectives

      • To compare the percentages of nursing activities performed correctly between the first and last attempts for each clinical case in the experimental group.
      • To evaluate the level of self-confidence in the training received in the experimental group.
      • To assess the degree of satisfaction with the use of the VS tool used in the experimental group.

      Assessment Instruments

      • Knowledge test. The 35 questions were developed and reviewed along with the clinical cases by experts from the NLN, Laerdal and Wolters Kluwer. Topics related to the different clinical cases were addressed.
      • Score obtained at the end of the simulations until achieving the targets. The program records each decision and compares the student's performance with the expert's performance, and ultimately yields a score (NLN, ). Each clinical case has a set of goals defined by the virtual platform. The participant is required to complete different Nursing Activities during the virtual simulation. Omission, incorrect execution, or carrying out of activities that could put the patient at risk, are penalized in the final score. Incorrect activities are categorized by asterisks into three levels: High risk (***) - for example, administering medication to the patient without asking if he and/or she has any allergies; Moderate risk (**) - for example, not providing health education to the patient or their family when necessary; and Low risk (*) - for example, failure to assess blood pressure after administering methyl prednisolone to the patient.
      • Selfconfidence obtained by the training with the VS platform. The Jeffries & Rizzolo questionnaire (
        • Jeffries P.R.
        • Rizzolo M.A.
        Designing and implementing models for the innovative use of simulation to teach nursing care of ill adults and children: a national, multisite, multimethod study.
        ) adapted to our context was used. The questionnaire comprises 13 items with a Cronbach's alpha of 0.9. It has 2 dimensions: satisfaction and self-confidence.
      • Degree of satisfaction with the platform. This was compiled through a questionnaire designed in 2 previous studies focused on student satisfaction with vSim® (
        • Foronda C.L.
        • Swoboda S.M.
        • Hudson K.W.
        • Jones E.
        • Sullivan N.
        • Ockimey J.
        • et al.
        Evaluation of vSIM for Nursing™: a trial of innovation.
        ;
        • Tjoflåt I.
        • Brandeggen T.K.
        • Strandberg E.S.
        • Dyrstad D.N.
        • Husebø S.E.
        Norwegian nursing students' evaluation of vSim® for Nursing.
        ).

      Variables

      • a)
        Socio-demographic and academic characteristics of the students: belonging to the control or experimental group, age, gender, level of English and computer skills.
      • b)
        Variables associated with learning:
        Pre and post training knowledge regarding the programmed cases.
        Score obtained at the end of the simulations until achieving the goals.
      • c)
        Variables related to the student's perception of learning:
      Level of selfconfidence in learning.
      Degree of satisfaction.

      Sample Size

      A sample size of 51 individuals was estimated, accepting an alpha risk of 0.05 and a beta risk of 0.2 in a bilateral contrast, with a 95% confidence interval. The calculation is based on a sensitivity to change for the skills acquired with the simulation equal to 1.145. A difference of ≥ 4.5 units is expected to be detected.

      Statistical Analysis

      Categorical variables were expressed as frequency and percentage. Fisher's or Chi-squared test was used for between-group comparisons. Quantitative variables were expressed as mean and standard deviation and median and interquartile range, and the data were compared using the Wilcoxon test for paired data and the Mann–Whitney U test for independent data. The effect size was calculated for the knowledge gained and Cronbach's alpha was obtained from the self-confidence scale. Data were analysed using IBM SPSS for Windows 25.0 software.

      Results

      132 participants were assessed for eligibility. Six not meeting inclusion criteria and fourteen declined to participate. Finally, 112 nursing students were analyzed, 56 of the experimental group and 56 of the control group.
      Most were women with a median age of 22 years. Three quarters of the students declared to have good or very good computer skills. More than 80% said their English skills were average, good or very good, and one third said they had some certificate of English (Table 1).
      Table 1General Characteristics of the Study Population.
      VariableExperimental Group (n = 56)Control Group (n = 56)p-Value
      Age in years (Median [IQR])22 [21.25-25]22 [21.25-24].758
      Mann-Whitney test.
      Gender (%[n])Female87.5% (49)76.8% (43).217
      Chi-square.
      Computer skills (% [n])Very good23.2% (13)21.4% (12).841
      Chi-square.
      Good51.8% (29)53.6% (29).896
      Chi-square.
      Average21.4% (12)23.2% (13).841
      Chi-square.
      Sufficient3.6% (2)1.8% (1).564
      Chi-square.
      Poor0.0% (0)0.0% (0)-
      Knowledge of English (% [n])Very good17.9% (10)12.5% (7).467
      Chi-square.
      Good30.4% (17)35.7% (20).622
      Chi-square.
      Average33.9% (19)35.7% (20).873
      Chi-square.
      Sufficient10.7% (6)12.5% (7).782
      Chi-square.
      Poor7.1% (4)3.6% (2).414
      Chi-square.
      Do you have any certificate of English? (% [n])Yes35.7% (20)30.4% (17).688
      Chi-square.
      IQR = interquartile range; n = sample.
      low asterisk Chi-square.
      Mann-Whitney test.

      Knowledge Acquisition. Intervention and Control Groups

      The score obtained in the knowledge tests showed a significant post-training increase in the experimental group (median = 5.6 vs. 9.4; p < .001). The improvement in the experimental group was also significant compared to the control group (median = 9.4 vs. 5.0; p < .001). In both cases the effect size is considered to be large. By contrast, the score of the control group is similar to the pretraining score of the experimental group (median = 5.0 vs. 5.6; p = .117) (Table 2).
      Table 2Evolution of prepost training knowledge.
      GroupKnowledge Test Score
      Presimulation Score(n = 56)Postsimulation Score(n = 56)p-ValueEffect Sizer (z)
      Experimental Group

      Median [IQR] (n = 56)
      5.63 [4.69-6.25]9.38 [8.13-10.00]<.001
      Wilcoxon test. Prepost simulation experimental group;
      0.870 (-6.513)
      Control Group

      Median [IQR] (n = 56)
      5.00 [4.69-5.94]<.001
      Mann-Whitney test. Post-simulation score experimental group vs control group;
      0.836 (-8.852)
      p-value0.117
      Mann-Whitney test. Presimulation score experimental group versus control group.
      IQR = interquartile range;
      low asterisk Wilcoxon test. Prepost simulation experimental group;
      Mann-Whitney test. Post-simulation score experimental group vs control group;
      Mann-Whitney test. Presimulation score experimental group versus control group.

      Skills Development. Intervention Group

      Figure A.1 (appendix material) shows the evolution of the scores between the first and last simulation carried out for each scenario. Repeating the simulations resulted in a significant increase in the scores obtained in all cases (p < .001). Moreover, after the first scenario the values of the first simulation showed a higher median. However, after the third simulation there is a decrease in the initial median in the fourth scenario, but the trend recovers in the fifth scenario. The average number of attempts per student to reach the target score was 2.9 ± 0.9 and the median was 2.6 [2.2-3.4]. Scenario one was repeated a median of 2.5 times [2.0-4.0], scenario two 3.5 times [3.0-5.0], scenario three 2.0 times [1.8-3.0], scenario four 2.0 times [2.0-4.0], and scenario five 2.0 times [1.0-3.0]. The repetition of the scenarios decreases significantly between the first and last scenarios (2.5 [2.0-4.0] vs. 2.0 [1.0-3.0]; p < .001), although it increases between the first and second scenarios (2.5 [2.0-4.0] vs. 3.5 [3.0-5.0]; p = .002).

      Satisfaction and Self-Confidence Through VS Activity. Intervention Group

      Satisfaction levels showed a median of 4.6 [4.0-4.8] out of 5. One individual (1.8%) disagreed with the statement that the simulations done provided materials and activities that favored their learning of the clinical curriculum. The median self-confidence in learning level was 4.3 [4.0-4.5] out of 5. Eight students (14.3%) disagreed with the statement that the training covered the critical contents of their clinical curriculum. Three students (5.4%) did not find that the activity performed had provided them with the skills and knowledge necessary to carry out their tasks in the clinical setting. Five students (8.9%) felt that it was not the teacher's responsibility to tell them what to learn from the content of the simulation activities (Table 3). The Cronbach alpha coefficient of the scale was 0.811.
      Table 3Satisfaction and self-confidence with the simulation training.
      Satisfaction with virtual simulation learningMedian [IQR]SDn (%)Dn (%)UNn (%)An (%)SAn (%)
      “Satisfaction” construct4.6

      [4.0-4.8]
      1. The teaching methods used in this simulation were helpful and effective.5.0

      [4.0-5.0]
      0

      (0.0)
      0

      (0.0)
      3

      (5.4)
      22 (39.3)31 (55.4)
      2. The simulation provided me with a variety of learning materials and activities to promote my learning the medical surgical curriculum.5.0

      [4.0 -5.0]
      0

      (0.0)
      1

      (1.8)
      4

      (7.1)
      22 (39.3)29 (51.8)
      3. I enjoyed how my professor taught through the simulation.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      4

      (7.1)
      17 (30.4)35 (62.5)
      4. The teaching materials used in this simulation were motivating and helped me to learn.4.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      8

      (14.3)
      28

      (50)
      20 (35.7)
      5. The way my instructor(s) taught the simulation was suitable to the way I learn.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      3

      (5.4)
      19 (33.9)34 (60.7)
      Self-confidence in Learning
      “Self-confidence” construct4.3

      [4.0 -4.5]
      6. I am confident that I am mastering the content of the simulation activity that my instructors presented to me.4.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      6

      (10.7)
      27 (48.2)23 (41.1)
      7. I am confident that this simulation covered critical content necessary for the mastery of medical surgical curriculum.4.0

      [3.0 -5.0]
      1

      (1.8)
      7

      (12.5)
      7

      (12.5)
      24 (42.9)15.2 (30.4)
      8. I am confident that I am developing the skills and obtaining the required knowledge from this simulation to perform necessary tasks in a clinical setting.4.0

      [3.0 -4.0]
      0

      (0.0)
      3

      (5.4)
      14

      (25)
      29 (51.8)10 (17.9)
      9. My instructors used helpful resources to teach the simulation.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      3

      (5.4)
      14

      (25)
      39 (69.6)
      10. It is my responsibility as the student to learn what I need to know from this simulation activity.5.0

      [4.25-5.0]
      0

      (0.0)
      0

      (0.0)
      1

      (1.8)
      13 (23.2)42 (75.0)
      11. I know how to get help when I do not understand the concepts covered in the simulation.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      4

      (7.1)
      15 (26.8)37 (66.1)
      12. I know how to use simulation activities to learn critical aspects of these skills.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      1

      (1.8)
      25 (44.6)30 (53.6)
      13. It is the instructor's responsibility to tell me what I need to learn of the simulation activity content during class time.4.0

      [3.0 -4.0]
      0

      (0.0)
      5

      (8.9)
      18 (32.1)25 (44.6)8 (14.3)
      (This questionnaire is © Copyright of the National League for Nursing, 2005).
      Note. A = agree with the statement; D = disagree with the statement; IQR = interquartile range; SA = strongly agree with the statement; SD = strongly disagree with the statement; UN = Undecided - you neither agree or disagree with the statement.

      Satisfaction With the Platform

      For all items of the platform satisfaction survey, the agreement was greater than 90%, except for question 8 (82.1%). Four students (7.2%) disagreed with the statement that vSim® was a useful tool (Table 4).
      Table 4Satisfaction with the tool used.
      Satisfaction with vSim®Median [IQR]SDn (%)Dn (%)UNn (%)An (%)SAn (%)
      1. vSim® was easy to navigate.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      1

      (1.8)
      20

      (35.7)
      35 (62.5)
      2. It was motivating to work with vSim®.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      2

      (3.6)
      14 (25.0)40 (71.4)
      3. vSim® was useful to learn new knowledge.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      3

      (5.4)
      21 (37.5)32 (57.1)
      4. vSim® was useful for reinforcing knowledge.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      3

      (5.4)
      17

      (30.4)
      36 (64.3)
      5. To work with vSim® was a good preparation for clinical practice.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      4

      (7.1)
      20 (35.7)32 (57.1)
      6. The content of vSim® was relevant for my role as a nurse.4.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      4

      (7.1)
      26 (46.4)26 (46.4)
      7. vSim® provided me with different learning possibilities that promoted learning.5.0

      [4.0 -5.0]
      0

      (0.0)
      0

      (0.0)
      5

      (8.9)
      18 (32.1)33 (58.9)
      8. vSim® is a useful tool.4.0

      [4.0 -5.0]
      1

      (1.8)
      3

      (5.4)
      6

      (10.7)
      21 (37.5)25 (44.6)
      9. I recommend vSim® for future courses.5.0

      [5.0 -5.0]
      0

      (0.0)
      1

      (1.8)
      1

      (1.8)
      8

      (14.3)
      46 (82.1)
      Total4.7

      [4.2-4.8]
      Note. A = agree with the statement; D = disagree with the statement; IQR = interquartile range; SA = strongly agree with the statement; SD = strongly disagree with the statement; UN = Undecided - you neither agree or disagree with the statement.

      Discussion

      The data obtained support the use of VS for nursing students under social distancing conditions. They show an improvement in both knowledge and skill acquisition. In addition, students showed high levels of satisfaction and selfconfidence with the training received. They found the VS platform useful and would recommend it for future training.

      Knowledge Acquisition

      In the intervention group, the post-training knowledge was increased and the magnitude of the effect size was considerable. Knowledge also showed a significant increase when comparing the control group with the experimental group, since baseline knowledge was the same in both groups. We found few publications related to the use of the vSim® that also evaluate the acquisition of knowledge with the tool.
      • Gu Y.
      • Zou Z.
      • Chen X.
      The effects of vSIM for NursingTM as a teaching strategy on fundamentals of nursing education in undergraduates.
      reported similar results regarding knowledge evaluation (experimental group = 73.31 ± 9.27 vs. control group = 65.36 ± 8.93; p = .032). Other studies carried out with different VS platforms in the health field describe equally positive results (
      • Watari T.
      • Tokuda Y.
      • Owada M.
      • Onigata K.
      The utility of virtual patient simulations for clinical reasoning education.
      ;
      • Borg-Sapiano A.
      • Sammut R.
      • Trapani J.
      The effectiveness of virtual simulation in improving student nurses' knowledge and performance during patient deterioration: a pre and post test design.
      ).
      On the other hand, a recent study carried out in Korea used vSim® in the mental health field (
      • Liu W.
      The effects of virtual simulation on undergraduate nursing students' mental health literacy: a prospective cohort study.
      ), and did not find an improvement in knowledge. According to the authors, a possible explanation was the “ceiling effect” (
      • Bond K.S.
      • Jorm A.F.
      • Kitchener B.A.
      • Reavley N.J.
      Mental health first aid training for Australian medical and nursing students: an evaluation study.
      ), where there is no room for improvement if knowledge is already high. Furthermore, they state that they did not hold debriefing sessions. This situation is also described by
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      , who find no differences between different teaching methodologies (experimental group 1 = 58.06 vs. experimental group 2 = 59.03 vs. control group = 55.31; p = .433), including VS. They point out that one of their limitations is the lack of debriefing after the cases. Many authors argue that debriefing is the most important part of simulations for the acquisition of knowledge and the development of skills through critical thinking (
      • Alhaj-Ali A.
      • Miller E.
      • Ballman K.
      • Bakas T.
      • Geis G.
      • Ying J.
      The impact of debriefing modalities on nurse practitioner students' knowledge and leadership skills in managing fatal dysrhythmias: a pilot study.
      ;
      • Schmidt M.
      • Haglund K.
      Debrief in Emergency Departments to Improve Compassion Fatigue and Promote Resiliency.
      ). In our case, there was room for improvement and specific debriefing was carried out after completing each VS scenario.

      Skill Development

      Our results support the claims that VS can serve as a tool to improve students' preparation for healthcare/medical/assistance practice (
      • Tabatabai S.
      COVID-19 impact and virtual medical education.
      ;
      • Tabatabai S.
      • Javadi M.A.
      Ophthalmic education and ophthalmologists growth trends in Iran (1979-2016).
      ). A significant improvement was observed between the first and the last simulation performed in each case. We observed a decrease in the median of the first simulation in the fourth scenario, which recovers in the fifth scenario. This may be due to the fact that scenarios 1, 2, and 3 belong to the fundamental care training module whereas scenarios 4 and 5 belong to the medical-surgical care module. Although both modules contain similar aspects, there are different skills to be addressed. Addressing several cases within each module seems to improve the score of the first simulations. Moreover, according to different authors, repeating the simulations helps to improve learning (
      • Springer R.
      • Mah J.
      • Shusdock I.
      • Brautigam R.
      • Donahue S.
      • Butler K.
      Simulation training in critical care: does practice make perfect?.
      ).
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      found that, in order to meet the training objective with vSim®, the students required several attempts; they did not specify the number of attempts, but reported that 78% and 69% of the respective experimental groups of students managed to reach 50% of the objectives at the first attempt. We found that the students required a median of 2.6 [2.2-3.4] attempts to reach the programmed target. Although we found a decrease between the repetition of the first and last scenario, we did not find a gradual trend between the scenarios. The difficulty and specific characteristics of each case may be the reason for this variation. What seems clear is that, in most scenarios, it is necessary to perform the scenario at least twice to obtain an adequate score. Moreover, most students repeated the simulations until they reached scores close to 100%. This may be related to the high satisfaction found with the use of the platform.
      • Yeh T.K.
      • Huang H.M.
      • Chan W.P.
      • Chang C.Y.
      Effects of congruence between preferred and perceived learning environments in nursing education in Taiwan: a cross-sectional study.
      state that students' preferences and positive perceptions significantly improve learning outcomes, although this is not always the case.
      • Üzen Cura Ş.
      • Kocatepe V.
      • Yıldırım D.
      • Küçükakgün H.
      • Atay S.
      • Ünver V.
      Examining knowledge, skill, stress, satisfaction, and self-confidence levels of nursing students in three different simulation modalities.
      evaluate 3 different simulation modalities in relation to chest auscultation: “standardized patient,” “high-fidelity manikin” and a “task trainer.” The highest student satisfaction was obtained with the standardized patient, but in terms of skill development it obtained the poorest ratings. This may be due to the increased stress that simulation with a standardized patient usually generates compared to other methods. Different authors relate these facts to psychological safety and state that the development of skills and learning improves when stress levels decrease (
      • Cheng A.
      • Kolbe M.
      • Grant V.
      • Eller S.
      • Hales R.
      • Symon B.
      • et al.
      A practical guide to virtual debriefings: communities of inquiry perspective.
      ). This may be the reason why VS gives high levels of satisfaction and high scores in the acquisition of knowledge and skills.
      • Kang S.J.
      • Hong C.M.
      • Lee H.
      The impact of virtual simulation on critical thinking and self-directed learning ability of nursing students.
      use the vSim® and affirm that students feel psychologically safe when performing the simulations. They argue that the students perform the simulations in a safe environment, such as their home, and at the time they consider most appropriate. We believe that this methodology can be the first stage to acquire knowledge and skills, so that later the teaching can be made more complex with more stressful situations. We believe that before presenting complex and stressful scenarios, it would be convenient for students to develop skills in psychologically safe environments. Such a situation is adequately recreated by VS. Further studies are required to confirm these assumptions.

      Satisfaction and Self-Confidence Through VS Activity

      Satisfaction and selfconfidence in learning levels with the training show high median scores (>4/5 points). It can also be seen that satisfaction with learning ratings are slightly higher than self-confidence ratings. Given the scarce research on VS, we did not find studies that analyze these parameters. Nevertheless, scores similar to those described in studies carried out with nursing students using high and medium fidelity face-to-face simulation have been observed (
      • Costa R.R.O.
      • Medeiros S.M.
      • Coutinho V.R.D.
      • Mazzo A.
      • Araújo M.S.
      Satisfaction and self-confidence in the learning of nursing students: randomized clinical trial.
      ;
      • Almeida R.G.
      • Mazzo A.
      • Martins J.C.
      • Baptista R.C.
      • Girão F.B.
      • Mendes I.A.
      Validation to Portuguese of the scale of student satisfaction and self-confidence in learning.
      ;
      • Franklin A.E.
      • Burns P.
      • Lee C.S.
      Psychometric testing on the NLN student satisfaction and self-confidence in learning, simulation design scale, and educational practices questionnaire using a sample of pre-licensure novice nurses.
      ). The good results can be explained by the structure of the training (prebriefing-simulation-debriefing), which according to different authors positively affects satisfaction, self-confidence and learning results (
      • Gu Y.
      • Zou Z.
      • Chen X.
      The effects of vSIM for NursingTM as a teaching strategy on fundamentals of nursing education in undergraduates.
      ;
      • Almeida R.G.
      • Mazzo A.
      • Martins J.C.
      • Baptista R.C.
      • Girão F.B.
      • Mendes I.A.
      Validation to Portuguese of the scale of student satisfaction and self-confidence in learning.
      ). On the other hand, the possibility of repetition and immediate feedback can increase the security and learning of the students (
      • Gu Y.
      • Zou Z.
      • Chen X.
      The effects of vSIM for NursingTM as a teaching strategy on fundamentals of nursing education in undergraduates.
      ; Wright
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      ).
      • Zapko K.A.
      • Ferranto M.
      • Blasiman R.
      • Shelestak D.
      Evaluating best educational practices, student satisfaction, and self-confidence in simulation: a descriptive study.
      , state that serial simulations during training seem to increase the confidence of the students.
      As in other studies, item number 13, which corresponds to self-confidence, was the one that obtained the least agreement. Of the students, 58.9% said they agreed or strongly agreed (mean = 3.6 ± 0.8 out of 5). These numbers are similar to those described by authors who use classroom simulation with a manikin: agreement of 68.42% (
      • Franklin A.E.
      • Burns P.
      • Lee C.S.
      Psychometric testing on the NLN student satisfaction and self-confidence in learning, simulation design scale, and educational practices questionnaire using a sample of pre-licensure novice nurses.
      ) and a mean of 3.55 ± 1.12 out of 5 (
      • Almeida R.G.
      • Mazzo A.
      • Martins J.C.
      • Baptista R.C.
      • Girão F.B.
      • Mendes I.A.
      Validation to Portuguese of the scale of student satisfaction and self-confidence in learning.
      ).
      Regarding the dimension “satisfaction,” item 4 was the worst rated. However, no participant disagreed with the statement that the materials were motivating. Some authors state that not all people find the same motivation in the use of technologies, which can alter the perception of this activity (
      • Foronda C.L.
      • Swoboda S.M.
      • Hudson K.W.
      • Jones E.
      • Sullivan N.
      • Ockimey J.
      • et al.
      Evaluation of vSIM for Nursing™: a trial of innovation.
      ).
      Concerning the Self-Confidence in Learning dimension, eight participants (14.3%) disagreed with item 7 and three participants (5.4%) with item 8. This may be due to the facts that the tool was used as a substitute for hands-on training in the COVID19 situation and that the subjects were last-year students. In their practical training period they rotate among Critical Care units. They were given fundamental nursing and medical-surgical scenarios that did not involve Critical Care cases. Other studies carried out with vSim® were done during initial nursing courses (
      • Gu Y.
      • Zou Z.
      • Chen X.
      The effects of vSIM for NursingTM as a teaching strategy on fundamentals of nursing education in undergraduates.
      ;
      • Tjoflåt I.
      • Brandeggen T.K.
      • Strandberg E.S.
      • Dyrstad D.N.
      • Husebø S.E.
      Norwegian nursing students' evaluation of vSim® for Nursing.
      ) or as a complement to training. It would be useful to obtain different scenarios that cover all curricular needs of these students (i.e., critical care simulations). As for item 8, it is true that VS is related to the development of nontechnical skills (
      • Redmond C.
      • Hardie P.
      • Davies C.
      • Cornally D.
      • Daly O.
      • O'Sullivan C.
      Increasing competence in wound care: A cross-sectional study to evaluate use of a virtual patient by undergraduate student nurses.
      ). It seems that it is necessary to add an activity that allows the development of technical skills at a distance, to complement the acquisition of skills.

      Satisfaction With the Platform

      As in other studies, a high percentage of students recommend the use of this platform (Wright,
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      ;
      • Foronda C.L.
      • Swoboda S.M.
      • Hudson K.W.
      • Jones E.
      • Sullivan N.
      • Ockimey J.
      • et al.
      Evaluation of vSIM for Nursing™: a trial of innovation.
      ;
      • Foronda C.
      • Budhathoki C.
      • Salani D.
      Use of multiuser, high-fidelity virtual simulation to teach leadership styles to nursing students.
      ), which suggests that it proved to be a positive and productive experience. Some studies show that there are students who have difficulties with navigation on the platform (
      • Tjoflåt I.
      • Brandeggen T.K.
      • Strandberg E.S.
      • Dyrstad D.N.
      • Husebø S.E.
      Norwegian nursing students' evaluation of vSim® for Nursing.
      ; Wright,
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      ;
      • Foronda C.L.
      • Swoboda S.M.
      • Henry M.N.
      • Kamau E.
      • Sullivan N.
      • Hudson K.W.
      Student preferences and perceptions of learning from vSIM for Nursing™.
      ). They state that both the language and the technology itself can be challenging for the use of this simulator. In our case we did not encounter these obstacles, possibly because: (1) our students have a good level of English. Moreover, the research team provided a document with the simulator controls adapted to their language; (2) the students claimed to have good computer skills. Furthermore, as in Tabatabai's study (
      • Tabatabai S.
      Simulations and virtual learning supporting clinical education during the COVID 19 pandemic.
      ), most of them are digital natives, so the simulated experience is something natural for them. Only one student disagreed, and this was the oldest student; (3) like
      • Kang S.J.
      • Hong C.M.
      • Lee H.
      The impact of virtual simulation on critical thinking and self-directed learning ability of nursing students.
      , we gave an online presentation of the platform to all students, which allowed direct visualization of both the controls and the possibilities of the simulator. We believe that this familiarization with the environment was key for the students to be able to navigate without losing time, as also stated by other authors (Wright,
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      ; Foronda et al.,
      • Foronda C.L.
      • Swoboda S.M.
      • Henry M.N.
      • Kamau E.
      • Sullivan N.
      • Hudson K.W.
      Student preferences and perceptions of learning from vSIM for Nursing™.
      ).
      • Wright R.R.
      • Tinnon E.A.
      • Newton R.H.
      Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
      recommended their students to watch a tutorial about the tool, but they did not hold an explanatory session in which the students could interact with the teacher. Finally, the students said they had difficulties with navigation, although they later confirmed that they had not in fact watched the tutorial.
      • Foronda C.L.
      • Swoboda S.M.
      • Henry M.N.
      • Kamau E.
      • Sullivan N.
      • Hudson K.W.
      Student preferences and perceptions of learning from vSIM for Nursing™.
      left the following question open: “How would the results have been if the technology had been learned online?.”
      • Liu W.
      The effects of virtual simulation on undergraduate nursing students' mental health literacy: a prospective cohort study.
      suggests the possibility of using this tool in situations of pandemics such as COVID-19. Based on the results obtained, we suggest that the implementation of 100% remote VS with this platform may be an adequate and feasible solution to develop fundamental, especially nontechnical, skills. However, it seems necessary to increase the number of cases adapted to the skills of each course and to find a way to combine it with distance learning for developing technical skills.

      Limitations

      The study was conducted at a single center. Although it has been used in different studies, the questionnaire on satisfaction with the tool has not been validated. The students did not carry out clinical cases related to critical care. vSim® is a fee-based platform, one license per student. Perhaps not all institutions are able to purchase it annually. Finally, we did not analyze whether the improvement in knowledge with the intervention is maintained over time.

      Conclusions

      Training using the vSim® increases knowledge, allows the development of skills and provides a high level of self-confidence in learning to nursing students. Satisfaction with both the training and the virtual tool is high. New clinical cases that allow the development of skills in all aspects of the nursing curriculum are necessary. This would open the door for further distance learning research that allows the acquisition of non-technical skills.

      Funding Statement

      This work was supported by “Innova UCM” program of the Complutense University of Madrid [grant numbers: 255, 2020].

      Ethics Approval Statement

      The research meets all ethical requirements. The study was approved by the Clínico San Carlos Hospital´s Ethics Committee under number: C.I.20/418-E_COVID.

      Contributors

      (1) The conception and design of the study, or acquisition of data, or analysis and interpretation of data: Zaragoza-García I; Raurell-Torredá M, Ortuño-Soriano I, Sánchez-Gómez R., posada-Moreno MP.
      (2) Drafting the article or revising it critically for important intellectual content: Zaragoza-García I; Raurell-Torredá M, Ortuño-Soriano I, Sánchez-Gómez R., posada-Moreno MP.
      (3) Final approval of the version to be submitted: Zaragoza-García I; Raurell-Torredá M, Ortuño-Soriano I, Sánchez-Gómez R., posada-Moreno MP.

      Declarations of Interest

      The authors declare that there is no conflict of interest regarding the publication of this article.

      Acknowledgments

      We thank Ron Hartong for his help with the language and writing assistance. In addition, we would like to thank all the nursing students who participated in the study.

      Appendices

      Figure A1
      Figure A.1Evolution of the scores of activities performed correctly between the first and last attempt of each scenario.

      References

        • Alamrani M.H.
        • Alammar K.A.
        • Alqahtani S.S.
        • Salem O.A.
        Comparing the effects of simulation-based and traditional teaching methods on the critical thinking abilities and self-confidence of nursing students.
        The journal of nursing research: JNR. 2018; 26: 152-157https://doi.org/10.1097/jnr.0000000000000231
        • Alhaj-Ali A.
        • Miller E.
        • Ballman K.
        • Bakas T.
        • Geis G.
        • Ying J.
        The impact of debriefing modalities on nurse practitioner students' knowledge and leadership skills in managing fatal dysrhythmias: a pilot study.
        Nurse education in practice. 2020; 42102687https://doi.org/10.1016/j.nepr.2019.102687
        • Almarzooq Z.I.
        • Lopes M.
        • Kochar A.
        Virtual learning during the COVID-19 pandemic: a disruptive technology in graduate medical education.
        Journal of the American College of Cardiology. 2020; 75: 2635-2638https://doi.org/10.1016/j.jacc.2020.04.015
        • Almeida R.G.
        • Mazzo A.
        • Martins J.C.
        • Baptista R.C.
        • Girão F.B.
        • Mendes I.A.
        Validation to Portuguese of the scale of student satisfaction and self-confidence in learning.
        Revista latino-americana de enfermagem. 2015; 23: 1007-1013https://doi.org/10.1590/0104-1169.0472.2643
        • Bond K.S.
        • Jorm A.F.
        • Kitchener B.A.
        • Reavley N.J.
        Mental health first aid training for Australian medical and nursing students: an evaluation study.
        BMC psychology. 2015; 3: 11https://doi.org/10.1186/s40359-015-0069-0
        • Borg-Sapiano A.
        • Sammut R.
        • Trapani J.
        The effectiveness of virtual simulation in improving student nurses' knowledge and performance during patient deterioration: a pre and post test design.
        Nurse education today. 2018; 62: 128-133https://doi.org/10.1016/j.nedt.2017.12.025
        • Cant R.P.
        • Cooper S.J.
        Use of simulation-based learning in undergraduate nurse education: an umbrella systematic review.
        Nurse education today. 2017; 49: 63-71https://doi.org/10.1016/j.nedt.2016.11.015
        • Cheng A.
        • Kolbe M.
        • Grant V.
        • Eller S.
        • Hales R.
        • Symon B.
        • et al.
        A practical guide to virtual debriefings: communities of inquiry perspective.
        Advances in simulation (London, England). 2020; 5: 18https://doi.org/10.1186/s41077-020-00141-1
        • Costa R.R.O.
        • Medeiros S.M.
        • Coutinho V.R.D.
        • Mazzo A.
        • Araújo M.S.
        Satisfaction and self-confidence in the learning of nursing students: randomized clinical trial.
        Escola Anna Nery. 2020; 24e20190094https://doi.org/10.1590/2177-9465-ean-2019-0094
        • Foronda C.
        • Budhathoki C.
        • Salani D.
        Use of multiuser, high-fidelity virtual simulation to teach leadership styles to nursing students.
        Nurse educator. 2014; 39: 209-211https://doi.org/10.1097/NNE.0000000000000073
        • Foronda C.L.
        • Swoboda S.M.
        • Hudson K.W.
        • Jones E.
        • Sullivan N.
        • Ockimey J.
        • et al.
        Evaluation of vSIM for Nursing™: a trial of innovation.
        Clinical Simulation in Nursing. 2016; 12: 128-131https://doi.org/10.1016/j.ecns.2015.12.006
        • Foronda C.L.
        • Swoboda S.M.
        • Henry M.N.
        • Kamau E.
        • Sullivan N.
        • Hudson K.W.
        Student preferences and perceptions of learning from vSIM for Nursing™.
        Nurse education in practice. 2018; 33: 27-32https://doi.org/10.1016/j.nepr.2018.08.003
        • Franklin A.E.
        • Burns P.
        • Lee C.S.
        Psychometric testing on the NLN student satisfaction and self-confidence in learning, simulation design scale, and educational practices questionnaire using a sample of pre-licensure novice nurses.
        Nurse education today. 2014; 34: 1298-1304https://doi.org/10.1016/j.nedt.2014.06.011
        • Gu Y.
        • Zou Z.
        • Chen X.
        The effects of vSIM for NursingTM as a teaching strategy on fundamentals of nursing education in undergraduates.
        Clinical Simulation in Nursing. 2017; 13: 194-197https://doi.org/10.1016/j.ecns.2017.01.005
        • Jeffries P.R.
        • Rizzolo M.A.
        Designing and implementing models for the innovative use of simulation to teach nursing care of ill adults and children: a national, multisite, multimethod study.
        in: Jeffries P.R. Simulation in nursing education: From conceptualization to evaluation. National League for Nursing, New York, NY2006: 147-159
        • Kang S.J.
        • Hong C.M.
        • Lee H.
        The impact of virtual simulation on critical thinking and self-directed learning ability of nursing students.
        Clinical Simulation in Nursing. 2020; X: 1-7https://doi.org/10.1016/j.ecns.2020.05.008
        • Liu W.
        The effects of virtual simulation on undergraduate nursing students' mental health literacy: a prospective cohort study.
        Issues in mental health nursing. 2020; (Advance online publication): 1-10https://doi.org/10.1080/01612840.2020.1793248
      1. National League for nursing., 2020. Available in: http://www.nln.org/centers-for-nursing-education/nln-center-for-innovation-in-education-excellence/institute-for-simulation-and-technology/vsim-for-nursing-medical-surgical

        • Newman N.A.
        • Lattouf O.M.
        Coalition for medical education-A call to action: a proposition to adapt clinical medical education to meet the needs of students and other healthcare learners during COVID-19.
        Journal of cardiac surgery. 2020; 35: 1174-1175https://doi.org/10.1111/jocs.14590
        • Redmond C.
        • Hardie P.
        • Davies C.
        • Cornally D.
        • Daly O.
        • O'Sullivan C.
        Increasing competence in wound care: A cross-sectional study to evaluate use of a virtual patient by undergraduate student nurses.
        Nurse education in practice. 2020; 44102774https://doi.org/10.1016/j.nepr.2020.102774
        • Rose S.
        Medical student education in the time of COVID-19.
        JAMA. 2020; 323: 2131-2132https://doi.org/10.1001/jama.2020.5227
        • Schmidt M.
        • Haglund K.
        Debrief in Emergency Departments to Improve Compassion Fatigue and Promote Resiliency.
        Journal of trauma nursing: the official journal of the Society of Trauma Nurses. 2017; 24: 317-322https://doi.org/10.1097/JTN.0000000000000315
        • Springer R.
        • Mah J.
        • Shusdock I.
        • Brautigam R.
        • Donahue S.
        • Butler K.
        Simulation training in critical care: does practice make perfect?.
        Surgery. 2013; 154: 345-350https://doi.org/10.1016/j.surg.2013.04.038
        • Tabatabai S.
        COVID-19 impact and virtual medical education.
        Journal of advances in medical education & professionalism. 2020; 8: 140-143https://doi.org/10.30476/jamp.2020.86070.1213
        • Tabatabai S.
        Simulations and virtual learning supporting clinical education during the COVID 19 pandemic.
        Advances in medical education and practice. 2020; 11: 513-516https://doi.org/10.2147/AMEP.S257750
        • Tabatabai S.
        • Javadi M.A.
        Ophthalmic education and ophthalmologists growth trends in Iran (1979-2016).
        Journal of ophthalmic & vision research. 2019; 14: 185-194https://doi.org/10.4103/jovr.jovr_24_18
        • Tjoflåt I.
        • Brandeggen T.K.
        • Strandberg E.S.
        • Dyrstad D.N.
        • Husebø S.E.
        Norwegian nursing students' evaluation of vSim® for Nursing.
        Advances in simulation (London, England). 2018; 3: 10https://doi.org/10.1186/s41077-018-0070-9
        • Üzen Cura Ş.
        • Kocatepe V.
        • Yıldırım D.
        • Küçükakgün H.
        • Atay S.
        • Ünver V.
        Examining knowledge, skill, stress, satisfaction, and self-confidence levels of nursing students in three different simulation modalities.
        Asian nursing research. 2020; 14: 158-164https://doi.org/10.1016/j.anr.2020.07.001
        • Watari T.
        • Tokuda Y.
        • Owada M.
        • Onigata K.
        The utility of virtual patient simulations for clinical reasoning education.
        International journal of environmental research and public health. 2020; 17: 5325https://doi.org/10.3390/ijerph17155325
        • Wright R.R.
        • Tinnon E.A.
        • Newton R.H.
        Evaluation of vSim for nursing in an adult health nursing course: a multisite pilot study.
        Computers, informatics, nursing: CIN. 2018; 36: 84-89https://doi.org/10.1097/CIN.0000000000000388
        • Yeh T.K.
        • Huang H.M.
        • Chan W.P.
        • Chang C.Y.
        Effects of congruence between preferred and perceived learning environments in nursing education in Taiwan: a cross-sectional study.
        BMJ open. 2016; 6e009925https://doi.org/10.1136/bmjopen-2015-009925
        • Zapko K.A.
        • Ferranto M.
        • Blasiman R.
        • Shelestak D.
        Evaluating best educational practices, student satisfaction, and self-confidence in simulation: a descriptive study.
        Nurse education today. 2018; 60: 28-34https://doi.org/10.1016/j.nedt.2017.09.006